WAP and mobile application development

Some of you may be using our Mobile Movie Times application or seen the demos of our various other mobile consumer apps. You may say “that sure looks a lot better than WAP”. Yes, all of our apps are locally executing rich clients optimized for their devices. Many of you may have noticed that most consumer-focused large web properties are pursuing (google, yahoo, ebay) are also not building WAP portals but instead focusing on locally executing optimized apps targeting a particular platform. So why is this? WAP’s been around for years. What’s wrong with it?The big problems with WAP that we see are no ability to execute functionality locally, lack of separation of presentation and data, a murky future standards picture, no ability to work with data offline, and no ability to take advantage of an increasingly diverse set of mobile device capabilities.For the record, there are several applications and use cases where none of these are problems, especially one way delivery of text only information. It may be that in an effort to allow an application to run on as many platforms as possible without rewriting them as native locally executing apps that use of WAP on some devices can be a way to multiply the payoff of investing in building the backend of a mobile application. That said, for most applications the following issues all inhibit WAP from delivering a truly optimal user experience on targeted devices<h3>Executing Logic Locally</h3>
The best applications (mobile, web-based and desktop) all execute some portion of their logic locally. This can be for:

• validation
• modifying forms, menus and screen display based on user behavior
• rapidly showing different data based on user actions and interests without roundtrips to the server

These ideas are behind much of the current trend of Ajax-oriented websites: highly responsive, interactive, adaptive web pages and user interfaces.

WAP sites don’t do this and they don’t have enough of a scripting model to enable these features. AJAX-capable browsers can try to do these things (the pros and cons of mobile Ajax browsers are the subject of another post). But most web and mobile app providers trying to create optimal user experiences are building “native applications” in the APIs of the mobile platform to achieve these benefits.

Separating Presentation and Data

Neither WAP nor traditional HTML pages separate presentation from data. Form controls and their embedded data are tied together in one stream of information. This is why most WAP sites that interact with users are painfully slow. By contrast, optimized native applications run locally on the device and send only the changed data back and forth. It is possible to approximate these benefits with Ajax and other use of Javascript on a mobile browser, but it is inherently more difficult and the data exchange less efficient (again, we’ll talk more about Ajax browsers later).

Future Standards

At one time it seemed like the mobile web industry might come together on some standards based on some derivative of both WAP and XHTML Mobile Profile. That effort ended with major vendors each taking their own tack on the mobile web using divergent subsets of a proposed WAP 2.0 standard. We’ll talk more about that later. But at this point, it is not a controversial statement that WAP 2.0 and XHTML MP are not being used consistently and robustly by major mobile software providers and device manufacturers.

Working with Data Offline

Mobile networks are still not always available with 100% coverage and this is likely to remain true for quite a while. The best mobile applications (such as Blackberry’s email) are optimized for syncing intermittently and transparently and allowing the user to use the application offline. The best usage experience even for “vertical” single purpose apps are written to the native device operating system, making optimal use of available storage, trickle syncing data to and from the server transparently to the user, and allowing the user to be as interactive as possible when not having coverage. WAP browsers (and Ajax browsers) are not capable of this.

Taking Advantage of Device Capabilities

Mobile phones are a diverse lot and, with the emergence of new “more than just voice call” features, becoming moreso by the day. There are several major device operating systems: BREW, Symbian, J2ME, Linux, Windows Mobile to name a few. More importantly phones have an increasingly differentiated set of capabilities: everything from taking pictures to recording voice to playing music to syncing contacts and PIM information to voice dialing to GPS. The best applications take full and optimal advantage each device’s capabilities in all of those areas and more. Browser based applications (of any flavor) just don’t do this. To do this requires writing to the APIs of the device, even when a device uses supposedly “open API” such as JSR extensions to J2ME for specific capabilities.

So How Do We Build Such Optimized Native Apps Efficiently?

All this said, its quite expensive and difficult to write application to these various devices in their native operating systems with their unique APIs in C, C++, or Java. And it becomes moreso once you want to target multiple devices. J2ME offers a variety of devices, but certainly not the majority of the market.

But this hasn’t daunted the the biggest web properties(such as Yahoo and Google) with extremely horizontal apps such as email and maps that can be targeted to many millions of users. The cost of hiring large teams of developers to spend months to years writing apps can pay off in these situations.

But is it possible to write apps at a higher level than the native procedural languages (maybe even higher level than WAP or HTML)? And can we do so and still allow single applications to be executed on a wide variety of other devices? And perhaps even better allow the apps to function at a gracefully degraded level by delivering content WAP and AJAX browsers on platforms and devices that its not economical to devote a lot of attention (while still just “writing them once”, as is referred to in another post here). All this is a topic for another post some time soon.

model oh model, where is my model?

One of the strengths of Rails and ActiveRecord is that it makes it easy to build your models. You Don’t Repeat Yourself, with your model attributes being derived directly from your database tables. Then all you have in your model definition itself is a discussion of its relationship to other models, relationship to tables and

There are problems introduced by this power however. When writing your application, where is your model definition? Your attributes are generally all in your table definitions. Your models relationship to other models is in your model definition file. Your validations are often in your model, unless your database is performing some form of validation (a practice that is generally encouraged outside the scope of Rails). So they are usually in both places! Plus there’s a good chance in your Rails application that you have introduced at least one or two views to handle things that Rails needs (such as the need for a single column primary key). So now there’s a third place for your model attributes definition.

Rails introduces another way of handling database definition and versioning: rake migration scripts. These are very powerful ways to version and maintain database schemas and data in a product neutral way. But they also spread the definition of “what is the model” out even further. I talk to Rails developers who say “I never look at my database administration tool – I just incrementally edit my Rake migrations”. The problem is that distributes the model definition out even further! It is spread amongst multiple rake migrate scripts.

Many developers used to programming with other tools and environments are also accustomed to some form of graphical frontend to define and manage their database tables. This can be at the “physical level” of tables or at a more conceptual level of entities and relationship (with tools such as ERWin or Visio). Such environments not only ease understanding and ease of manipulation and modify database (and model) relationships but they are also much better at concentrating the model definition into one place.

So here’s my suggestion: An incredibly useful improvement to the Rails app development toolset would be a graphical “model and rake migration designer” frontend. Probably an Eclipse plugin that plays nicely with RadRails (part of RadRails in the future?).

This frontend would allow you to define models, graphically draw relationships between models choosing from amongst the Rails palette of possible model relationships, and generate rake migration scripts from model attribute definitions. It would generate increment rake migrations when model changes were made. It would also allow you to set validations on attributes and models that would be reflected in the model definitions themselves. It would generate migration scripts for join tables when called for by as has_and_belongs_to_many relationship. There are a lot of other features I’d like to see in this tool, but I’ll leave off for now. No, I’m not considering building this. But I and many other Rails developers I know would be delighted by having this tool to use.

Rails, REST and SOAP web services

I’ve spent a lot of my career in the web services world (before it was even called web services). So a lot of my colleagues often ask me “you like Rails?! how is that possible, the Rails community hates SOAP-based web services?”. I’ve often wondered about this myself. More specifically I’ve wondered at the seemingly religious antipathy of the Rails community (including the illustrious DHH himself) to the “WS-Deathstar”.So (primarily for the people who are confused by a web services’ zealots embrace of Rails) here’s my take on the matter: REST is great for many problems and applications. The first class support that DHH is providing for REST in the Rails core is very useful. I love the scaffold_resource generator and what it provides (as I’ve mentioned earlier here). I also like the approach to providing it: the idea of single controllers that respond to different clients (HTML, REST, and others) with different content. I do have reservations about the need to put wordy respond_to clauses in each controller action (I’ve proposed what I think is a simpler way here). Anyway, automatic support for REST is a good thing. REST is a great way to build simple point to point distributed app to app connectivity. And I agree that, for someone who wants to be aware of the innards of the code they are working with that it is simpler. I also agree that there are scenarios where a REST interface just makes it easier for multiple arbitrary clients on more diverse platforms and languages (there is perhaps stil a small chance that it may be difficult to integrate with a client SOAP stack on a device, but there’s basically no chance that it will be hard to invoke a REST interface from another platform).

But does that mean that SOAP is bad? In the simple app to app connectivity scenario my opinion is that it basically doesn’t matter. In the vast majority of languages and platforms the work of consuming a SOAP-based web service is just as easy as consuming a REST service. But I agree that there’s little tangible advantage to using SOAP if all you ever need is a distributed method call.

The value of the SOAPbased WS-Deathstar is when more complex connectivity is required: multihop message routing, guaranteed delivery, publish-subscribe one-to-many integration, more advanced security in the plumbing than https (the latter probably only necessary when you’re routing messages). If this is never going to be necessary (probably true for the typical consumer-facing website), then the whole SOAP vs. REST debate is moot. And going with the default easier path of REST doesn’t hurt one whit. For more advanced distributed connectivity problems (endemic to many “enterprise applications” that are seemingly anathema to the Rails community), robust SOAP stack with extensive WS-Deathstar support would be useful for developers building complex connectivity. And yes I mean a whole bunch of those nasty evil WS-* specs, including those not yet widely implemented. My top requests include WS-Addressing, WS-Eventing, and full WS-Security. Doing these would also obviate the need to provide other APIs for such such services as publish/subscribe message routing. Due to Ruby’s stunning productivity advantages I think any and all of these would be trivial to implement. I remember implementing a very early WS-Eventing client prototype in C# in about a page of code. It would be much easier in Ruby (if there’s demand for it, I’d consider rewriting it, if only to demonstrate that a seemingly complex spec can in fact be made easy to use).

Are the WS-* overly complex? Yes, I think many of them are. There are plenty of “design by committee” artifacts in there. For the most part such unnecessary complexity can be hidden from the programmer using them. A good API with reasonable default behavior (convention over configuration) covers a thousand design sins. And the layered nature means that you can just pick and choose what facilities help you. Embracing the Deathstar specs would make Ruby a more powerful tool for a wider set of applications. The availability of Rails plugins (or just Ruby libraries) for these capabilities would make it more likely that Ruby gets used for a wider variety of programming tasks which can only help to grow and advance the Ruby and Rails community.

consuming web services from your Rails app

Despite the fact that SOAP-based web services seem to be considered politically incorrect by the Rails community, it is amazingly easy to expose web services from Rails. Ironically enough, it can be a little tricky to consume web services from Rails. Several of my colleagues have asked me to post this. This is mostly due to the strange web services client implementation in Rails (Action Web Services’ client). Essentially, you should just use the native Ruby web services client capability (SOAP::WSDLDriverFactory).Specifically, if you use the guidelines below it should be easier.

• Don’t try to use the Action Web Services client. It only works with AWS servers.
• Don’t use WSDL2Ruby.rb. Its buggy and not complete.
• DO use the native Ruby SOAP services such as in the following code (the example points to a free sample Trivia web service)require ‘soap/wsdlDriver’ triviaEndpoint=”http://mobquiz.com/question/api”; triviaWsdl=wsendpoint+”?wsdl”service=SOAP::WSDLDriverFactory.new(smswsdl).create_rpc_driver question=service.askQuestion(‘Entertainment’) puts question.question