A new thin client

What graphical elements do we need?

Posted by mufasa Sat, 13 Sep 2008 05:41:00 GMT

A new thin client is defined by the GUI widgets that it implements, such as image widgets and text input boxes. I’ve come up with a list of widgets, or elements, that a new thin client should have, starting with those that most are aware of. Obviously, much of this can also be implemented in the current web stack by employing javascript, but that creates new security and performance problems since adding a language runtime like javascript, java, or silverlight makes the web stack a rich client, not a thin client, with the attendant responsiveness, security, and performance problems. I reference the relevant HTML tags where possible.

The old widget standards, you can skim these quickly if you’re at all familiar with software:

1. Text – In HTML, you can place text almost anywhere, as long as it’s not obscured by other widgets. However, it’s probably best to have an explicit text widget for layout reasons, plus for the various fonts and styles.
2. Panel/Frame – <div> <span> <frame> Layout widgets are needed to properly organize other widgets.
3. Grid – <table> A crucial element for any thin client, as data is often best organized in a grid.
4. File chooser – <input type="file"> A widget for choosing locations on the local file system when uploading and downloading files.
5. Image – <img> The widget that really made HTML, all the standard image formats can be supported by each thin client implementation, with plugins to add support for new image formats. If images are often being updated very rapidly, it might make sense to add a quick-update or vector differences capability to allow for some sort of quasi-animation capability, but this may need frame-rate limits for performance reasons.
6. Input elements – <form> I lay out the various input elements below, however there’s no need for a form element as in HTML, because every input element should be able to update to the server independently, as AJAX tries to allow today. There’s also no need for a radio button as in HTML, because it’s almost never used, plus it can probably be implemented by a general button widget if necessary. The only place one sees radio buttons nowadays is in a few webpages, because the option is so readily available in HTML.
   1. Button – <input type="button"> <button> The old workhorse button widget that is crucial to all software.
   2. Checkbox – <input type="checkbox"> Another workhorse that can probably be subsumed into a general button widget.
   3. Text input – <input type="text"> <textarea> A text input widget, with the option to obfuscate input characters for password entry.
   4. Dropdown menu – <select> The way this element is often implemented by browsers doesn’t allow for real nestable, stackable menus, which are widespread in native computer GUIs but often have to be hacked into HTML using CSS or javascript or other means. I would like to see a menu widget that is implemented more like the familiar dropdown menus in native GUIs, rather than the often strange way that browsers implement this widget.

Now we get to the new elements:

1. Audio and video – HTML doesn’t have explicit multimedia tags, but audio and video formats can be included using more general-purpose tags and all the browsers support some grab bag of multimedia, with HTML 5 bringing explicit <audio> and <video> tags. These widgets should have built in player controls that can be overridden by custom player controls if wanted. They should also support different download options, such as buffered streaming and full downloads. The different multimedia formats can be handled the same way as mentioned for image formats, a handful of built-in multimedia formats with plugins for new formats.
2. Links – <a href=> The most basic element and killer app of HTML and really what made it different. There is a clear need for linking text and other widgets, but you probably can’t keep URLs when you don’t have discrete pages like HTML. AJAX has run into this problem already. An app can easily implement local links and generate URLs for outside links, at least until there’s another option, but I don’t believe the existing URL scheme is the best way to do global linking. I’d rather get rid of text URLs and the address bar and replace them with a hyperlink data format, that has a handful of variables to specify links to outside content. I’ll write a post on reworking URLs later, certainly URLs can be reused for now.

And finally a new feature that isn’t really a GUI element:

• GUI logic – In writing this post, I realized that one of the benefits of the <form> element from HTML, that is lost with independently updating widgets, is that it’s likely you’ll now see a bunch of small packets constantly being sent to the server with each widget update. No doubt AJAX, and other hacks to make HTML more dynamic, encounter this problem also. Also, since a real thin client has no language runtime like javascript, one cannot use that outlet to hack in a workaround. Instead I suggest a binary format for downloadable logic maps, so that logic for easily evaluated keyboard and mouse input can be pre-cached and quickly evaluated on the client. I’ll illustrate this with an example: suppose there’s an image of a dog that is linked to an image of a cat. Rather than having the user click on the dog and wait for a round trip server hit and the attendant wait, the server can send both images ahead of time, along with a logic map that says to load the cat image if the user clicks on the dog. By limiting this logic map format’s input to a subset of user input, limiting the computation allowed on this input to simple boolean logic, and limiting the output to clearly specified GUI widget updates, such as disabling, enabling, or updating certain widgets, one avoids the problems of a scripting runtime while still gaining a fair amount of GUI responsiveness and minimizing network traffic. Properly engineering such a logic map format and allowing the server to fine-tune these logic maps to optimize interactivity for each session may represent a great advance over current thin clients. The client could record all mouse and keyboard input and application state during such “offline” phases and upload all of this accumulated data at once, once the currently available logic map is insufficient to determine the next action. All kinds of GUI data can be prefetched according to this model. HTML is inefficient and unresponsive by comparison, and although AJAX mitigates this somewhat, one can avoid the giant attack surface of a VM language like javascript by using such a boolean logic or state machine with carefully limited inputs and outputs.

That is all the required GUI widgets and associated features that I could think of, I hope commenters can fill in any that I’ve missed. All of these widgets would be implemented in a binary format that optimized network traffic further. There’s no need for HTML’s list <li> element, as the grid element can handle that, and no need for object, script, or applet tags, as such general-purpose tags present security and performance problems. However, various implementations might implement additional specific widgets. A z-index property for all widgets can be used to determine stacking order. Also, I think it’s best to focus on set pixel resolutions to begin with, such as a few common mobile display resolutions along with the standard desktop screen resolutions, to avoid layout problems that come with arbitrary window sizes. Eventually one can work in functionality to interpolate layouts between these standard resolutions.

Future features that are worth considering:

• Composing applications with widgets from different servers is ripe with promise. HTML doesn’t really allow for this but it can sort of be hacked in today, for example with blogs that have commenting functionality provided by another server. I think this composition of widgets will be hugely important going forward and any thin, or rich, client technology will have to consider how best to enable it.
• Alpha blending and animations provide for smoother operation and some eye candy but can be put off for later.
• A slidebar and other higher-level widgets may be necessary later.
• Local scripting that allows the user to more finely control their environment will be important. This would be analogous to Greasemonkey rather than the downloaded javascript that most websites use. Also, user-defined hotkeys would help speed interactivity.

Clearly, the devil is in the details of how all this is implemented, but listing what is necessary to implement is the first step. There is nothing revolutionary here, but thin clients have been around for awhile. HTML probably provides 70% of what’s necessary from a thin client; it’s just that the current web stack, with further additions like javascript and flash, is badly engineered. However, a well-engineered thin client can make the user experience much more responsive, capable, and secure with these straightforward technology choices.

Questions:

• What other GUI elements or associated features would you like to see?

Posted in Features | Tags widgets | 10 comments