A new thin client

What graphical elements do we need?

Posted by mufasa Sat, 13 Sep 2008 05:41:00 GMT

A new thin client is defined by the GUI widgets that it implements, such as image widgets and text input boxes. I’ve come up with a list of widgets, or elements, that a new thin client should have, starting with those that most are aware of. Obviously, much of this can also be implemented in the current web stack by employing javascript, but that creates new security and performance problems since adding a language runtime like javascript, java, or silverlight makes the web stack a rich client, not a thin client, with the attendant responsiveness, security, and performance problems. I reference the relevant HTML tags where possible.

The old widget standards, you can skim these quickly if you’re at all familiar with software:

1. Text – In HTML, you can place text almost anywhere, as long as it’s not obscured by other widgets. However, it’s probably best to have an explicit text widget for layout reasons, plus for the various fonts and styles.
2. Panel/Frame – <div> <span> <frame> Layout widgets are needed to properly organize other widgets.
3. Grid – <table> A crucial element for any thin client, as data is often best organized in a grid.
4. File chooser – <input type="file"> A widget for choosing locations on the local file system when uploading and downloading files.
5. Image – <img> The widget that really made HTML, all the standard image formats can be supported by each thin client implementation, with plugins to add support for new image formats. If images are often being updated very rapidly, it might make sense to add a quick-update or vector differences capability to allow for some sort of quasi-animation capability, but this may need frame-rate limits for performance reasons.
6. Input elements – <form> I lay out the various input elements below, however there’s no need for a form element as in HTML, because every input element should be able to update to the server independently, as AJAX tries to allow today. There’s also no need for a radio button as in HTML, because it’s almost never used, plus it can probably be implemented by a general button widget if necessary. The only place one sees radio buttons nowadays is in a few webpages, because the option is so readily available in HTML.
   1. Button – <input type="button"> <button> The old workhorse button widget that is crucial to all software.
   2. Checkbox – <input type="checkbox"> Another workhorse that can probably be subsumed into a general button widget.
   3. Text input – <input type="text"> <textarea> A text input widget, with the option to obfuscate input characters for password entry.
   4. Dropdown menu – <select> The way this element is often implemented by browsers doesn’t allow for real nestable, stackable menus, which are widespread in native computer GUIs but often have to be hacked into HTML using CSS or javascript or other means. I would like to see a menu widget that is implemented more like the familiar dropdown menus in native GUIs, rather than the often strange way that browsers implement this widget.

Now we get to the new elements:

1. Audio and video – HTML doesn’t have explicit multimedia tags, but audio and video formats can be included using more general-purpose tags and all the browsers support some grab bag of multimedia, with HTML 5 bringing explicit <audio> and <video> tags. These widgets should have built in player controls that can be overridden by custom player controls if wanted. They should also support different download options, such as buffered streaming and full downloads. The different multimedia formats can be handled the same way as mentioned for image formats, a handful of built-in multimedia formats with plugins for new formats.
2. Links – <a href=> The most basic element and killer app of HTML and really what made it different. There is a clear need for linking text and other widgets, but you probably can’t keep URLs when you don’t have discrete pages like HTML. AJAX has run into this problem already. An app can easily implement local links and generate URLs for outside links, at least until there’s another option, but I don’t believe the existing URL scheme is the best way to do global linking. I’d rather get rid of text URLs and the address bar and replace them with a hyperlink data format, that has a handful of variables to specify links to outside content. I’ll write a post on reworking URLs later, certainly URLs can be reused for now.

And finally a new feature that isn’t really a GUI element:

• GUI logic – In writing this post, I realized that one of the benefits of the <form> element from HTML, that is lost with independently updating widgets, is that it’s likely you’ll now see a bunch of small packets constantly being sent to the server with each widget update. No doubt AJAX, and other hacks to make HTML more dynamic, encounter this problem also. Also, since a real thin client has no language runtime like javascript, one cannot use that outlet to hack in a workaround. Instead I suggest a binary format for downloadable logic maps, so that logic for easily evaluated keyboard and mouse input can be pre-cached and quickly evaluated on the client. I’ll illustrate this with an example: suppose there’s an image of a dog that is linked to an image of a cat. Rather than having the user click on the dog and wait for a round trip server hit and the attendant wait, the server can send both images ahead of time, along with a logic map that says to load the cat image if the user clicks on the dog. By limiting this logic map format’s input to a subset of user input, limiting the computation allowed on this input to simple boolean logic, and limiting the output to clearly specified GUI widget updates, such as disabling, enabling, or updating certain widgets, one avoids the problems of a scripting runtime while still gaining a fair amount of GUI responsiveness and minimizing network traffic. Properly engineering such a logic map format and allowing the server to fine-tune these logic maps to optimize interactivity for each session may represent a great advance over current thin clients. The client could record all mouse and keyboard input and application state during such “offline” phases and upload all of this accumulated data at once, once the currently available logic map is insufficient to determine the next action. All kinds of GUI data can be prefetched according to this model. HTML is inefficient and unresponsive by comparison, and although AJAX mitigates this somewhat, one can avoid the giant attack surface of a VM language like javascript by using such a boolean logic or state machine with carefully limited inputs and outputs.

That is all the required GUI widgets and associated features that I could think of, I hope commenters can fill in any that I’ve missed. All of these widgets would be implemented in a binary format that optimized network traffic further. There’s no need for HTML’s list <li> element, as the grid element can handle that, and no need for object, script, or applet tags, as such general-purpose tags present security and performance problems. However, various implementations might implement additional specific widgets. A z-index property for all widgets can be used to determine stacking order. Also, I think it’s best to focus on set pixel resolutions to begin with, such as a few common mobile display resolutions along with the standard desktop screen resolutions, to avoid layout problems that come with arbitrary window sizes. Eventually one can work in functionality to interpolate layouts between these standard resolutions.

Future features that are worth considering:

• Composing applications with widgets from different servers is ripe with promise. HTML doesn’t really allow for this but it can sort of be hacked in today, for example with blogs that have commenting functionality provided by another server. I think this composition of widgets will be hugely important going forward and any thin, or rich, client technology will have to consider how best to enable it.
• Alpha blending and animations provide for smoother operation and some eye candy but can be put off for later.
• A slidebar and other higher-level widgets may be necessary later.
• Local scripting that allows the user to more finely control their environment will be important. This would be analogous to Greasemonkey rather than the downloaded javascript that most websites use. Also, user-defined hotkeys would help speed interactivity.

Clearly, the devil is in the details of how all this is implemented, but listing what is necessary to implement is the first step. There is nothing revolutionary here, but thin clients have been around for awhile. HTML probably provides 70% of what’s necessary from a thin client; it’s just that the current web stack, with further additions like javascript and flash, is badly engineered. However, a well-engineered thin client can make the user experience much more responsive, capable, and secure with these straightforward technology choices.

Questions:

• What other GUI elements or associated features would you like to see?

Posted in Features | Tags widgets | 2 comments

Building responsive sessions

Posted by mufasa Thu, 31 Jul 2008 07:03:00 GMT

First, what do I mean by sessions? The HTTP request/response model is supposed to be stateless but the vast majority of page requests employ some form of state, whether it’s for advertising or saving application state, implemented by using cookies or javascript. I take it as a given that this type of session state is necessary and would like to make these sessions as responsive as possible. Either because of network or rendering delay caused by the bloat of the current web stack, most sessions on the web are not very responsive. Developers today work around this using various hacks like AJAX or COMET but these often involve bringing in additional problematic technologies like flash or silverlight. A new thin client would take this into account from the beginning by being designed for responsive sessions from the get-go.

First, we have to let go of the page model. Every object on the screen should be able to be updated independently, just as AJAX tries to let the browser do now. This may cause some problems for the URL concept but URLs can be reworked to be used in different ways (I’ll write more about reworking URLs later). Second, we have to allow the connection to be stateful, as that describes most usage of thin clients. Remember, TCP is stateful and most network traffic runs over TCP, so it can be done efficiently. Third, we have to focus on rendering speed on the client and ditch all VM-based languages like javascript, silverlight, and java. Those languages have no place in a thin client anyway, as a true thin client only deploys graphical/view data. Finally, someday this may necessitate deploying new network protocols but for now I’m sure current protocols like TCP, SCTP, and IP should do the job.

We can do so much better at creating a usable thin client experience than we are doing now. Only apathy and ignorance cause people to continue using the outdated tools of the past rather than trying to build better tools and platforms. A new thin client could do this.

Questions:

• How else can we make a responsive thin client?
• Why was request/response used for HTTP/HTML in the first place? Other than the fact that that is how it was originally designed, as it was created for moving large scientific datasets, why didn’t Netscape and others move to a more responsive session model during the evolution of HTTP/HTML in the mid-90’s?

Posted in Features | Tags sessions | 2 comments

Keeping the platform open

Posted by mufasa Mon, 26 May 2008 06:55:00 GMT

In the past, software platforms have often been used to lock developers in. This led to a drive for open standards so that no one company could control the platform and to give all the implementers input into the standard. However, the standards process is largely a relic of hardware standards and doesn’t really work for application software. Standards are best suited to network protocols that are programmed into routers, routers for which the software then stays the same without being updated for years. The modern PC is updated very often, at least 2-3 times a year, so the standards process is a straight-jacket on desktop software. It takes years for the standards bodies to reach consensus on a standard, which once ratified shows the dents of design by committee and then cannot be modified easily. The inadequacy of HTML/HTTP shows the results of this process. I will now suggest a way to keep platforms open, open to developers and competition, without the standardization that chokes off innovation. I believe these concepts should be used for a new thin client but they can be applied to almost any new software platform, such as rich clients, widget toolkits, or other software libraries.

First, let’s look at each of the ways that companies can potentially lock up a thin client platform: APIs, copyright, and patents.

• APIs – Although it appears that there is no protection for APIs, software copyrights are murky legal territory so it’s always better to be safe. The people behind an open thin client platform should not assert any intellectual property rights over the public APIs. Also, any API that is disclosed to one licensee should be disclosed to all licensees so that some developers don’t have access to exclusive APIs. These protections will allow a competitor to reimplement any APIs, just as the Wine guys have reimplemented the win32 APIs (though they better watch out as Microsoft might use this API issue to come after them some day). Of course, just because a competitor is free to reimplement the APIs doesn’t mean that is all there is to it. Open standards have a full specification that lays out much of the design. However, I see no reason to require an open platform to have an open specification, this decision should be left up to the creators. Outside developers can often figure out how the original implementation works and reimplement it, provided the original license does not prohibit doing so. Often this is the best way to reimplement software, as almost no specification is so precise that it lays out every detail of the implementation accurately. If there is an external API, a developer should be able to figure out how it works from the software, so there should be no licensing prohibition against using the original implementation to gather knowledge necessary for reimplementation. A thin client will have a compact set of documented public APIs so it should be easy to reimplement.

• Copyright – This is the main way in which software is currently protected and copyright is necessary to both closed and most open source software and business models. While open APIs allow developers to build on top of a software platform with the assurance that they’re not locked in, they do not allow the developer to modify the thin client binary itself. I see no reason to add any requirements about this however, as it should be up to the people who develop the platform as to how much control of the software binary itself that they want to give to third-party developers. Some will allow plugins and other modifications, others won’t. This is often a security issue and there are a range of possibilities: it should not be necessary to restrict the developer of the platform in this regard. I suspect that most software in the future will be a mixture of open and closed source- rather than exclusively one or the other as most of it is now- so I don’t think exclusive, monolithic control of a software platform through copyright will be much of an issue in the future.

• Patents – Let me start off by saying that most software patents are a joke and are only possible because the current patent system is a complete debacle. However, as people use software patents anyway and a few have some justification, I’d like to account for them. What’s important for a software patent is that all patents on an open platform must be disclosed. Also, no disclosed patent can restrict someone from reimplementing a feature of the platform because it is the only way to implement it. So for example, if someone were to patent quicksort, you could always use bubble sort so that’d be okay. However, if someone were to patent something ridiculous like Akamai’s distributed hosting patent, there would be no way to implement something similar so it wouldn’t be kosher.

Taken together, these restrictions can make up an Open Platform Promise, or OPP, consisting of the following:

1. No intellectual property rights will be asserted on any publicly disclosed APIs.
2. Any APIs that are disclosed to one developer will be disclosed to all developers.
3. No restrictions will be made on usage of the software to help reimplementation.
4. All patents will be disclosed.
5. No patent can preclude reimplementation of a feature of the platform.

Some may wonder why all this discussion of intellectual property is even necessary. My argument is that software platforms can be extremely widely deployed- a thin client platform in particular- so it is important to consider issues of lock-in and monopoly. Most current software platforms are either at one extreme of closed-source, patented software, exemplified by Microsoft and their win32 APIs, or the other extreme of open standards often implemented by open-source toolkits, exemplified by HTTP/HTML. The goal of the preceding is to fashion a melding of the two extremes that works better than either one does individually. I do not address the issue of open- or closed-source as I believe that most software will end up being a mix of the two. Instead, I address the other API and IP issues that go with keeping a platform open. The goal of the OPP is to allow developers of a platform to create something they can own and improve upon while allowing third-party developers the freedom to choose their platform and its implementation.

Questions:

• I’m no intellectual property lawyer so I’m going to need more info from those who’ve dealt with this stuff in more detail. What other provisions do software companies use to lock developers into a software platform?

Posted in Open platform | Tags apis, copyright, patents | 415 comments

We need a new thin client

Posted by mufasa Thu, 15 May 2008 02:32:00 GMT

We need to replace HTML. HTTP/HTML is an almost two decade-old prototype that has kept its basic form while many buggy and slow extensions have been mashed on top of it. It is now a hopelessly inefficient, quasi-rich client that does almost nothing well while maintaining a legacy language, HTML, that doesn’t do much and that nobody cares for. Worse, as an “open standard,” there are multiple incompatible implementations, either because the spec is vague or the implementers want to differentiate themselves. I would like to throw all this garbage out and start afresh with a new thin client. If we can get this right, maybe then we can start thinking about a rich client. Here is what I would like to see:

• Better GUI elements – The UI elements of HTML may have been appropriate to 1990 but have become dated and limiting. Why do we make designers come up with all kinds of hacks for basic elements like menus or proper spacing? A good thin client should have a limited subset of GUI elements built into the specification, those that are the most popular and easiest to secure and implement. HTML comes nowhere close to fulfilling this criteria.
• Sessions – HTTP is based on a request/response model because of the bandwidth limitations of the late 80s that no longer exist. Today, we trick the browser into a session model using AJAX or COMET or Name Your Acronym. Why not build a thin client that uses sessions as its atomic unit?
• Binary encoding – It makes sense for a research prototype like HTML to use text to encode GUI elements but why are we still using text to this day? When a widely distributed thin client employs inefficient and useless methods, it just leads to a lot of waste. Almost nobody edits HTML at the text level so there is no use for markup.
• A GUI for design, not code – No designer wants to muck around in code in order to get their GUI looking just right. A new thin client should allow designers to use a graphic design tool to design their application’s views and get them looking just right, before the views are compiled to a binary encoding.
• No standards process – The internet standards process has not helped application protocols like HTML. Rather, it has led to multiple implementations that each has their own quirks. Part of this is because of the vagueness of important sections of open standards, perhaps it is impossible to precisely detail any reasonably complex technology. Part of it is because every implementer tries to differentiate themselves by adding special features that are incompatible with the other implementations. What is important for a software platform is that it is open to reimplementation and the resultant competition, allowing developers to always have the choice of moving to another implementation, not that it conforms to some preconceived standard.

The essence of a thin client is to implement the most common GUI elements across multiple hardware and OS platforms while maintaining security. A thin client can be thought of as a first stab at ubiquitous network computing, where the security hassles of having the information superhighway coming into your PC- and the internet bandits and thieves who can now drive right up to your computer door- are handled by the thin client platform so that developers don’t have to. We can do much better at this than we are doing now. And finally, think of the children. If we don’t come up with a new thin client, what will we leave them?

I would like to hear more ideas on this subject from others and I will expand on each of the topics above in more detail. This blog is a place for discussion and sharing ideas, I would love to hear what others who also hold their nose at the current HTML/AJAX mess think.

Posted in Overview | no comments