A few weeks ago i have mentioned the Sage project, and it’s time for a deeper look into the project itself and its creator, Don DeCoteau.

Tell us a little bit about yourself.

I live in San Diego. I used to work for a company called SAIC. In my early days at SAIC I wrote the SAIC-HTTP  web server and ran a technology lab (best job ever). From there I moved on to architecture and technology strategy consulting (still at SAIC) and only programmed as a hobby. That job turned into mostly writing white papers and doing studies and lots and lots of traveling. I got bored with it and quit (in mid 2002) to take a sabbatical and reboot. 3 months in I was bored and started programming again to keep myself busy. I ended up selling the technology I developed to a startup. In developing that technology I had to build a Swing-based IDE and it was very painful (I generally prefer writing servers and hate writing UIs). So, for my next pet project, I decided to implement an idea I had back in 1996; mainly taking what was great about web-based apps (URL-centric, markup-based, scriptable), bringing that to the desktop, and most of all, making it as painless as possible for people to use (couldn’t do the scriptable part back then).

What is your “elevator pitch” for Sage?

Sage is a special purpose browser designed to render applications instead of documents. It lets you easily deploy web-enabled desktop apps that can run online, offline, or both. The online capabilities offer the same flexibility and dynamic customization capabilities of traditional web apps (customizable per-user, per-device, per-company, based on user-bandwidth, etc.) without sacrificing the power and convenience of a desktop application.

Traditional browsers use HTML to describe documents and JavaScript to make those documents interactive. Instead of HTML, Sage uses a special CSS-like markup language to describe the application, actions, and interactions. It supports JavaScript, Ruby, Python and a number of other scripting languages for controlling the application and enabling complex interactions. JavaScript is the default scripting language and comes embedded with the engine (via Rhino and BSF 3). Additionally, Sage’s functionality can be augmented via dynamically downloaded java libraries or OS specific shared libraries (the same way you can download browser plug-ins and ActiveX components).

Why another library? Is it your own itch to scratch, or are you addressing the deficiencies of the existing frameworks?

Sage is not a library. It is actually a headless browser. You give sage a URL and it generates an application based on the MIME type of the URL. If you give it a URL to an image, it will launch an image viewer to render the image. For a plain text document it will launch a document viewer. For an html document it will launch a window with a JEditorPane (the document viewer), or an embedded version of Internet Explorer (via JDICPlus), or it will spawn the system browser to view the document (the decision is based on the OS and the engine’s configuration). Sage has a special markup language (SDF) and associated MIME type (text/x-sdf) that lets it render Swing-based applications, the same way a web browser uses HTML to render documents. The demo applications are just a bunch of markup files in conjunction with JavaScript files. The IDE is actually a Sage app and a large portion of the code is written in JavaScript. Java webstart is used as a mechanism for delivering the Sage engine and any supplemental jars that may be required for the demo. The applicationURL parameter in the JNLP file is what determines the application that will be presented to the user. This URL is essentially the first parameter that you would pass to the engine if you were invoking it manually. Sage supports any standard URL (that java supports) as well as some special Sage URLs (like lib: which is URL that points to a file in the JVM’s class path as opposed to jar: which points to a specific jar file). For performance and bandwidth reasons most of my demos are packaged in jars and use a lib: URL. The iTunes Clone demo however, runs directly from the web server in order to demonstrate the deferred image loading capability (you see the music icon prior the artwork being loaded) as well as changing the application on the file as I make enhancements.

It looks like the developer-facing APIs are intentionally targeting the wider audiences familiar with CSS, JavaScript, Python and other scripting languages. Is this a personal preference, or an inclination to follow the market trends?

I am not a big trend follower (which is why I wrote my own markup language instead of going the XML route. I do support XML and have an XML schema defined for my configuration objects but it is an undocumented feature and comes for free as a byproduct of a tool I have). The reason for the scripting languages and markup is because Sage is a runtime and everything is done in real-time. The markup is interpreted at runtime to create the actual application.

Have you looked at JavaFX script? Any intentions to support it in Sage?

I have looked at JavaFX and I am excited about the underlying technology but I think the scripting language is DOA. Sage supports any JSR-223 scripting language so JavaFX script is inherently supported (so far the only languages other that JavaScript that I have tested are Ruby and Python, I made a request to the JRuby guys to create a smaller footprint version of JRuby for client-side use). What I plan on doing in the future (and what I think Sun should do) is creating some high-level scriptable objects that encapsulate some common high-value JavaFX functionality. These objects will be available to any scripting language.

What are your biggest Swing pain points, and what can be done to address them?

Some of my biggest pain points are:

• Inability to completely customize the painting of a component without overriding the component. There is lot of time and effort being spent to get around this limitation (decorators and such). I had to override every component mainly in order to allow the painting behavior to be changed dynamically while still allowing the use of third party L&Fs.
• The lack of consistency in behavior and meaning. For instance the inconsistency in the behavior of setOpaque(). This one little thing has caused me a lot of grief.
• Inability to specify the component that should receive focus when a JPopupMenu goes away (one of the things I can’t fix).
• Inability of finding the triggering event for a focus change (mouse, keyboard, other).

Why did you choose Swing to implement the runtime engine?

I chose Swing because it was pure java. Even though it has a lot of issues and pitfalls, I know I can generally fix or work around them to suit my needs by subclassing or wholesale replacement. Also, I am more interested in usability and looks than in achieving native fidelity. I love the way iTunes looks on my PC and it looks nothing like my other PC apps but is does put the scrollbar buttons in the location that I am accustomed to (on the PC).

Sage provides a Widget Explorer and an IDE to edit and inspect Sage applications. What are your thoughts on visual designers in general? Do they make the initial learning simpler? Do they discourage the understanding of the underlying layers?

I think IDEs lessens the learning curve. People who are not inclined to dig deeper would just find something else that is easy for them (but not necessarily better). I myself use the Sage IDE in combination with a plug-in text editor (for NetBeans) for doing Sage development. Sometimes it is just faster and easier to use a text editor. The curious people will dig into the examples and source code to see how things are done, and use the IDE to save time doing the simple repetitive stuff. Plus the IDE cuts down on questions. I my case, the IDE also serves as a demonstration of the capabilities of Sage and on how to use Sage to build more complex applications. The IDE exploits most of the more advanced Sage features.

Given full control, would you change anything in Swing core layer going forward?

I would:

• Provide a mechanism to completely change the painting of a component without having to override the component. In Sage I have the concept of a paint primer/finisher. The primer prepares the paint surface and returns the graphics object to paint on (the primer is passed to original Swing graphics object). If the primer returns a null surface then no further painting action is taken by the component. The finisher is passed the original Swing graphics to finish the painting. Sage animations, special effects and transitions are implemented as paint primer/finishers that insert themselves into the paint stream. I would love to see something like this added to swing.
• Add the concept of an application context to swing, or at least expose the app context facility that is currently provided for applets and provide an API for retrieving that context and storing context specific state. This would allow for the building of Swing-based window managers for embedded-devices or even desktops. Currently I am doing my own context management but it is not as robust as it could be. It is currently used by the IDE to provide fast previewing (the preview function in the IDE creates a new Sage ApplicationContext and runs the preview within the same JVM as the IDE).
• Add a mechanism to determine the trigger event for a focus change

There is a lot of exploration in RIA space from both major and minor players. Cutting the marketing and hype, what should the developers pay attention to as far as the trends go?

What I personally believe, and what drove me to write sage, is that we are starting to hit the browser wall as far as building highly functional applications. You can build some beautiful web apps, but flash does not correspond to functionality. No matter the web app, you can build a better desktop version simply due to the limitations of the browser and the constraints that they have to work within. For example, most people are not willing to give up their desktop mail client for web version (and there are some really nice web versions). However, desktop apps have traditionally been inflexible, non-portable (tied to a specific device), and difficult to write. The next evolution will be to take what is great about desktop apps and what is great about web apps and bring them together while drastically simplifying the creation of such apps. Mozilla tried this with XULRunner but their implementation was lacking (cumbersome, not easy, required an install, and only addressed the UI). The only commercial company that seems to be going down this path is Adobe. I am always looking at new offerings to see if any is doing anything close to what I am doing. Adobe AIR is the closest (this it to is lacking incapability but I am biased). However, they cannot compete with Java or .Net as far as ecosystem, tools, and functionality. It is much easier to integrate Flash into a Swing or .Net app than it is to integrate a Swing or .Net app into AIR/Flash.

Anything else that you would like to add?

Already added liberally to my other answers :)
However, the one thing I must note that makes Sage significantly different for AIR and XULRunner is that Sage doesn’t just let you markup the UI, it also allows you to markup data. Being able to markup data is what truly makes an application dynamic. This is the main capability of HTML browsers that make them great. When you get data from a database you take that data and markup it up with HTML in order to create the appropriate visual representation. The visual representation can change dynamically based on any number of conditions. Sage allows you to do the same thing with data send to any of its list type widgets.

Thanks Don, and to finish the interview i would like to showcase the capabilities of Sage to create modern and visually rich applications. The latest Sage demo is an iTunes clone. You can view the screencast, browse the complete sources and launch the WebStart app.

The first screenshot shows the grid view of the available albums (click to see fullsize view), along with the navigation tree and the media controls:

The second screenshot shows the coverflow view of the albums, along with the available tracks (click to see fullsize view):

And if you wondered, this is the live WebStart demo running on my Windows Vista machine.

Here are some Swing links that you might have missed during the last week:

• Karra Dakshinamurthy has announced release 2.0 of Marathon GUI test tool. It allows to play and record scripts against a Java Swing UI application, and new in this release is support for scripts in JRuby and Jython.
• Kyle Cordes writes about writing a Swing application using JavaScript and Rhino scripting engine.
• David Gilbert showcases integration of JFreeChart library with a custom magnifying glass effect based on the JXLayer library.
• Mauro Carniel overviews the tools available for developing Swing applications, including client-server tier architectures, server communication options, data binding and component suites. The last part is somewhat biased towards the OpenSwing suite, which is not very surprising given that Mauro is its lead developer.
• Clemens Eisserer continues his award-winning ;) work on the XRender pipeline for Java2D, and this week he’s using the LightBeam testbed for testing the performance improvements on the TransformedBlit part of the pipeline.
• Daniel Wilson has announced the first release of standalone Swing PDF viewer from the Big Faceless Organization (BFO). It is a commercial component that supports printing, saving, search/selection, forms, bookmarks, reorderable thumbnails, annotations. It has full support for PDFs up to Acrobat 9 and can be controlled from JavaScript.
• And finally, Jean Francois Poilpret provides an update on his “takeover” of the DesignGridLayout project, fixing the last outstanding high priority bug. He is seeking feedback on the release schedule for the official “big” 1.0 release, contemplating whether it is worth to postpone it to make room for a few enhancement requests.

Even before Microsoft has chosen the ribbon component as the emerging trend in its desktop applications, it has spent significant resources not only on the design itself, but also on the painstakingly elaborate user interface design guidelines. The complete document for “Office Fluent UI” weighs in as a 119-page PDF, and it looks like much of the specification is going to find its way into the main MSDN site. There are numerous commercial offerings of compliant implementations available for .NET, Delphi, and even Silverlight developers, and the companies behind these offerings make sure to provide the checklist of features that are enumerated in the official guidelines.

The ribbon component is still being met with a lot of criticism from the developers (and on a personal note, my wife disliked it so much that she reverted to Office 2003 even though i had two more installations). Realistically speaking, though, Microsoft is here to stay, at least in the near future, and while the blogosphere may pat itself on the back for “burying” Vista (am i the only one that didn’t have any problems worthy of mentioning over the last 20 months, or is it just the silent majority that couldn’t care less about “Vista is teh fail” hoopla), it is going to continue setting the de-facto standards for the majority of user-facing applications.

According to Microsoft, one of the main reasons behind the rather strict design guidelines for the ribbon component is that once the user has a firm grasp on the ribbon concepts (presumably from Office 2007), the learning curve for other ribbon-based applications will be much less steeper. While most of the UI concepts behind the ribbon can be found in older applications, most of the users find themselves at odds with a large variety of the new ways to interact with Office – the application menu button, the quick access toolbar, contextual tabs, mini toolbar, galleries and automatic resizing. A consistent user-facing behavior of these features is a key element to well spread user adoption of ribbon-based applications, and may be one of the reasons why Microsoft has invested so many resources in both creating the UI guidelines and pushing the concept down to the bundled Windows applications.

A few weeks ago a chance had me stumbling across AutoCAD 2009 (whereby my lovely wife was faced once again with the ribbon). Autodesk is the world’s largest design software company, and its flagship AutoCAD software line is ubiquitous in the engineering industry. Going back to 1982 (predating even Photoshop), it is estimated to have more than a billion active files. If you think about applications with a large user base, AutoCAD would certainly be on that list. And so it is even more surprising (at least to me) that the latest AutoCAD 2009 release is using the ribbon component as the default UI.

Matt Stein is a senior product designer working in Autodesk and the two-part interview (part 1, part 2) published in June 2008 is a must read for the design teams considering switching their products to use the ribbon. In the beginning Matt mentions the same “feature bloat” problem that has made AutoCAD prohibitively complicated for novice users. Over a thousand commands, random floating toolbars and pallettes in user-submitted configuration screens, stacks of docked vertical and horizontal toolbars – these are the same problems that were discussed at length by Jensen Harris (design lead for Office 2007). It becomes more interesting after Matt has this to say:

The Office Ribbon is such a radical departure from the existing paradigm of menus and toolbars and is deployed so widely that the ribbon interface will eventually become mainstream. From a consistency standpoint that made sense — many of our users have Office and AutoCAD running side by side. Having the applications behave somewhat consistently is important. Does this mean we follow Office to a T? No.

Once it was decided that we would go with our take on the ribbon — because the Office Ribbon was not entirely sufficient — we did our own due diligence and research into what an AutoCAD ribbon would become.

If you read the rest of the interview and spend at least a few minutes with AutoCAD 2009, you would see that the “somewhat consistently” is an overstatement. Let me show you a handful of screenshots that illustrate how different the AutoCAD 2009 ribbon is from Office 2007 ribbon.

It starts off quite innocently with the default UI for editing a new empty document (click to see the full size):

While the appearance of most of the elements (gradients, curved corners, title bar and the application menu button) is not as polished as Office 2007, the ribbon is easily identifiable. It becomes more interesting with the little gear button in the bottom right corner. It allows switching between the different workspaces. One of the workspaces is the 3-D, which presents a ribbon with completely different contents:

And yet another option which has been fondly named as “putting things back to normal”, that reverts AutoCAD to the old menu bar / toolbar UI:

The little down arrow to the right of the tabs allows switching between three different ribbon modes: full, panel titles and tabs only. The panel titles mode is not present in Office 2007 and shows the tabs as well as the titles of all the panels of the selected tab:

In this mode, moving the mouse over a panel title slides out this panel (unlike the full slide out in the minimized Office 2007 ribbon):

Moving the mouse over the panel title that has the triangular arrow and the panel slides out even further revealing additional controls:

The slide-out functionality explicitly goes against one of the basic Office 2007 ribbon tenets – all the available commands are visible when the relevant tab panel is showing. The panels slide out even in the full mode. Here is a screenshot of the Draw panel in full mode (note the triangular arrow in the bottom right):

And here it is when the mouse is clicked over the title panel:

The pin button in the fully expanded panel allows pinning down the slide out state, resulting in an unpleasant jagged footprint of the ribbon:

The square application menu button is nothing more than the traditional menu bar shoehorned into a two-level presentation with the sliding tree on the right hand side. Office 2007 puts only a few chosen commands there, each one accompanied by a rich description.

Perhaps the biggest departure from the Office 2007 ribbon is that AutoCAD 2009 allows floating its ribbon and docking it on all four sides of the application. Here is the floating ribbon:

And here it is vertically docked on the left:

Furthermore, individual panels can be “torn off” the ribbon and become floating windows (bringing back the one of the original pain points mentioned by Matt). The screenshot below shows one panel already torn off, and another being positioned to be grouped with it (you can have multiple panels floating independently):

Finally, contextual tabs have also received their share of rework. Matt specifically mentions that having the contextual tab take over the entire ribbon has not agreed with the user requirements, and the resulting implementation looks a little odd (but perhaps the usage patterns justify this). Once the contextual tab is shown, its panels are always present to the right of the panels of the currently selected tab – see the panels with the light blue titles on the right hand side of the ribbon:

Finally, AutoCAD provides complete freedom to arrange the ribbon contents as the user sees fit. In addition to floating, docking and tearing off individual panels, the user can hide specific tabs and panels. Furthermore, AutoCAD 2009 provides a UI to rearrange the contents of the panels themselves:

Is it worth bending the guidelines for the user requests? Is it worth restricting the user freedom to provide experience consistent with other applications? Do you make a clean cut away from the traditional menu bar / tool bar / palettes approach, or do you provide a way to switch back? Are you prepared to invest significant design effort in restructuring your UI around the ribbon guidelines, or are you willing to take shortcuts as you see fit? These are interesting questions, and AutoCAD 2009 has made quite a few far-reaching choices as far as the compliancy with the Microsoft guidelines goes.

Here are some Swing links that you might have missed during the last week:

• Ken Kousen has posted the third part in his explorations of mixing Swing and Groovy, addressing the EDT aspects of his earlier post. Swing EDT rule is very simple – anything UI related must be done on the EDT thread unless explicitly stated in the API specification. Ken’s narrowing of the rule to apply to updates only is unfortunate and is the same as using synchronization on setters but not on getters. It might bite you less, but it will bite you nonetheless.
• Matt Nathan reports on cleaning his SwingX incubator space. His incubator contains components (clock, stopwatch, inspection panel), a number of effects, rollover support and scalable icons (similar in concept to Flamingo‘s resizable icons).
• Greg Brown has announced release 1.0 of the Pivot framework that aims to allow building high-quality, cross-platform applications that are deployable both via the web and to the desktop.
• Robert Bajzat has published beta 0.7 release of Thinlet Swing components. It has a number of widgets and utilities, such as balloon tooltips, balloon dialogs, form fields with validations and rich editors (including calendar and tree), simple charts and more.
• Geertjan Wielenga interviews the new lead of Spring RCP project. Lieven Doclo has taken over the development which has been stopped and started too many times before him, and time will tell how much energy will be left after the first few weeks.
• Unlike in the years before where JavaPolis Javoxx Devoxx has positioned itself as a conference with hand-picked selection of by-invite-only speakers, this year the organizers have decided to issue a call for participation. Three active and well-maintained Java projects will be there. David Gilbert will present a session on JFreeChart, Maxim Zakharenkov will present a session on SwingExplorer, and Alex Potochkin will present a session on JXLayer.