In this entry i’m going to talk about the third step in the process of taking the UI definitions from your designer and converting them to a live implementation. This process is illustrated by taking design of Cookbook UI (from the My Dream App contest) and turning it into a Swing application using Substance look-and-feel. The code behind the process is available in the Substance Samples project that aims to provide a collection of blueprints for creating visually rich UIs in Swing.
Overview of the application and Substance decoration areas
In the first step we have identified the decoration areas of Cookbook UI, and in the second step we mapped these areas to Substance decoration areas. The screenshot below shows the application decoration areas:
And the screenshot below shows an overlay of the Swing container hierarchy and the tagging of relevant containers associating them with the matching Substance decoration areas:
It is now time to implement the visuals of Cookbook decoration areas.
Fidelity to the original design
During the implementation stage you should strive to be as close as possible to the original design. While sometimes the technical limitations of the specific UI toolkit / library may prevent achieving the exact visuals and dynamic behavior as designed, you as the developer should not decide to change the design without providing feedback to the design team and waiting for the reworked visuals to implement.
As far as the implementation is concerned, a valid and in most cases simpler option would be to work on the image level. Most probably the design team uses a tool that allows compositing multiple layers into the final files that they are passing to the development. Those layers that represent decoration textures can be used directly to provide high-fidelity implementation. There is a number of important points to remember if you decide to go this route:
- Target user machines can have different monitor sizes, and some environments may be multi-monitor setups. The images need to be big enough to accomodate dynamically resizable UIs that can sometimes span multiple monitors.
- If you decide to use a smaller size image and replicate (tile) it to cover the entire span of the relevant area, make sure that the image edges provide seamless appearance when placed in succession. Even the smallest discrepancy will unconsciously bother the users and degrade the user experience.
- You need to decide on the anchor point. Is it the top-left corner of the monitor, or the top-left corner of the application window? Some UI toolkits (such as Swing) may employ drawing optimization techniques that can result in image shearing on partial redrawing.
Another option is to implement the target design programmatically. While it is harder to achieve the exact visuals using pure code (even with such an advanced API as Java2D), this approach may be suitable for the programmers who are at ease with the APIs presented by the relevant layers. While the exact visuals will be most probably impossible to achieve, with the right amount of tweaking and feedback from the designers it is possible to arrive at a solution that satisfies both sides.
Implementing Substance decorations
To show my appreciation of the power of Java2D (and as a programmer myself), this specific implementation is going to use pure code to achieve visuals that are significantly close to the target UI. To speed up the development process i’m going to use the excellent collection of filters available from Jerry Huxtable under the Apache 2.0 license.
The header and the footer portions of the Cookbook UI use a texture that looks like a cross between brushed metal and wood. To implement this texture, i’m using a combination of <span style="color: darkblue;">BrushedMetalFilter</span> and <span style="color: darkblue;">LookupFilter</span> to create a brushed texture that is mapped to the colors of the relevant Substance color scheme (more on the color scheme selection in the next phase).
Here is the texture image that is going to be painted on top of the gradient fill:
The screenshot on the left shows a portion of Cookbook UI header area, and right next to it is its implementation with a custom Substance decoration painter based on the above filters:
The screenshot on the left shows a portion of Cookbook UI footer area, and right next to it is its implementation with a custom Substance decoration painter based on the above filters:
The sidebar of the Cookbook UI uses a fine-wood texture. To implement this texture, i’m using a combination of <span style="color: darkblue;">WoodFilter</span> and <span style="color: darkblue;">LookupFilter</span> to create a wood texture that is mapped to the colors of the relevant Substance color scheme.
Here is the texture image that is going to be painted on top of the gradient fill:
The screenshot on the left shows a portion of Cookbook UI header area, and right next to it is its implementation with a custom Substance decoration painter based on the above filters:
Positioning and sizing the generated textures
As mentioned in the mapping step, the implementation must not expose the discrepancy between application and implementation areas. In our case, the sidebar spans two adjacent panels, and the footer spans three panels that don’t even have the same immediate parent. It is absolutely critical not to let these details escape into the UI visuals. The visuals presented to the user must be seamless as though painted in the same step.
Considerations when using images are applicable for the pure Java2D implementation. If you’re going to tile smaller images, you need to make sure that the edges are seamless. If you’re going to create one big image, you need to query the toolkit and compute the combined bounds of all monitors (in this aspect it is easier to go pure-Java2D route than use images). Last, you need to anchor the images so that partial rendering does not result in sheared tiling.
The specific implementation uses APIs available on <span style="color: darkblue;">GraphicsEnvironment</span>, <span style="color: darkblue;">GraphicsDevice</span>, <span style="color: darkblue;">GraphicsConfiguration</span> to query the monitor configuration and compute the combined bounds of all monitors. The images created using filters thus cover the entire screen setup. There are two disadvantages of doing so – longer computation time and higher memory footprint. On the other hand, we do not need to worry about tile edges. Also, large areas will be painted in one Java2D call, as opposed to possibly multiple renderings of different areas of a smaller tile. Last, the anchoring is done to the top-left corner of the application window.
When the specific panel needs to be painted, we compute the offsets to the top-left corner of the application window. These offsets are then used to compute the area of the texture image to be painted on the panel. This ensures that even though the “perceived” (application) decoration area is implemented by multiple containers / components, the appearance is seamless and continuous.
Implementing the lights
The “light bar” of Cookbook UI proved to be the most challenging of the visuals. While it can be considered a tiled texture, it is explicitly anchored to a specific point in the window (the bottom-left corner of the light holder). While other textures (header, footbar) can be arbitrarily anchored (as long as all relevant areas use the same anchor point), this assumption does not hold here.
In addition, remember that the application sidebar is implemented as two separate Swing panels that do not share the same immediate parent. This is due to the functional restrictions detailed in the first step. If we mark each one of these panels as <span style="color: darkblue;">GENERAL</span>, we’re going to store an explicit reference to the top part of the sidebar (the light holder) in our decoration painter. This is not desirable since it creates an explicit dependency between the skin and the application, which may prove problematic if we want to reuse this skin in other portfolio applications.
The solution presented in the second step is to mark the center panel of the sidebar container as <span style="color: darkblue;">GENERAL</span>, and mark the inner footer panels as <span style="color: darkblue;">FOOTER</span>. As the parent decoration area type is propagated to all children that do not define their own type, the sidebar panels are painted with <span style="color: darkblue;">GENERAL</span> type. When the specific sidebar panel is painted, we compute the offsets to the farthest container parent that is decorated with <span style="color: darkblue;">GENERAL</span>. These offsets will effectively hold the distance to the bottom-left corner of the custom component that implements the light holder visuals (more on this in the next step).
Having these offsets in hand we can start tiling an image of a single light until we cover the entire width of the currently painted panel. This ensures that even though the sidebar application decoration area is implemented by multiple containers / components, the appearance of the light bar is seamless and continuous.
It might be easier to ask the designers to provide a single translucent image of a light bulb with the matching diffused light cone. The present implementation imitates this with a blurred mushroom shape with a translucent gradient applied to it. However, my recommendation would be to use an image-based approach for such fine visuals as this.
Here is the result of Java2D-based implementation of a single light:
The next screenshot shows the light bar area of the Cookbook UI:
And the Java2D-based implementation which is not as close as the other visuals presented in this step:
Here are some Swing links that you might have missed during the last week:
- Fred Lavigne adds a JTable feature that he is missing from the Windows Explorer application – table header filler that spans to fit the available width when there is extra space after the columns. I’m not sure why he calls it a resize policy, but the code itself is quite straightforward. The only missing case is support for RTL tables (where the filler would be at the left end of the table header).
- Roman Kennke continues his explorations of AWT / Swing on platforms beyond the usual target set of the platform matrix officially supported by major vendors (Sun, Apple, IBM). His current research is to have AWT non top-level widgets being painted by Swing (pure Java2D) code. Roman is one of the very few people outside Sun working on the UI side of OpenJDK. Responses to his mails on the relevant OpenJDK mailing lists are generally indicative of possible inclusion of his work into the main branch, and this process is fulfilling the promise of opening the JDK to the general open source community. However, i still don’t see this promise coming to full fruition. Personally i have expected more openness from Java2D, AWT and Swing teams in areas such as long-term plans, bug fixing plans, shaping new APIs and more. Dmitri’s comment on one of my last entry is indicative of the processes happening behind the closed doors (or at least the appearance of such). The specific case (API for shaped and translucent windows) is quite important for modern UIs. It was introduced in a slightly behind-the-scenes fashion for 6u10 as an internal API mainly targetting JavaFX. Bringing the community into the API design loop might not have been viewed as necessary at that point (since the relevant class is in an internal package), but if this is going to be a part of the official Java 7 API, it’s much better to gather the community thoughts now before doing the actual implementation.
- Continuing on a tangent of unclear long-term plans for official Sun distribution of Java, Jacek Furmankiewicz has created a new project called ClassBuilder. The goal of this project is to provide framework for bytecode transformation of existing Java classes, in order to make them easily usable for data binding (and other uses) when developing Java desktop applications. This certainly fires a shot across the bow of BeansBinding project which hasn’t seen any development activity during the last ten months (with the exception of upgrading the project files to NetBeans 6.1 which can hardly be considered a development activity by any standards).
- And finally, Ken Orr has announced the Mac Widgets for Java project. The goal of this project is to provide a collection of widgets seen in OS X applications, offered in a cross-platform Java API.
In this entry i’m going to talk about the second step in the process of taking the UI definitions from your designer and converting them to a live implementation. This process is illustrated by taking design of Cookbook UI (from the My Dream App contest) and turning it into a Swing application using Substance look-and-feel. The code behind the process is available in the Substance Samples project that aims to provide a collection of blueprints for creating visually rich UIs in Swing.
Overview of the application areas
In the first step we have identified the decoration and functional areas of Cookbook UI. The screenshot below shows the application decoration areas:
and the next screenshot shows the application functional areas:
As you can see, the application decoration and functional areas are not the same. During this phase we are going to map the application functional areas to Swing container hierarchy and the application decoration areas to Substance decoration areas.
Mapping functional areas to Swing
This phase will in most cases take you from the realm of pure design to the limitations of Swing. While the application functional areas will in most cases define the hierarchy of your Swing containers, the mapping itself might not be straightforward. Designer’s work on the UI mockup screens does not consider the implementation limitations of the specific UI toolkit. This is especially true for such a dynamic area as resizability behavior – what happens when the user resizes the application window (if he is allowed to do so by the application code)?
If the UI design is done in a pixel-based tool (such as Photoshop), the most obvious choice for a Swing implementor would be to use absolute (null) layout manager. This is usually considered a bad Swing practice. Instead, you would usually employ a combination of core, third-party and custom layout managers that handle the initial sizing of the components and the dynamic resizing behavior.
Mapping decoration areas to Substance
This phase will take you further away from the pure design to the limitations of Substance. If you are not familiar with the concept of decoration areas and painters in Substance, please first read the following documentation:
- Decoration painters
<span style="color: darkblue;"><a href="https://substance.dev.java.net/docs/api/SetDecorationType.html">setDecorationType()</a></span> API.<span style="color: darkblue;"><a href="https://substance.dev.java.net/docs/api/GetDecorationType.html">getDecorationType()</a></span> API.
The list of available Substance decoration areas is defined in the <span style="color: darkblue;">org.jvnet.substance.painter.decoration.DecorationAreaType</span> enum.
As the documentation of <span style="color: darkblue;"><a href="https://substance.dev.java.net/docs/api/SetDecorationType.html">setDecorationType()</a></span> specifies, the passed decoration area type is applied to the component itself and recursively on all its children, unless a child is marked with another decoration area type (with the same API). When is this behavior useful? We’re going to see a more complicated example in a short while, but in the meantime here is a simple one:
This screenshot shows a footer bar of the recipe list panel. As you can see, while the add / remove buttons are using the same colors as the footer bar itself, the text field is skinned with a different color scheme. In Substance API, you would mark the footer bar with <span style="color: darkblue;">FOOTER</span> and the search text field with <span style="color: darkblue;">NONE</span>. All the buttons would get the <span style="color: darkblue;">FOOTER</span> from their parent (footer bar), while the search text field will use the color scheme bundle of the <span style="color: darkblue;">NONE</span> decoration are type.
While not necessarily required, it is useful to remember that most core (and supported third-party) components are already marked to belong to a specific Substance decoration area. This comes especially handy for the header application area. The header portion of Cookbook UI can be mapped to a decorated title pane (<span style="color: darkblue;">PRIMARY_TITLE_PANE</span> type) and an unfloatable toolbar (<span style="color: darkblue;">TOOLBAR</span> type). While not necessarily required, reusing existing Swing components will make the implementation part easier.
What is another usage of using different decoration area types on a parent component and one of its children? The answer is quite simple – when the application functional and decoration areas are different (like is the case with Cookbook UI). The screenshot below shows the sidebar part of Cookbook:
There is a clear visual continuation between the category list and recipe list as far as the decoration areas go. The textures (dark mahogany and golden brush) flow seemlessly across the boundaries of the functional areas, and the top bar with evenly-spaced diffused lights further enforces this continuation. The final polish comes from the vertical separator that uses matching colors in the different decoration areas.
If, for the reasons stated above, you have decided to build your Swing container hierarchy based on the application functional areas, you can use the matching Substance decoration area types on the specific sub-panels (without mixing the different types in the same hierarchy path). However, using the <span style="color: darkblue;"><a href="https://substance.dev.java.net/docs/api/SetDecorationType.html">setDecorationType()</a></span> API is only one part of the implementation. The second part comes from your custom decoration painter that is implementing the painting itself. While this topic is subject of the next phase, the ease of implementation is going to dictate mixing different decoration area types as shown in the next section.
Swing containers and Substance decoration areas
This section will show the Swing hierarchy behind the Substance-based implementation of Cookbook UI and the matching decoration area types. While most of the reasons behind this implementation are given in the sections above, one of them is part of the next phase (simplifying the implementation complexity of the custom decoration painter).
We’re going to start with the layout of the content pane:
Here, we’re using the core <span style="color: darkblue;">BorderLayout</span> and add toolbar at <span style="color: darkblue;">NORTH</span>, sidebar at <span style="color: darkblue;">WEST</span> and main recipe panel at <span style="color: darkblue;">CENTER</span>.
The next screenshot shows the children of the sidebar panel:
The top part is the only custom component which is not painted by Substance. This is done for purely implementation complexity considerations – it is much easier to override the <span style="color: darkblue;">paintComponent()</span> method to emulate the light holder than to do it with Substance decoration painters. The <span style="color: darkblue;">CENTER</span> part is another panel which is decorated with <span style="color: darkblue;">GENERAL</span> area type. As the next phase will show, this makes it easier to implement the evenly-spaced diffused lights (the continuous look of the texture is not a factor in this particular decision).
Inside this <span style="color: darkblue;">GENERAL</span>-decorated panels we have the <span style="color: darkblue;">BorderLayout</span> with two panels:
Note that these two panels do not have any decoration area type installed on them – they get the <span style="color: darkblue;">GENERAL</span> from the parent panel.
The last diagram shows the last level of Swing hierarchy as far as mapping the decoration areas goes:
Here, each (functional) panel has a footer bar and the main area. The footer bar is a panel decorated with <span style="color: darkblue;">FOOTER</span>, thus overriding the decoration area type inherited from the grandparent panel. The main areas do not have any decoration area types installed on them, inheriting the grandparent panel settings.
The final piece comes by installing the <span style="color: darkblue;">NONE</span> type on the search text field in the second footer bar, thus overriding the inherited <span style="color: darkblue;">FOOTER</span> setting.
In this entry i’m going to talk about the first step in the process of taking the UI definitions from your designer and converting them to a live implementation. This process is illustrated by taking design of Cookbook UI (from the My Dream App contest) and turning it into a Swing application using Substance look-and-feel. The code behind the process is available in the Substance Samples project that aims to provide a collection of blueprints for creating visually rich UIs in Swing.
Identifying application decoration areas
The first step in implementing a specific UI is identifying the application decoration areas. Here is a thumbnail of the main Cookbook design:
This screen has four main decoration areas. The following screenshots show a translucent light blue overlay to highlight each one of the areas.
The top portion of the application window is the header:
The bottom portion of the application window is the footer:
The left portion of the application window is the sidebar:
The center-right portion of the application window is the main:
Identifying application functional areas
Unlike the decoration areas that are primarily delineated by similar colors and textures, the functional areas are delineated primarily by the user-facing business logic. In Cookbook UI there are three main functional areas (in addition to the general control area in the header):
Note how the functional areas and decoration areas are not identical. In fact, every functional area intersects two decoration areas (and vice versa in most cases). Here is a fll-size view of the sidebar area in Cookbook UI:
Note that while the lists and footers have different visuals (colors, textures, gradients), they belong to the same functional group. For example, the footer controls for the category list are located directly beneath it, and the footer panel itself has the same width as the category list. This is especially true for dynamic scenarios when the entire UI is resized or re-proportioned.
This step may cross the line between the pure design and the implementation (Swing or other UI toolkits). As the developer, you will mostly be operating on the functional UI areas. It is important to understand that Substance supports the notion of different decoration and functional areas. However, in some cases the certain limitations of the Substance decoration layer will impose some restrictions on the way you structure the Swing hierarchy.
The next part will talk about mapping application functional areas to Swing container hierarchy and mapping the application decoration areas to Substance decoration areas.
