This has been on my to-do list for quite a long time, and over the last couple of months i have finally started to add unit tests to the Flamingo component suite. Before long, i have found myself stuck deep in a quagmire of nested runnables, complex interactions with the Event Dispatch Thread (EDT) and code that is hard to read, extend and maintain. And even then the tests behaved unpredictably, with some failing randomly – indicating incorrect interactions with the EDT.
At that moment i have realized that the true potential of open source is collaboration and reuse of existing libraries – unless you’re feeling that you can do better, of course :) The final tipping point came in July when Alex has published a blog entry on how FEST Swing library can facilitate writing clean and maintainable interactions with UI components – and this was it. Since then i have been adding FEST-driven unit tests to test different aspects of Flamingo components, including model manipulation, layout, and interaction with mouse and keyboard. Here, i’m going to show a small subset of these tests – illustrating the relevant parts of FEST Swing.
Before starting to delve into the project documentation, start with the blog entry on writing EDT-safe UI tests. It’s short, it’s clean and it’s quite illuminating. Then, head over to the main project page, read a little bit about the library and download the latest distribution (i’m using the latest 1.2a3 version). Once you’re done, add the following jars to your classpath:
fest-assert-1.1.jar
fest-reflect-1.1.jar
fest-swing-1.2a3.jar
fest-swing-junit-1.2a3.jar
fest-util-1.1.jar
junit-4.3.1.jar
Now let’s take a look at how a unit test class looks like. It starts by extending the base FEST class:
public class ActionCommandButtonTestCase extends FestSwingJUnitTestCase {
The biggest advantage of using this base class is that it automatically installs a custom repaint manager that tracks EDT violations – keeping both your unit tests and your main code clean from them. Next, it’s time to create the main window and a command button for the testing:
JFrame buttonFrame;
int count;
JCommandButton button;
@Override
@Before
public void onSetUp() {
URL resource = ActionCommandButtonTestCase.class.getClassLoader()
.getResource("utest/common/edit-paste.svg");
Assertions.assertThat(resource).isNotNull();
final ResizableIcon icon = SvgBatikResizableIcon.getSvgIcon(resource,
new Dimension(32, 32));
Pause.pause(new Condition("Waiting to load the SVG icon") {
@Override
public boolean test() {
return !((AsynchronousLoading) icon).isLoading();
}
});
GuiActionRunner.execute(new GuiTask() {
@Override
protected void executeInEDT() throws Throwable {
buttonFrame = new JFrame();
buttonFrame.setLayout(new FlowLayout());
button = new JCommandButton("test", icon);
button.setDisplayState(CommandButtonDisplayState.BIG);
buttonFrame.add(button);
buttonFrame.setSize(300, 200);
buttonFrame.setLocationRelativeTo(null);
buttonFrame.setDefaultCloseOperation(JFrame.DISPOSE_ON_CLOSE);
buttonFrame.setVisible(true);
count = 0;
button.addActionListener(new ActionListener() {
@Override
public void actionPerformed(ActionEvent e) {
count++;
}
});
}
});
GuiActionRunner.execute(new GuiTask() {
@Override
protected void executeInEDT() throws Throwable {
Point locOnScreen = buttonFrame.getLocationOnScreen();
locOnScreen.move(10, 20);
robot().moveMouse(locOnScreen);
}
});
}
While the documentation describes two ways of locating components for testing – fixtures and finders – i found it simpler to just store references to the relevant components, especially for test windows that have a very small number of controls.
The onSetUp method has the following main stages:
Create an SVG-based icon and wait for it to load. Here, Pause.pause is used to wait for the specific Condition – the name of the test() method is quite unfortunate, and IMO should be renamed to better reflect its purpose.
Then, GuiActionRunner.execute(GuiTask) is called to create the main window and the command button. This is by far my favorite method in FEST Swing – it runs the specified code on EDT and waits for it to finish. And while closures could have made the code even more readable, even in its present form it has enormous potential to simplify UI-related tests.
Finally, the same method is used to move the mouse away from the button – for subsequent tests of mouse interaction.
Once we have the setup method in place, time for the a very simple unit test:
Here, my second best-favorite feature – GuiActionRunner.execute(GuiQuery) – is used. It provides an EDT-safe way to query the current state of the specific UI component.
At this point it is important to note that UI testing can only be done to a certain degree. While the test above checks that the getText() returns the right value, it does not check that the visual representation of the button on the screen actually display this text (or any text for that matter). Testing the correctness of the painting routines is not simple. Even a straightforward approach of using an offline collection of “expected” screenshots will not work under look-and-feels that use the font settings of the current desktop – which varies not only across operating systems, but also across different DPI settings. As such, at the present moment my tests are focusing on checking the model, layouts and mouse / keyboard interactions.
Here are three tests to check that the associated action listener is invoked when the command button is activated with mouse, keyboard or API:
Here i’m using different APIs of the Robot class to simulate the user interaction with keyboard and mouse, as well as EDT-safe invocation of the AbstractCommandButton.doActionClick() API.
Having these building blocks in place allows you to create more complex scenarios, testing various flows through your model classes. Here is a test case that checks that pressing the mouse, moving it away from the button and then releasing it does not activate the registered action listener:
@Test
public void pressButtonAndMoveAwayBeforeRelease() {
robot().pressMouse(button, AWT.centerOf(button));
robot().waitForIdle();
robot().moveMouse(button, new Point(-10, 10));
robot().waitForIdle();
robot().releaseMouseButtons();
robot().waitForIdle();
// no action listener should have been invoked
Assertions.assertThat(count).isEqualTo(0);
}
Finally, a more complex scenario tests the API that activates the registered action listener on mouse press – as opposed to mouse release:
@Test
public void fireActionOnPress() {
GuiActionRunner.execute(new GuiTask() {
@Override
protected void executeInEDT() throws Throwable {
button.getActionModel().setFireActionOnPress(false);
}
});
Assertions.assertThat(GuiActionRunner.execute(new GuiQuery() {
@Override
protected Boolean executeInEDT() throws Throwable {
return button.getActionModel().isFireActionOnPress();
}
})).isFalse();
// press mouse over the button
robot().pressMouse(button, AWT.centerOf(button));
robot().waitForIdle();
// no action listener should have been invoked
Assertions.assertThat(count).isEqualTo(0);
// release mouse
robot().releaseMouseButtons();
robot().waitForIdle();
// action listener should have been invoked
Assertions.assertThat(count).isEqualTo(1);
// mark the button to fire the action listeners on mouse press
GuiActionRunner.execute(new GuiTask() {
@Override
protected void executeInEDT() throws Throwable {
button.getActionModel().setFireActionOnPress(true);
}
});
Assertions.assertThat(GuiActionRunner.execute(new GuiQuery() {
@Override
protected Boolean executeInEDT() throws Throwable {
return button.getActionModel().isFireActionOnPress();
}
})).isTrue();
// press mouse over the button
robot().pressMouse(button, AWT.centerOf(button));
robot().waitForIdle();
// action listener should have been invoked
Assertions.assertThat(count).isEqualTo(2);
// release mouse
robot().releaseMouseButtons();
robot().waitForIdle();
// no action listener should have been invoked
Assertions.assertThat(count).isEqualTo(2);
}
This entry showed a very small subset of what FEST Swing can do. I am still exploring this library as i continue adding more test cases to cover the command buttons – towards unit tests that will cover the ribbon component.
Print out one of the PDF markers in the NYArToolkit/Data. For my demo, i’m using pattHiro.pdf
Import all projects under NYArToolkit in your Eclipse workspace. You can ignore the ones for JOGL and QuickTime for this specific demo.
In these projects, tweak the build path to point to the JMF / Java3D jars on your system.
Plug in your web camera.
Run jp.nyatla.nyartoolkit.jmf.sample.NyarToolkitLinkTest class to test that JMF has been installed correctly and can display the captured feed.
Run jp.nyatla.nyartoolkit.java3d.sample.NyARJava3D class to test the tracking capabilities of the NYArToolkit. Once the camera feed is showing, point the camera to the printed marker so that it is fully visible. Once you have it on your screen, a colored cube should be shown. Try rotating, moving and tilting the marker printout – all the while keeping it in the frame – to verify that the cube is properly oriented.
The demo from the video above is available as a part of the new Marble project. Here is how to set it up:
Sync the latest SVN tip of Marble.
Create an Eclipse project for Marble. It should have dependencies on NYArToolkit, NYArToolkit.utils.jmf, NYArToolkit.utils.java3d and Trident. It should also have jmf.jar, vecmath.jar, j3dcore.jar and j3dutils.jar in the build path.
Run the org.pushingpixels.marble.MarbleFireworks3D class. Follow the same instructions as above to point the webcam.
There’s not much to the code in this Marble demo. It follows the NyARJava3D class, but instead of static Java3D content (color cube) it has a dynamic scene that is animated by Trident. For the code below note that i’m definitely not an expert in Java3D and NYArToolkit, so there might as well be a simpler way to do these animations. However, they are enough to get you started in exploring animations in Java-powered augmented reality.
Each explosion volley is implemented by a collection of Explosion3D objects. Each such object models a single explosion “particle”. Here is the constructor of the Explosion3D class:
public Explosion3D(float x, float y, float z, Color color) {
this.x = x;
this.y = y;
this.z = z;
this.color = color;
this.alpha = 1.0f;
this.sphere3DTransformGroup = new TransformGroup();
this.sphere3DTransformGroup
.setCapability(TransformGroup.ALLOW_TRANSFORM_WRITE);
Transform3D mt = new Transform3D();
mt.setTranslation(new Vector3d(this.x, this.y, this.z));
this.sphere3DTransformGroup.setTransform(mt);
this.appearance3D = new Appearance();
this.appearance3D
.setCapability(Appearance.ALLOW_TRANSPARENCY_ATTRIBUTES_WRITE);
this.appearance3D
.setCapability(Appearance.ALLOW_COLORING_ATTRIBUTES_WRITE);
this.appearance3D.setColoringAttributes(new ColoringAttributes(color
.getRed() / 255.0f, color.getGreen() / 255.0f,
color.getBlue() / 255.0f, ColoringAttributes.SHADE_FLAT));
this.appearance3D.setTransparencyAttributes(new TransparencyAttributes(
TransparencyAttributes.BLENDED, 0.0f));
this.sphere3D = new Sphere(0.002f, appearance3D);
this.sphere3DTransformGroup.addChild(this.sphere3D);
this.sphere3DBranchGroup = new BranchGroup();
this.sphere3DBranchGroup.setCapability(BranchGroup.ALLOW_DETACH);
this.sphere3DBranchGroup.addChild(this.sphere3DTransformGroup);
}
Here, we have a bunch of Java3D code to create a group that can be dynamically changed at runtime. This group has only one leaf – the Sphere object.
As we’ll see later, the timeline behind this object changes its coordinates and the alpha (fading it out). Here is the relevant public setter for the alpha property:
Not much here – just updating the transparency of the underlying Java3D Sphere object. The setters for the coordinates are quite similar:
public void setX(float x) {
this.x = x;
Transform3D mt = new Transform3D();
mt.setTranslation(new Vector3d(this.x, this.y, this.z));
this.sphere3DTransformGroup.setTransform(mt);
}
public void setY(float y) {
this.y = y;
Transform3D mt = new Transform3D();
mt.setTranslation(new Vector3d(this.x, this.y, this.z));
this.sphere3DTransformGroup.setTransform(mt);
}
public void setZ(float z) {
this.z = z;
Transform3D mt = new Transform3D();
mt.setTranslation(new Vector3d(this.x, this.y, this.z));
this.sphere3DTransformGroup.setTransform(mt);
}
The main class is MarbleFireworks3D. Its constructor is rather lengthy and has a few major parts. The first part initializes the camera and marker data for the NYArToolkit core:
NyARCode ar_code = new NyARCode(16, 16);
ar_code.loadARPattFromFile(CARCODE_FILE);
ar_param = new J3dNyARParam();
ar_param.loadARParamFromFile(PARAM_FILE);
ar_param.changeScreenSize(WIDTH, HEIGHT);
Following that, there’s a bunch of Java3D code that initializes the universe, locale, platform, body and environment, and creates the main transformation group. The interesting code is the one that creates the main scene group that will hold the dynamic collection of Explosion3D groups:
mainSceneGroup = new TransformGroup();
mainSceneGroup.setCapability(TransformGroup.ALLOW_TRANSFORM_WRITE);
mainSceneGroup.setCapability(Group.ALLOW_CHILDREN_EXTEND);
mainSceneGroup.setCapability(Group.ALLOW_CHILDREN_WRITE);
root.addChild(mainSceneGroup);
nya_behavior = new NyARSingleMarkerBehaviorHolder(ar_param, 30f,
ar_code, 0.08);
nya_behavior.setTransformGroup(mainSceneGroup);
nya_behavior.setBackGround(background);
root.addChild(nya_behavior.getBehavior());
nya_behavior.setUpdateListener(this);
locale.addBranchGraph(root);
The NyARSingleMarkerBehaviorHolder is a helper class from the NYArToolkit.utils.java3d project. It tracks the transformation matrix computed by NYArToolkit based on the current location of the marker and updates the transformation set on the main scene group. As you will see later, there is no explicit handling of the marker tracking in the demo code – only creation, update and deletion of the Explosion3D objects.
Finally, we create a looping thread that adds random firework explosions:
// start adding random explosions
new Thread() {
@Override
public void run() {
while (true) {
try {
Thread.sleep(500 + (int) (Math.random() * 1000));
float x = -1.0f + 2.0f * (float) Math.random();
float y = -1.0f + 2.0f * (float) Math.random();
float z = (float) Math.random();
addFireworkNew(x * 5.0f, y * 5.0f, 5.0f + z * 12.0f);
} catch (Exception exc) {
}
}
}
}.start();
The code in this method computes a 3D uniform distribution of small spheres that originate at the specific location (explosion center) and move outwards. Each Explosion3D object is animated with the matching timeline. The timeline interpolates the alpha property, as well as the coordinates. As you can see, while x and y are interpolated linearly, the interpolation of z takes the gravity into the account – making the explosion particles fall downwards. All the timelines are added to a parallel timeline scenario. Once a timeline starts playing, the matching branch group is added to the main scene graph. Once the timeline scenario is done, all the branch groups are removed from the main scene graph:
private void addFireworkNew(float x, final float y, final float z) {
final TimelineScenario scenario = new TimelineScenario.Parallel();
Set scenarioExplosions = new HashSet();
float R = 6;
int NUMBER = 20;
int r = (int) (255 * Math.random());
int g = (int) (100 + 155 * Math.random());
int b = (int) (50 + 205 * Math.random());
Color color = new Color(r, g, b);
for (double alpha = -Math.PI / 2; alpha <= Math.PI / 2; alpha += 2
* Math.PI / NUMBER) {
final float dy = (float) (R * Math.sin(alpha));
final float yFinal = y + dy;
float rSection = (float) Math.abs(R * Math.cos(alpha));
int expCount = Math.max(0, (int) (NUMBER * rSection / R));
for (int i = 0; i < expCount; i++) {
float xFinal = (float) (x + rSection
* Math.cos(i * 2.0 * Math.PI / expCount));
final float dz = (float)(rSection
* Math.sin(i * 2.0 * Math.PI / expCount));
float zFinal = z + dz;
final Explosion3D explosion = new Explosion3D(x * SCALE, y * SCALE,
z * SCALE, color);
scenarioExplosions.add(explosion);
final Timeline expTimeline = new Timeline(explosion);
expTimeline.addPropertyToInterpolate("alpha", 1.0f, 0.0f);
expTimeline.addPropertyToInterpolate("x", x * SCALE, xFinal
* SCALE);
expTimeline.addPropertyToInterpolate("y", y * SCALE, yFinal
* SCALE);
expTimeline.addCallback(new TimelineCallbackAdapter() {
@Override
public void onTimelinePulse(float durationFraction,
float timelinePosition) {
float t = expTimeline.getTimelinePosition();
float zCurr = (z + dz * t - 10 * t * t) * SCALE;
explosion.setZ(zCurr);
}
});
expTimeline.setDuration(3000);
expTimeline.addCallback(new TimelineCallbackAdapter() {
@Override
public void onTimelineStateChanged(TimelineState oldState,
TimelineState newState, float durationFraction,
float timelinePosition) {
if (newState == TimelineState.PLAYING_FORWARD) {
mainSceneGroup.addChild(explosion
.getSphere3DBranchGroup());
}
}
});
scenario.addScenarioActor(expTimeline);
}
}
synchronized (explosions) {
explosions.put(scenario, scenarioExplosions);
}
scenario.addCallback(new TimelineScenarioCallback() {
@Override
public void onTimelineScenarioDone() {
synchronized (explosions) {
Set ended = explosions.remove(scenario);
for (Explosion3D end : ended) {
mainSceneGroup
.removeChild(end.getSphere3DBranchGroup());
}
}
}
});
scenario.play();
}
The dependent libraries that are used here do the following:
JMF captures the webcam stream and provides the pixels to put on the screen
NYArToolkit processes the webcam stream, locates the marker and computes the matching transformation matrix
Java3D tracks the scene graph objects, handles the depth sorting and the painting of the scene
Trident animates the location and alpha of the explosion particles
Release 5.2 of Substance look-and-feel made a few visual changes to the Raven Graphite skins, but these did not address the overall usability of these skins – especially the contrast between the background and the controls, and the background / foreground contrast of text components.
The latest drop of version 5.3dev (code named Reykjavik) features significant overhaul of both Raven Graphite skins, aiming to address the contrast usability issues raised by the users.
Here is a screenshot of a sample application under the Raven Graphite skin in the latest stable 5.2 release:
And here is the same application under the 5.3dev drop:
Here is another screenshot of the same application under the old Raven Graphite visuals:
and the new visuals under the latest 5.3dev drop:
The main changes are:
Removing the watermark that contributed significant visual noise
Darker border color for controls, bringing more delineation to check boxes and radio buttons
Darker background color for text components, resulting in better readability
The same changes were made for the Raven Graphite Glass skin. Here is the sample application under the stable 5.2 release:
and here is the same application under the latest 5.3dev drop:
In addition to the visual changes above, the Raven Graphite Glass skin has removed the glass arc gradient from the toolbars and added a two-tone separator to delineate the title bar / menu bar from the rest of the application content.
To illustrate the visual difference in a larger content, here is a screenshot of a big UI under the stable 5.2 release (click to see the full-size view):
and the same application under the 5.3dev branch:
If you want to take the new visuals for a spin, click on the WebStart button below and change the skin to Raven Graphite and Raven Graphite Glass from the “Skins” menu:
You’re more than welcome to take the latest 5.3dev drop for a spin and leave your comments.
Concluding the series on adding animations to enable rich interactivity expected from modern Swing applications, here is what we have seen so far:
Part 1 – adding simple animation behavior to such scenarios as component appearance (fade in), rollovers and window disposal (fade-out) using built in and custom class attributes and setters.
Part 2 – adding animated load progress indication while the application is loading data.
Part 3 – loading the album art matching the specific search string and asynchronously displaying the associated images.
Part 4 – scrolling the album covers showed in the container and adding animations to the scrolling.
Run the org.pushingpixels.onyx.DemoApp class, passing your Amazon key as the only parameter to this class, adding the Amazon, Trident and Onyx classes to the classpath
If all went right, you should see the main application running and displaying Sarah McLachlan albums as in this video:
I hope you enjoyed this series. If you’re interested in adding rich animations to your Swing applications, you’re more than welcome to explore Trident and Onyx and report any bugs and missing features in the project forums and mailing lists.
