March 29th, 2017

How deep the rabbit hole goes

Back in the olden days of 1999 it was pretty much the only movie that I watched in the theaters. In pre-digital days it took a few months for a movie to complete its theatrical rollout across the globe, and once it got into theaters, it stayed for much longer than it does these days. Such was the story of “The Matrix” for me. It stayed in local theaters for at least six months, and I was a single guy with not much to do in the evening after work. So every week, at least twice a week, I would go to watch it again. And again. And again. It’s quite unlikely, in fact, that there’s ever going to be a movie that I’ll watch more times than I’ve watched “The Matrix”.

Back in those olden days, people didn’t wake up to write a new Javascript library. People woke up to write a Matrix rain screensaver. Those would be the mirrored half-width kanas, as well as Latin characters and arabic numerals.

A few years later, “Matrix: Reloaded” came out, taking the binary rain into the third dimension as the glyphs were sliding down multiple virtual sheets of glass. And I finally decided to dip my toes into the world of making my own Matrix rain screensaver, complete with many of the visual effects that were seen in that movie. There’s a bunch of old code that I’ve uploaded as an historical artifact right here. Fair warning – this was 13 years ago, and as many do when they first start out, I reimplemented a bunch of stuff that was already there in the JDK. If you dive into the code, you’ll see a home grown implementation of a linked list, as well as a rather gnarly monstrosity that exposed something that resembled a canvas / graphics API. Don’t judge me. Anyhoo, on to the topic of this post.

One of the things I’ve wanted to do in that screensaver was to take a string as input and render it in the style of Matrix titles:

In here, every glyph undergoes one or more transformations (cropping, displacement, segment duplication). In addition, there are connectors that link glyphs together. It is these connectors that I’m going to talk about. Or, more precisely, how can you come up with the “best” way to connect the glyphs of any input string, and what makes a particular connector chain the “best” chain for that string?

This image captures the “essence” of quantifying the quality of a connector. In the title sequence of the original movie, as well as the sequels, the connectors are only placed at three vertical positions – top, middle and bottom. That is the starting point of this diagram. In addition, there are the following factors at the level of an individual glyph:

  1. On the scale from 1 to 5, how far the connector would have to go “into” the glyph to connect to the closest pixel? So, the bottom part of A gets 5’s on both sides, and the top part gets 2’s on both sides. The middle part of J gets 0 on the left (as the connector would have to “travel” across the entire glyph) and 4 on the right (as the connector would need to go past the rightmost point of the top serif).
  2. Defining a “natural” connection point to be (in the diagram above green marks such a point while red signifies that the point is not natural):
    • Anything on top and bottom – this is an escape valve that would make sure that any input string has at least one connector chain
    • Serifs in the middle – such as the right side of G
    • Crossbars in the middle, extending to both sides of the glyph – such as A, B or R.

Then, a valid connector chain would be defined as:

  1. No two consecutive connectors can be placed at the same vertical position. In the example of the original title, the connector chain is top-bottom-middle-bottom-top.
  2. A connector must have positive (non-zero) value on both sides. For example, you can’t connect A and J in the middle because the left side of J places value 0 on the middle position.
  3. A connector must have at least one natural connection point. For example, N and O can’t be connected in the middle, while N and P can (as P’s left side defines the middle position as a natural connection point)

Finally, the overall “quality” of the entire connector chain is computed as:

  1. The sum of connection point values along both sides of each connector
  2. Weighed by the chi-square value of the connector vertical positions along the entire chain
  3. Weighed by the mean probability of the connector vertical positions along the entire chain

The last two factors aim to “favor” chains that look “random”. For example, there is not much randomness in a top-bottom-top-bottom-top-bottom chain. You want to have a bit of variety and “noise” in a chain so that it doesn’t look explicitly constructed, so to speak. As can be seen in the diagram above, the middle vertical position is not a natural connection point for a lot of glyphs, and both of these factors aim to bring a well-distributed usage of all vertical position into the mix.

It is true that the basic underlying rules of defining how a glyph connector chain is constructed are based on the visuals of the Matrix movie titles. You might think of this as the basic rules of physics that apply to the particular universe. However, the evaluation of a specific constructed chain is a softer framework, so to speak. There is nothing explicit in these rules that would force the quality score of the particular connector chain that you see in the final graphics for these particular six letters to be the highest of all valid chains.

When I first ran the finished implementation, it was one of those rare moments of pure, unadulterated geek joy:

These are all possible valid connector chains for the word “MATRIX”, ordered by the quality score that is based on values of individual connector points, as well as statistical variation that accounts for predictability and randomization within a specific sequence. Yes, the top score goes to the sequence that was used in the movie title!

Let’s look at “RELOADED” next:

And these are the top 39 valid connector chains for that word:

While my algorithm found the perfect match for “MATRIX” connector chain, the connector chain that was used in the movie for “RELOADED” is scored at place #37. You can see where it falls flat – in the top connector between L and O. The score value for top connector on the right side of L is 1 out of 5, and while the score value for top connector on the left side of O is 5 out of 5, that drastically lowers the overall score. In addition, the last four connectors are bottom-middle-bottom-middle which lowers the median probability factor applied to the entire quality score of this chain.

The connector chain selected by the third movie for the word “REVOLUTIONS” is not considered a valid one based on the rules that I chose after “Reloaded” was out. Specifically, the middle connector between U and T is not valid, as there is neither a serif not a crossbar in these two glyphs. And the same applies to the middle connector between I and O.

Finally, the “ANIMATRIX” title deviates slightly in the “MATRIX” part, using middle connector placement between M and A. How did my algorithm fair on scoring this chain?

This was a close one. The connector chain used in the movie title scores at the second place, with the only difference being in the very first connector (top instead of middle).

It’s hard to quantify artistic choices, and I don’t presume to claim that the top-scored connector chain for “RELOADED” based on the rules of my algorithm is clearly superior to what ended up in the actual movie titles. Would it be worth to tweak the scoring system? I don’t think so. There are a couple of noticeable “week” connectors in the connector chain in the movie title, and relaxing the scoring rules would only introduce more randomness into the process without necessarily bumping up that chain up the ranks.

Perhaps the artistic choice of choosing a long top L to O connector was based on introducing a bit of variance and randomness into the mix. Or perhaps I should check to see who was in charge of the title graphics and ask them :)

December 26th, 2013

Metric-driven design

One of my recent resolutions (not for 2014, but for mobile software in general in the last few months) was to evaluate designs of new apps and redesigns of existing apps from the position of trust. Trust in the designers and developers of those apps that they have good reasons to do what they do, even if it’s only one or two steps on the longer road towards their intended interaction model. But Twitter’s recent redesign of their main stream keeps on baffling me.Putting apart the (somewhat business-driven from what I gather) decision of “hiding” the mentions and DMs behind the action bar icons and adding the rather useless discover / activity tabs, I’m looking at the interaction model in the main (home) stream.

Twitter never got to the point of syncing the stream position across multiple devices signed into the same account. There is at least one third-party solution to do that, which requires third-party apps to use their SDK and users to use those apps. The developer of that third-party solution has repeatedly stated that in his opinion Twitter is not interested in discouraging casual users that check their streams every so often. If you start following random (mostly PR-maintained) celebrity streams, it’s easy to get lost in the Twitter sea, and when you check in every once in a while and see hundreds – if not thousands – of unread tweets, you might start feeling that you’re not keeping up.

As I’ve reached the aforementioned decision a few months ago, I’ve uninstalled all third-party Twitter apps I had on my phone, and switched to the official app. It does a rather decent job of remembering the stream position, as long as – from what I could see – I check the stream at least once every 24 hours. When I skip a day, the stream jumps to the top. It also seems to do that if the app refreshes the stream after I rotate the device, so some of this skipping can be attributed to bugs. But in general if I’m checking in twice a day and am careful not to rotate the device, the app remembers the last position as it loads the tweets above it.

In the last release Twitter repositioned the chrome around the main stream, adding discover / activity tabs above it and the “what’s happening” box below. While they encourage you to explore things beyond your main stream, it also looks like they’re aware that these two elements take valuable vertical space during the scrolling. And the current solution is to hide these two bars when you scroll down the stream.

And here’s what baffles me the most. On one hand, the app remembers the stream position, which means that I need to move the content down to get to the newer tweets (as an aside, with “natural” scrolling I’m not even sure if this is scrolling up or scrolling down”). On the other hand, the app hides the top tabs / bottom row when I move the content up.

Is the main interaction mode with this stream is getting to the last unread tweet and then going up the stream to skim the unread tweets, as hinted by remembering the stream position? Or is it getting bumped to the top of the stream and scrolling a few tweets down just to sample the latest few entries in it, as hinted by hiding the two chrome rows and providing more space during the scrolling?

I don’t want to say what the app should or shouldn’t do in the stream (as pointed out by M.Saleh Esmaeili). It’s just that I can’t get what the designers intend the experience to be.

A few days ago The Verge has posted an article around metric-driven design in Twitter mobile apps, an for me personally the saddest part of this article is how much they focus on engagement metrics and how little the guy talks about informed design. Trying to squeeze every possible iota of “interaction” out of every single element on the screen – on its own – without talking about the bigger picture of what Twitter wants to be as a designed system. Experiments are fine, of course. But jacking up random numbers on your “engagement” spreadsheets and having those dictate your roadmap (if one can even exist in such a world) is kind of sad.

When every interaction is judged by how much it maximizes whatever particular internal metric your team is tracking, you may find yourself dead-set on locating the local maximum of an increasingly fractured experience, with no coherent high-level design, and no clear path that you’re taking to arrive to the next level. Or, as Billie Kent’s character in Boardwalk Empire says, “always on the move, going nowhere fast.”

May 22nd, 2013

The conclusion does not follow

Over the last few months I’ve read at least a couple dozen online articles, threads and discussions about skeuomorphic and flat design. The articles came in three waves. The first wave was about how skeuomorphic is well past its prime time and needs to go away. The second wave was about how flat is the exact opposite of skeuomorphic, and how it is the new direction of visual interface design. And finally, the third wave is about the poor usability of flat design.

Some of this discussion is happening on Twitter. There’s only so much one can say in 140 characters. Even when you break your argument into multiple consecutive tweets, the argument – more often than not – gets a simplified presentation. And sometimes the simplification of presentation leads to cutting logical corners.

Flat is indeed the exact opposite of skeuomorphic, but only if you consider the constraints that the designer employs in choosing his palette. Leaving the restraint part out of this argument leads one to believe that as the opposite of skeuomorphic, flat design is always done well. That, of course, can’t be further from the truth. This omission, by the way, can be deliberate.

If one can simplify the argument to pure one-dimensional comparison, and follow the stripped down logic to the “necessary” conclusion that flat is good – and immediately show bad examples of flat design, then the entire argument falls apart. Now, if you trace the refuted conclusion back to its beginning, the implied coup de grace is that flat is bad.

The opposite of a poorly done skeuomorphic design is not a well done flat design. The logical fallacy here is taking one part of the constraints/restraint spiral and extending the comparison to include the second one. The opposite of a poorly done skeuomorphic design is a well done skeuomorphic design. The opposite of a poorly done flat design is a well done flat design.

The opposite of a poorly done design is a well done design. Well done design that takes a long, deliberate and careful look at the available tools (constraints) and plunges into a long exercise of applying restraint in using a subset of those tools to arrive at the final product.

May 21st, 2013

Fifty shades of flat

Design lives and dies by two things – constraints and restraint. Constraints bound the variety of tools at designer’s disposal, while restraint guides the designer’s hand at how the chosen tools are used to create the final product. It’s rarely a sequential process, but rather a spiral that tightens toward the end, where the outcome of the previous steps informs and guides the next iteration. And each step – in both spaces – requires painstaking attention to details, paced deliberation and continuous pursuit of honing one’s craft.

There is nothing inherently wrong in what people have come to call “skeuomorphic” design. The choice of fewer constraints gives you a wider selection of tools to work with. Glossy bevels, generous drop shadows, intricate textures, strong gradients and more – all of these justly belong in a rich skeuomorphic visual palette. There’s no shortage of examples of strikingly beautiful visual design. Design that combines multiple textures, gradients and materials in a consistent form. Design that exercises restraint in how the rich palette is used. Restraint in how elements are chosen, which elements are left out and, most importantly, how the chosen elements are combined together.

On the opposite hand of the spectrum, if you will, is the “flat” design. Design that chooses to constrain itself to a very small number of visual elements. Sharp corners, solid colors, simple shapes, absence of drop shadows or, for that matter, absence of a global lighting model. The stark beauty of flat design is particularly impressive given the constrained choice of the tools. One may even be led to believe that the simplicity of this palette translates directly into the simplicity of the overall design iteration spiral.

This couldn’t be farther from the truth. Tighter constraints that result in a reduced palette do not necessarily obviate the need for restraint. Restraint is not only which tools you choose from that palette. It is also – and much more so – about how you combine the tools together. How you define and implement a visual language that stays true to the flow and shape of the interaction.

It is trivial to create a bad skeuomorphic design. There are entire blogs dedicated to ridiculing poorly-executed mashes of garish textures, exaggerated lighting and gaudy 3D controls. It’s not surprising to see these sites – and most recent blogo-bashing of skeuomorphism – focusing on pure visual aspects of such interfaces. On the other hand, as more designers dip their toes into the field, we see an increasing number of badly executed flat designs. The drive-by critics focus, this time around, on the usability blunders.

When you constrain yourself to simple shapes, solid colors and lack of depth, it becomes much easier to sacrifice usability. A hurried design executed from a rich skeuomorphic palette will look garish, but at least there are enough visual tools to help set the different pieces of content apart. With so many textures, gradients and shapes to choose from – and little restraint – one can still create a fairly usable interface. Usable in the sense that a button “looks” like a button – something that can be pressed to initiate some kind of an action. When you directly migrate this hurriedness into flat design, it is much easier to end up with an interface that jeopardizes the structure of your content.

Flat design necessitates deliberate definition of the visual language. How do you convey the logical structure of your content and maintain the logical separation of blocks in the visual realm? How do you convey the behavioral difference between the different blocks? How do you convey the logical importance and relationships between the content blocks?

When everything is a flat rectangle, and you only have a few accent colors to use, you have to exercise a much higher degree of restraint in defining that visual language. When your tool palette is restrained to a very few elements, you cannot afford to use the same element to mean two different things with no significant user-facing distinction. If your buttons and your section headers are the same flat solid-fill magenta rectangle, but the section headers are not tappable, you are not exercising enough restraint in how you’re using your palette.

It is hard work to create a strikingly beautiful, aesthetically pleasing and well-behaving skeuomorphic design. The restraint that the designer must exercise at every turn of the spiral to use the rich variety of tools at his disposal takes attention, energy and time.

It is hard work to create a strikingly beautiful, aesthetically pleasing and well-behaving flat design. The restraint that the designer must exercise at every turn of the spiral to use the small variety of tools at his disposal takes attention, energy and time.

Finally, skeuomorphic and flat are just two extremes of the sameĀ  spectrum. One does not have to go from one extreme to the other. There’s enough space in between to carve out your own, unique niche. The closer you get to one of the ends, the closer you get to the “extreme” choice of your tools – be it the rich palette of skeuomorphism or the small palette of flat. But the size of the palette has nothing to do with the next phase of the spiral – defining, refining and restraining your usage of that palette.