September 17th, 2013

Android performance bits and pieces, part III – sleight of hand

As part of the bigger redesign of the Play Store (cards, cards everywhere) we also created a new tab strip. What’s nice about this strip that it can fit multiple tab titles on larger screens, while also allowing swiping it independently of the view pager itself and selecting tabs all the way at the “other end”. That functionality, however, resulted in a very jarring sliding transition on the underlying content. It can’t really be conveyed with a sequence of screenshots, but if you have a pre-4.3.10 version on your device, go to the landing page of apps or movies, swipe the tab strip all the way to the end and tap on one of the titles. If you are on a fast connection (4G / WiFi), the data for that tab is loaded before view pager completes its sliding transition to select that tab, and the UI thread is swamped with too many pixel operations to be able to both slide the tab and fill its content. The fact that we’re also configuring the pager to load the content of one tab to the left and one tab to the right of the selected one is not helping.

The omni-present Adam Powell suggested waiting until the view pager sliding transition is complete and only then do the data load. That deceptively simple sentence has lead to a rather gnarly, but stable enough solution that has made its way into the latest release of the Play Store. Let’s look at those gnarly details, starting with what happens when you tap a tab title:

  1. Click listener registered on the tab text view calls ViewPager.setCurrentItem
  2. ViewPager calls instantiateItem of its adapter for the newly selected tab and two tabs on the sides.
  3. ViewPager starts the sliding animation and transitions from IDLE to SETTLING state notifying the registered OnPageChangeListener.onPageScrollStateChanged listener callback.
  4. ViewPager notifies the registered OnPageChangeListener.onPageSelected listener callback that a new page has been selected.
  5. ViewPager completes the sliding animation and transitions from SETTLING to IDLE state notifying the registered OnPageChangeListener.onPageScrollStateChanged listener callback.

What we want to do is to set some kind of an indication before step 2 above to defer data loading / display until step 5 has completed. The first gnarliness comes from the sequence itself, where tab selection event happens way after the adapter was requested to instantiate the newly selected tab. If you start data loading in instantiateItem and you’re on a sufficiently fast network (or already have cached data locally), you will end up starting to bind the data for the selected tab well before the sliding transition has completed. I personally would have preferred a slightly different sequence of events, but hey, I’m not going to complain. So…

Given that we “own” our custom tab strip implementation, we can fire a “pre-select” event before calling ViewPager.setCurrentItem. In the handler for that event we propagate the boolean bit to postpone data loading until after step 5 has been completed. In step 2 our adapter checks the value of that bit and does not initiate data loading. In step 5 we go over all deferred tabs and ask each one of them to start data loading.

We end up effectively postponing data load in favor of a much smoother UI response to the tab click. Yes, the data will arrive later than it used to. If you’re on a fast network, the UI will return to a usable state at roughly the same time, as the UI thread completes the sliding transition much faster. If you’re on a slow network, the delay in beginning the data load is not very significant (pager sliding transition completes quite quickly).

There’s a new point of failure here. What happens if we don’t exit that deferred mode? We’re relying on a very specific sequence of events that needs to happen in a very specific order. If, for any reason (not that I’m saying that Adam has bugs in his code, but just saying) we don’t get the SLIDING -> IDLE state transition, the newly selected tab will never have its data loaded. That’s not good. A rather gnarly and brittle (but apparently functioning) hack is to post a delayed Runnable on the UI thread in our pre-select callback handler. If 500ms pass since we’ve posted that Runnable and we didn’t have the IDLE -> SLIDING transition, we force-exit the deferred data load mode for all the tabs. Otherwise, if that transition does happen (step 3 in the sequence above) we cancel the delayed Runnable and let the sequence complete.

This change has been made rather late in the development cycle, and one of the reviewers’ suggestions was to postpone the data binding instead of data loading. The contention is over the UI thread – between tab sliding and tab data binding. Why postpone the data loading then? Start loading the data, and only postpone the binding of the loaded data. Without any guarantee [TM], this is what we’ve added in the development branch. The sequencing of the events is still the same, but instead of deferring the data load in instantiateItem we defer the data binding when we get the data back (from network or local cache). The UI thread is handling the sliding transition, while the background worker threads fetch and massage the data. As the data arrives, we look at the current state of the sequence. If the sequence is complete, we bind the data immediately. If the sequence is not complete, we enter the deferred data binding mode. As the sequence completes, we go over all tabs in that deferred mode and notify them that they can start binding the data.

Gnarly? Check.
Brittle? Check.
Could be better if I knew how to bribe Adam to change ViewPager without breaking a gazillion apps that rely on it? Check.

But hey. It seems to be working. And, nobody said that you will always write nicely looking code that removes jank.

September 17th, 2013

Android performance bits and pieces, part II – use fewer Views

One of Romain Guy‘s hobbies is to file bugs on people to use fewer Views. But you didn’t hear it from me. Anyhow…

The screenshot below shows a full-width banner row with image that cross-fades into solid-color background, with title and (optional) subtitle to its right. In the previous release of the Play Store app we used six child views. Now we use three. What did we remove?

The first one to go was a View that spanned the whole width and height of the parent minus the paddings. That view had the main background fill color set on it. Now that background color is set on the parent view, with one minor caveat. If you call View.setBackgroundColor, it will set the color on the full bounds including the padding. Instead, what you want to do is to use an InsetDrawable and wrap it around a PaintDrawable initialized with your color. I mentioned this before, and I’ll mention it here. Do not use ColorDrawable as a bug on older OS versions will cause it to ignore the insets.

The next one was used to create the cross-fade between the right edge of the image and the solid fill. One option is to tweak the bits of the bitmap itself after you load it from network or local cache. Any kind of device-side image processing (in general so YMMV) is expensive in terms of both memory and CPU, so we opted out for overlaying the right part of the ImageView with a View that had a GradientDrawable set as its background. That drawable was initialized with two end points – full-opacity fill color on the right, and the same fill color with 0x00FFFFFF mask applied to it on the left. You usually don’t want a “random” transparent color used on such a gradient, as the intermediate pixels will not look right across a variety of solid colors. The gradient drawable should be created once in the constructor and then have its setBounds method called from within the onLayout of your custom view group (so that it is properly positioned on its view).

In the latest release we’re achieving the same visual effect using fading edges. Here, you extend the ImageView class and do the following:

  • call setHorizontalFadingEdgeEnabled(true)
  • call setFadingEdgeLength passing the width of the fade area in pixels
  • override getLeftFadingEdgeStrength() to return 0.0f (in our case)
  • override getRightFadingEdgeStrength() to return 1.0f (in our case)
  • override getSolidColor() to return the ARGB value of the matching solid fill to be used in the fading edge part
  • override hasOverlappingRendering() to return true and onSetAlpha(int alpha) to return false

The last two are needed to indicate that the fading edge should be respected during image view fade-in sequence.

Finally, the last view to go was a full-span child that provided light blue highlight visuals for pressed/focused state. Instead, we override the draw method to paint those states explicitly. If isPressed() and isClickable() return true, we call setBounds on the press Drawable and call its draw(Canvas) method. Otherwise, if isFocused() returns true, we do the same for the focus Drawable.

Note that none of this results in improving the overdraw. We’re touching all the pixels the same number of times we used to. However, we’re spending slightly less time inflating the view itself, as well as on measuring and laying out the child views.

September 17th, 2013

Android performance bits and pieces, part I – measure everything

There’s this great saying in Russian – доверяй но проверяй. It means “trust but verify”, except it doesn’t have quite the same visceral impact. Probably because it doesn’t rhyme in English. Anyhow.

We got to spend some time in the latest Play Store release to improve scrolling performance. There’s a lot of rules that you can come up with. The most important one for me is – measure everything. If you don’t measure and just blindly start “optimizing”, you’re just treading water. You can improve some things. Or you can make some things worse. And then there’s another rule – trust nobody. Not even the framework code.

In this release we started to support sale prices. Items that are on sale will show the full price as grey strikethrough text right next to the full price.

My first thought was to add another TextView to all cards. But I was right in the middle of a larger refactoring to reduce the number of views on cards (which is a topic for another entry). So that would’ve been quite awkward. And then I though about spans. We essentially have two spans, one for full price, and another for current price. So I whipped up a CL that switched from TextView.setText(String) to use a SpannableStringBuilder with three spans (color+strikethrough for full price, color for current price). It looked as expected, and after testing a few data scenarios I submitted it for review. Then it went in. And then our testers filed a bug about ANRs on scrolling book streams. And general poor scrolling performance on those streams.

And then I started measuring the actual performance of using SpannableStringBuilder on TextView. Because you know, you kind of rely on framework APIs to not only work correctly, but also be fast. Except when they are not fast.

It turned out that using spans is expensive. And by expensive I mean that the cycle of creating spanned content, setting it on the TextView, measuring and drawing it comes up to around 6ms (see links at the end). For one card. Say you have nine cards visible. That’s 54ms. Just for prices. Which is what you see as freakishly tall blue spikes in the middle screenshot. They kind of kick in every time we populate the next row of cards.

Luckily it was found in time before it went public. We ended up creating a custom view that measures and draws these texts with explicit calls to Layout, FontMetrics and Canvas APIs. The performance went down to what we had before. It’s even slightly better than pure TextView with String content, as we could make a few assumptions that the text is always single-line and never elided. So we’re back to normal. Well, you do see a single spike going above the 16ms threshold in the right screenshot, but we’re not done with improving our stream scrolls.

So the lesson is – measure everything. Trust nothing. Even the framework code that’s been there since API v1.

June 30th, 2013

Responsive everywhere

It’s been a little under two years since I’ve first written about responsive mobile design. A lot has changed since, but one things has remained consistent – we are surrounded by an ever-increasing variety of screen sizes, aspect ratios and form factors, and our users demand a consistent user experience that adapts and responds to the device they are currently using.

As we were getting our first bearings in the world of responsive mobile design, we’ve started with details pages of individual items to flesh out the higher-level design approaches, as well as lower-level implementation details (see here, here and here for more information). In parallel, around last summer it has become painfully clear that our main pages – streams or collections of items – were built on a very rigid and inflexible foundation that had very little ability to scale to the ever-increasing demands from both design and merchandising teams. And so it was that a few people took a long break in an undisclosed location, to emerge a few months later with a new design. A design that is now taking its first baby steps across all Play apps.

Two of our designers – Marco and Owen – were guests at the last week’s “Android Design in Action” show to talk about the various aspects of this design. If you haven’t watched that video, I highly recommend that you do. They have covered a lot of high-level ground, and in this article I’m going to delve a little bit deeper and talk about the pervasive presence of responsiveness across the entire level stack of the new design.

This is the main landing page for the books section in the Play store. Here you can see item collections (such as Summer Deals) represented by a single row of three cards, a personalized collection (Recommended for You) with a single card and an edge-to-edge banner that links to an editorial collection. The Summer Deals cluster is the first example of micro-level responsiveness. When the book title can fit on a single line, the book author is displayed on the second line, while the price always stays in the bottom-right corner. As the author moves to its own separate line, it can span the entire width of the card (3rd book), whereas if it’s on the same line as the price, it gets cut off to prevent content overlap (2nd book).

A second, and much more fundamental example of responsiveness is the Recommended for You cluster that has only a single card. Here our backend indicates that it’s not a collection of books created by our content people. It is a collection of books where each book has a reason (or two) to be included. Specifically, “Snow Crash” has +1’s from two people in my circles. This signal elevates the importance of the item in the stream, and we switch to a more prominent visual representation of the content. Not only we show the title, author, cover and price, but we also show the reason itself, as well as a few lines of the book description in the rest of the space.

Not all signals are created equal. A recommendation based on the +1’s from my social circles is not the same as a recommendation from top / popular lists. One is not necessarily better than the other – as you interact with the stream itself, dismissing recommendations or buying items, that information can be fed into the backend algorithms to tweak the content to each individual user, elevating content based on the signals that are most appropriate for this user. On the client side, this elevation is reflected by displaying more information about the specific item.

Switching between different card layouts is not confined to individual sections – it can also be applied to the full stream as shown above. On right, New movie releases are shown in a three-column layout of mini cards. On left, the full Recommended for You stream uses a two-column layout that allows displaying individual reasons for each item.

The stream is flexible enough to mix three-column and two-column card clusters, based on the content within each section.

Now let’s take a step back and look at what can we do at a slightly higher level – showing the same content on devices of different screen sizes and how the card clusters adapt – or respond, if you will – to such transitions.

This is the same Recommended for You collection on three devices – portrait Nexus 4, portrait Nexus 7 and landscape Nexus 10. As the screen size grows, so does the number of columns and the number of cards shown in the cluster. The data is the same, but the visual representation of it is different. We go from a single item on Nexus 4 to 5 items on Nexus 7 and Nexus 10 (arranged in different templates for the last two). Note how the cluster arrangement visually promotes the very first item in the collection (with larger cover and more space available for the description).

Now the same collection on the same device – Nexus 7 – in portrait and landscape orientation. We switch from three columns and five items to five columns and only three items. The main reason here is that we aim to limit each cluster to the confines of a single screen-sized area. As such, we switch from two card rows in portrait to only one card tow in landscape.

And this is the same collection on Nexus 10 – in portrait and landscape. And yet again the representation of the content responds to the current device configuration – going from a 4-column cluster with one very large card and four smaller ones to a 6-column cluster with one medium-sized card and the same four smaller ones.

An important point to repeat here is that the cluster template (layout of the cards within the specific section) is chosen not only based on the device width, but also on the device height. And so, while portrait Nexus 7 and landscape Nexus 4 use the same number of columns (three), the templates are quite different, as we can fit more content vertically on a portrait Nexus 7. Then, as the template is chosen, we go back into adapting the item presentation based on the specific card – how big the cover is, how big the font for the title is, how many lines of title to show, where to display the reasons (if present), whether to display the item description, etc.

The design is full of these decisions – what to respond to, and how to respond to it. Are we responding to the screen size, are we responding to the specific bits of data, what do we do when we don’t have enough screen estate to show all pieces of data?

This is another example of such responsiveness. The two banners (Lady Antebellum and Daft Punk) are not static images as they were in the previous Play store design. Instead, they are defined by the main image, background fill color, collection title and collection subtitle. At runtime these elements are combined together into a single unit, and the layout of that unit depends on (or responds to) the device size and orientation. If the title needs to go to two lines, it will go to two lines. If the title needs to go to three lines, the entire banner will grow vertically. Then, the title and subtitle are treated as a single unit (that has its inner content left-aligned) which is center-aligned in the space to the right of the image (take a look at the horizontal alignment in the right screen). Finally, the main image itself is displayed in a way that does not take too much width on portrait phones. Compare how much of the Lady Antebellum image is visible in portrait vs. landscape – and how the main interest point of that image is anchored to the same relative point in the overall banner structure.

The store app (and all Play apps in general) are full of these decisions that permeate every level of logical and visual hierarchy – such as switching to two-column layout on search results when we have enough space to do so.

Responsiveness is not something that you add as an afterthought. It needs time and patience to understand and distill. It needs time and patience to introduce to all levels of your design. And, when done properly, it shows respect to your user. Respect to her choice of device and respect to her choice of how to interact with it.