As I was watching “Prometheus” again after the interview with Shaun Yue, paying closer attention to how human-computer interaction is presented in the various systems, a particular point that he made during the interview stood out:

When you look inside NASA control centers and spacecraft, the graphics do not appear very futuristic. Same if you go inside an air traffic control room and nuclear submarines – all of these very mission-critical settings. The graphics don’t become more sophisticated. If anything, they become simpler and more basic. You’re dealing with such important data that has to be processed.

In one scene, Shaw hobbles over to the medipod to initiate a Caesarian to remove the trilobite from her body. She presses the red “Emergency” label on the screen (it’s not clear if this label is red before she touched it, and the earlier introduction of the equipment does not provide a clear view of the default presentation). In response, the automated voice says “Emergency procedure initiated, please verbally state the nature of your injury.” The first part is reinforcing the acknowledgment of the selection, and the second part guides Shaw to state the next command verbally. As Shaw says “I need Caesarian” the voice replies “Error, this medpod is calibrated for male patients only. It does not offer the procedure you have requested. Please seek medical assistance elsewhere.” She then initiates manual selection mode, making quick taps on the selector wheel and saying “Surgery. Abdominal. Penetrating injuries. Foreign body. Initiate.” which completes the selection phase and opens the pod.

Christopher Noessel over at the “Make It So” blog dissects this sequence:

Aside from the pointless “tension-building” wrong-gender plot point, there are still interface issues with this step. Why does she need to press the emergency button in the first place? The pod has a voice interface. Why can’t she just shout “Emergency!” or even better, “Help me!” Isn’t that more suited to an emergency situation? Why is a menu of procedures the default main screen? Shouldn’t it be a prompt to speak, and have the menu there for mute people or if silence is called for? And shouldn’t it provide a type-ahead control rather than a multi-facet selection list?
…
Why does Shaw need to speak in this stilted speech? In a panicked or medical emergency situation, proper computer syntax should be the last thing on a user’s mind. Let the patient shout the information however they need to, like “I’ve got an alien in my abdomen! I need it to be surgically removed now!”

I disagree. Natural language processing is a daunting task on its own. While false interpretation of voice commands is annoying when you dictate a memo, incorrectly interpreting a medical emergency from the words of a distressed patient can have quick and lethal implications. Particularly in a system clearly designed for a particular person – Peter Weyland.

After the first voice-based interaction, as the system correctly recognizes an unsupported precisely-defined medical procedure, it does not presume to offer alternatives, or attempt to parse additional voice descriptions of Shaw’s situation. Instead, the manual selection interface offers an impressively quick way for Shaw to navigate the tree of available options and choose the specific supported procedure. And while the first impression is that Shaw is continuing to issue voice commands to navigate this menu, the second part of the sequence clearly shows her using the on-screen selector wheel – merely commenting her selections for the viewer’s sake.

The remarkable efficiency of this selection requires a certain degree of proficiency interacting with computer systems – which is implicitly expected from all passengers aboard this scientific expedition. The speed of this interaction is even more remarkable after watching the 3D re-release of “Jurassic Park” and its infamous “It’s a UNIX system! I know this!” scene, where it takes Lex 56 seconds to navigate the three-dimensional flyover rendering of the file system to locate the component that locks the laboratory doors.

While the sequence highlights the novelty of the interface, it also underscores the poor usability of the system even if you are familiar with it – with unnecessarily long transitions that follow each particular selection, and grotesquely zooming in on a very small part of the overall hierarchy.

It takes a more focused watching of “Prometheus” to see how pervasive computer screens are aboard the ship – on the bridge, in the science lab, in the airlock chamber and around the medipod. And yet the interaction is minimal. Actually, the only explicit interaction seems to be when Charlie turns on the projection cube and proceeds to swipe the hologram projections of cave paintings. The rest is relegated to pure visualization – such as the initial descent path of the ship, the DNA sequence matching or the diagnostic scan of Shaw’s abdomen inside the medipod. It’s as if the software is written to know what needs to be presented at any point in time without lengthy explicit instructions from the crew.

When you don’t have complex interaction patterns, you have fewer points of failure. While a crash in the DNA sequence matcher is a marginal waste of time, a crash in the descent tracker or the medipod is a matter of life and death. Removing interaction complexity or, in fact – removing the need for the explicit interaction once the intent is understood – is a key part in how the human-computer interaction is presented throughout “Prometheus”.

There are two systems worth noting for their uncommonly complex visualization layer. The first is the Holotable on the bridge, and the second is the Engineers’ spacecraft control room – the Orrery. Both are featured prominently throughout the movie, but yet both are displayed as low-risk systems.

While the Holotable displays the visual structure of the dome as scanned by the laser “pups”, what happens if some part of that system crashes? Not much probably. The crew would miss some part of the scan, or see incorrect position of the probes, or miss the Engineers’ spacecraft. In fact, the sequence where Janek realizes that the underground structure is a spacecraft shows quite clearly that it was visible in the scan the entire time after David has opened its doors – even as he turned off his shoulder camera. If anything, failing to see the spacecraft in the rendering is a human failure.

The Orrery itself is an interesting experimentation in visualizing planetary systems. However, it seems to purposefully stay as far away as possible from presenting the “quantum leap” interfaces that might be just within our reach if only we stop focusing on the next incremental improvements to the existing ways we interact with computers. If anything, the overhead shots of the deck Engineer, combined with booting the system by playing a flute portray this interaction as playing a musical instrument. If you’re interested in a deeper analysis of this navigational interface, click over to the “Make It So” blog.

Overall, the visual prominence of holographic stereo projections – mainly highlighted in the Holotable and the Orrery sequences –  overshadows an interesting take on portraying human-computer interactions in movies. Computer screens are pervasive aboard the ship, and yet the amount of actual interaction required to accomplish the specific tasks is minimal.

Natural language voice input for a predefined set of tasks, predictive surfacing of relevant information, concise visual representation of salient data and simple touch-based interaction as the fallback for cases where incorrect interpretation of the user intent has life-threatening implications. These are some great ideas that deserve exploration in the real-world software.

When we are surrounded by glowing screens wherever we go, what does it take to create believable and yet attractive computer interfaces for big-budget movie productions? In this conversation Shaun Yue talks about realism in representing the human-computer interaction in “Skyfall” and “Prometheus”, what does it take to place hundreds of live monitors on the set, his work on game cinematics for “Crysis” and “Call of Duty”, and how we may be interacting with information in the next few decades.

Kirill: Tell us a little bit about yourself.

Shaun: I’m originally from Melbourne, Australia. I studied Multimedia Design at Swinburne University; it was a mix of web, animation, video and graphic design. During that time I worked as a web designer, and this web agency shared office space with a film production company, Exit Films, who made commercials and music videos. I was really interested in the work they were doing, so during the time I was working at this web company – I helped them do motion graphics, and also worked as a director’s assistant. I tried to get on set and see the whole process of what it takes to make a film.

In hindsight I was so lucky with the filmmakers I worked with – people like Garth Davis, Glendyn Ivin, Greig Fraser, they’ve subsequently gone on to achieve so much. It was an experience where I knew I had so much to learn; and was lucky to be in the right place at the right time.

After that I kept working mainly in animated commercials, and was then employed by the Australian Centre for the Moving Image which is a cultural institution which exhibits moving image in all forms, from film to games and contemporary art. I did all of their motion graphics promotional work. It was an interesting break from commercial clients, and great to explore a wide breadth of the moving image cultures.

In 2006 I was lucky enough to win a grant with the British Council to come to London and meet with some British designers who I admired. Through that I met Toby Glover who designed “Batman Begins” and a whole host of other screen graphics for films. We got along quite well, and worked together on some projects, and a year later he said “The Dark Knight” was going to be made, and asked if I would be interested, and I said “Of course!”

Kirill: Were you always interested in working in movies, or did it come together like that?

Shaun: I always loved watching movies and loved sci-fi. Whenever I saw a computer on film, I always felt strange. It was one of those things where I felt that maybe I could do that better.

Kirill: Jumping a little bit forward as you talk about making things “better”. The “Alien” franchise started in 1979, and you joined it working on “Prometheus”. How do you define “better”? Is it in terms of raw processing power that you have at your fingertips and how intricate you can get? Is it about improving the interaction design, the visual design, or something else?

Shaun: It wasn’t specifically a technical software solution to making it look more sophisticated, for example I think the computers in Alien are great, they have a very functional aesthetic. But so often screen interaction in film appears very naive, unbelievable as a computer user. And now that I’ve worked on a couple of movies, I can understand why it is a little bit fictitious or a little bit over-dramatized. But back then when I was watching movies, I was thinking that that’s not how computers work, and if I did it, I’d do it properly.

Kirill: And by “everyone” you mean people who are actually into computers…

Shaun: Right, the biggest thing about film is that they have a very broad audience. There’s often dramatic storytelling reasons to present something as it finally appears, rather than being authentic to technology. Balancing design authenticity with narrative concerns is probably the greatest challenge of the job.

Kirill: Is it a back-and-forth process on the set, defining how “realistic” the interactions with computers are? Do you ever get feedback that what happens on the computer screens is too boring?

Shaun: It depends on the movie. For example, in “Dark Knight” all the screen designs – which were designed with Toby Glover and Andrew Booth – are all DOS-based, engineering-based, very realistic, often sparse. It was about believability, and in that case the director never said that it’s too boring and let’s jazz it up. On the other side, “Prometheus” and “Skyfall” are very much about how can we show something that is beyond what a normal person is used to. “Prometheus” is set in the future, and it has to be more than what viewers are used to now, similarly with “Skyfall”, an element of a higher level of computers that the public doesn’t usually see. Obviously it’s a little bit of fantasy, but it was an important aspect of how the directors wanted to present computers in those films.

Continue reading »

In this installment of the “In Motion” series I talk with Tim Grimes about his work on the movie “Last Night”, on switching between working on feature films and TV series, on what changes digital productions bring into the world of crafting physical sets and his appreciation of film as a medium.

Kirill: Tell us about yourself and the path that lead you to become a production designer.

Tim: I started off as a production assistant in an office. I was in bands for a while, and then I moved out of New York, and four months later I moved back and my intent was to somehow get into the film business. I had no idea how that would happen, I was 27 at the time and I ended up getting a job as a waiter. I used to work at “Kim’s Video” in New York and I met this guy who came in and asked me if I wanted to be a production assistant on this film that ended up under the name “Return to Paradise”.

I was interested in the film business, but I didn’t really know what I wanted to do. After about a year and a half working as a production assistant on a bunch of TV shows, like “Spin City” and “The Corruptor” with Chow Yun-Fat, I became a property person. I did that for a long time, from 1999 to about 2004, and I was really fortunate to have worked on the second season of “The Sopranos”. I was on a bunch of films and met property master Peter Gelfman who ended up being my boss for a long time. This lead me to work with Kevin Thompson, who is a production designer and a friend of mine. He’s amazing and really talented; he just did “Bourne Legacy” with Tony Gilroy [director].

And through Kevin I met Harris Savides who just passed away. He was my mentor, and almost like my father. We worked together on “Birth” directed by Jonathan Glazer, with Harris shooting, Kevin designing and me doing props under Peter Gelfman. That’s how I met Harris Savides and we became close friends. He worked with David Fincher, Sophia Coppola and Woody Allen, he was very picky about the jobs that he would take. He knew that I had a desire to move on, to do something more challenging, to do the next thing. He was getting ready to do “Last Days” with Gus Van Sant and he asked me during the re-shoots of “Birth” if I was interested in joining the crew. I ended up getting the job as art director, which was basically production designer. I was the head of art department, and it was very stressful and exciting. So he basically gave me my first shot, and that’s how I got into production design.

I then went on to do smaller films, continuing to do props on the side. Then I got an agent and that opened a bunch of doors, and one thing led to another. I tried to use the Harris Savides model to not just take any job, but doing things that interested me, jobs that I thought would stand out or I’d learn a lot from, different things that wouldn’t pigeonhole me into a sort of category. That thing can happen pretty easily. And that’s how I took “Last Night”. I had never done a romantic drama; it’s not something that completely interests me and I wouldn’t do them all the time, but I definitely wanted to do one and I thought that Massy Tadjedin [director/writer] was an interesting writer. I liked her take on this simple European-style romantic drama. That’s what attracted me to the project.

Kirill: As you moved to assume larger responsibilities, do you find yourself going back to do the small things on your sets?

Tim: “Last Night” had the budget of $7M, which is not that big. I was heavily involved in it. We didn’t do a lot of building, it was almost all locations. We build this small set for Paris which ended up with flashback photographs that two of the characters look at at the end of the film. It was work on transforming locations. I’m very hands on, I’m there helping the decorator decide what we’re putting up. I’m very much involved on a project like that.

Continue reading »

“Moonrise Kingdom” is by far the most enchanting and charming movie that I saw in 2012. It is a great pleasure to have the opportunity to host Gerald Sullivan, the art director of this wonderful production, and to ask him a few questions about his craft, and his work on the movie.

Kirill: Tell us a little bit about yourself.

Gerald: I am a graduate of the Southern California Institute of Architecture, SCI-ARC. I began working as a set designer in ’95 without much prior knowledge of film making. Since then I’ve had the good fortune of working on a variety of films, gaining insight from many great production designers and working with some of the industries most highly regarded directors.

Kirill: In your experience, what’s the role of an art director in the overall production, and what skills do you bring to the table?

Gerald: An art director takes on many roles throughout the production. We need to be shape shifters. Initially we need to be able to conceptualize the scenery needed to tell a certain story. Budget and schedule need to be established. The art director has to be able to react, adjust and respond to inevitable changes through out each project. We are in constant contact with the assistant directors, the UPM [unit production manager], construction, set decoration, SPFX [special effects], the Director of Photography, the key gaffer, the key grip, etc. Every art director I know has an appreciation for art, architecture, decoration, and the history of each. The best understand we must be learning more all the time, constantly expanding our knowledge, what we bring to the table.

Kirill: From set decorator to art director to production designer. Is it a natural progression, or just one path to follow?

Gerald: No natural progression, no one path to follow. Doesn’t need be a progression that aims toward, or ends up at, production designer. Whatever your best at, and take pride in doing, that’s where you should be.

Kirill: How did you end up working on “Moonrise Kingdom”?

Gerald: I had worked with Adam Stockhausen on a film the previous summer in Michigan. We got along well. I am a big fan of Wes’s work. Adam had worked with Wes on “Darjeeling Express” and a few commercials as an art director. When Adam let me know he was going to work on “Moonrise” as production designer, I let him know I was interested in art directing.

On the set of Bishop family house. Photography by Niko Tavernise, courtesy of Gerald Sullivan.

Outside shot of Bishop family house. This extension was built to match the house and provide the director with what he needed for each scene. It also camouflaged a non-period sun room.

Continue reading »