District 9: a review (sort of)

I'll leave a full review to those more adept - besides, there are already plenty of them. But I will say that I thoroughly enjoyed the movie, in large part for its unique premise which somehow contributed to the film's believability for me. One of the best films I've seen this year, no doubt.

I'm one to take notice of computers in movies - what they do, how they are used, etc. I watch intently when a computer screen appears, and always enjoy the filmmakers vision of the interface. For example, the way agents in the Bourne series can quickly drum up information of all sorts is both a contributor to the real-time suspense of the film and a thought provoking illustration of the power of data access (do you really think the NSA can operate as they do in Ultimatum? - I'm quite curious). HCI as a visual and plot element naturally comes into its own in Sci-fi. Despite this, I often find myself disappointed in the computer interfaces that appear in movies. Many movies, like Star Wars for example, completely lack any compelling vision of HCI. For all the droids and hyperdrive ships in the Star Wars galaxy, it is frustratingly unimaginative for all the cockpits to have panels of buttons no more advanced than those in submarines 30 years ago. Also, obsessive gadgetry too often takes the place of truly compelling technological ideas (Bond movies are particularly sickening in this respect, although, to be fair, they are not sci-fi).

The HCI elements in District 9 were well conceived and nicely integrated. There are few scenes featuring computer interaction, with most of the film brooding on the destitution of the slum environment, highlighted by the presence of the insect-like Prawns. But the few moments of HCI are immensely satisfying. The audience first sees a static hologram of the Prawns' home-world, but soon is offered a glimpse at a more interactive holographic interface when the main character, Wikus van de Merwe, takes the helm of the alien command module. Later Wikus climbs into a robotic suit that transforms him into an augmented fighting machine. While he is in the robot, a camera shot of his face with holographic elements hovering in front is revisited as the action unfolds. A particularly cool shot shows that bullets hitting the robot's armor are being represented to the pilot as orange disturbances in the holographic sphere that surrounds Wikus's head. As the robot suit becomes damaged over the course of the firefight, its stuttering movements reflect Wikus's growing exhaustion. In one of the final scenes, we see the Prawn Christopher Johnson taking over the controls of the command module where he expertly manipulates the holographic interface, which it seems to utilize all three dimensions for gesture recognition and allows manual manipulation of the floating holographic elements.

The plot of the movie revolves around the fact that only the Prawns can operate their advanced weapons, which are somehow activated by their DNA. At first I thought this to be a convenient (possibly cheap) plot element and nothing more. However, I think there is something very appealing about the idea that the Prawn technology is completely continuous with their biology. This vision of computer interaction is a coherent theme of the movie - we see it again when Wikus discovers that he must press his alien hand into a blob of goo to activate the command module's interface. Likewise, the robot suit autonomously kills a group of Nigerian gangsters before it offers Wikus, who at this point is half alien, the pilot seat. Wikus becomes increasingly competent with the suit during the combat scenes which parallels his accelerating transition to the alien form. Clearly these ideas are appealing for a disciple of extended mind philosophy, like myself, and for those who think of technology as augmentations of our biological capabilities.

Why is cool HCI such a great movie element? Interacting with computers is a universal experience (at least among those watching movies), and there is an easily appreciable thrill when a character casually interacts with a computer in a seamless manner. Our interactions with computers at the present time are far from seamless, but Hollywood's visions of computer interaction can play a role in inspiring thought about what computer interaction should be like. And sometimes the interfaces just look so damn good. All told, I'd say this movie should be considered a notable vision of HCI, probably second only to Minority Report* in that category.



* Minority Report was the first film to my knowledge to bring in HCI scientists, including Jason Lanier, to consult on the computer technologies depicted in the film. I'd be interested to hear who gave the creative direction in D-9.

---

District 9 Official Trailer

Review of the film from New Scientist focusing on the science of the film.

[pdf] A Survey of Human-Computer Interaction Design in Science Fiction Movies (2008)

Eyes as windows into the mind

http://thirtysixthspan.com/openEyes/eye_tracking_chess.mpg

Maybe not the most gripping video, but it struck me as great demo of a key principle in philosophy of extended mind: that our visual system does not work by constructing an internal representation of the world, but rather references the visual scene on a as-needed basis, allowing the external world to serve as the best representation of itself. You can actually see thoughts tracing themselves out on the board and the ongoing process of visual exploration that accompanies a computational task like playing chess.

That the mind does not form explicit internal representations of the world is fairly uncontroversial in cognitive psychology, but the extended mind research program actually takes the notion a step further. It suggests that the exploratory way in which we reference the world in cognitive activities does not just show that we eschew internal representations when external ones will work just as well for much less cognitive cost. Instead we should understand our involvement in the world as part and parcel of cognitive activity - that the mind depends on the world to accomplish cognitive tasks and exploits the world not just for cognitively cheap representation, but also as an extended working memory. Consider solving a calculus problem on paper. This activity is an iterative process consisting of writing, thinking about what you have written, and writing again, etc. The paper and pencil are not just aids to your mind because in many cases you couldn't solve the problem without the pencil and paper, by just thought alone. They are active participants in the cognitive process that eventually yields an answer. Andy Clark calls this "active externalism," and characterizes it as consisting in cognitive feedback loops that link brain, body, and world into a single coherent, temporally extended, cognitive process.

This is a compelling notion for human computer interaction. Eye tracking has been of great interest recently: we generally have to look at an object on a computer screen before we can manipulate or interact with it, so presumably we can greatly improve the efficiency and naturalness of the interface if we are able to take eye movement into account. This interest has largely focused on the potential use of eyes as a pointing device - a replacement for the mouse. I propose that the utility of eye tracking could be much greater than just this if we are to take the extended mind seriously. After all, vision is our primary sensory modality, and, as such, it figures prominently in cognitive activities as the world-to-brain component of the cognitive loop. Perhaps the eyes can tell us not just what we are thinking about on the screen, but instead how we are thinking about it. By tracking eye movements over time during a cognitive activity, a computer may be able to key in on elements of the thought process as it unfolds.

It's true that the notion of computers reading our thoughts is somewhat vague, so I pose it as only a suggestion. Much must be learned before anything of the sort could be fleshed out. Nonetheless, it seems, in principle at least, that keying in on eye movements will, in a manner of speaking, bring the computer to our cognitive level and allow it to tightly embed itself in our cognitive loops. To put it differently, we should hope for a day when our cognitive loops are more than just screen-deep.

I'll add as an aside another implication of extended mind for HCI. If we take the notion of feedback loops comprising cognitive activities seriously, then we have the basis for a new characterization of interface design. The traditional view is that interfaces mediates the interaction of human and computer. While this is no doubt true, the notion of extended mind begs for an amendment. The design of computer interfaces could be seen as the engineering of an external component that, when it enters a cognitive loop with the user agent, becomes an active piece of the user's thought stream. This extends as well to all sorts of design - from formatting and layout to product design. It is in this way that design can be understood as the creation of an external module of mind. We don't need neural implants to incorporate computers into our thoughts - active externalism accomplishes that through our senses.

(This one reminds me of a video game, which is obviously cool. And augmented reality applications come to mind.)

---

Clark, A. (1997). Being there: Putting brain, body and world together again. Cambridge, MA: MIT Press.

Clark, A. (2008). Supersizing the Mind: Embodiment, Action, and Cognitive Extension. New York: Oxford UP.

Johnson, S. (1999). Interface culture: How new technology transforms the way we create and communicate. San Francisco: HarperEdge.