Maybe not the most gripping video, but it struck me as great demo of a key principle in philosophy of extended mind: that our visual system does not work by constructing an internal representation of the world, but rather references the visual scene on a as-needed basis, allowing the external world to serve as the best representation of itself. You can actually see thoughts tracing themselves out on the board and the ongoing process of visual exploration that accompanies a computational task like playing chess.
That the mind does not form explicit internal representations of the world is fairly uncontroversial in cognitive psychology, but the extended mind research program actually takes the notion a step further. It suggests that the exploratory way in which we reference the world in cognitive activities does not just show that we eschew internal representations when external ones will work just as well for much less cognitive cost. Instead we should understand our involvement in the world as part and parcel of cognitive activity - that the mind depends on the world to accomplish cognitive tasks and exploits the world not just for cognitively cheap representation, but also as an extended working memory. Consider solving a calculus problem on paper. This activity is an iterative process consisting of writing, thinking about what you have written, and writing again, etc. The paper and pencil are not just aids to your mind because in many cases you couldn't solve the problem without the pencil and paper, by just thought alone. They are active participants in the cognitive process that eventually yields an answer. Andy Clark calls this "active externalism," and characterizes it as consisting in cognitive feedback loops that link brain, body, and world into a single coherent, temporally extended, cognitive process.
This is a compelling notion for human computer interaction. Eye tracking has been of great interest recently: we generally have to look at an object on a computer screen before we can manipulate or interact with it, so presumably we can greatly improve the efficiency and naturalness of the interface if we are able to take eye movement into account. This interest has largely focused on the potential use of eyes as a pointing device - a replacement for the mouse. I propose that the utility of eye tracking could be much greater than just this if we are to take the extended mind seriously. After all, vision is our primary sensory modality, and, as such, it figures prominently in cognitive activities as the world-to-brain component of the cognitive loop. Perhaps the eyes can tell us not just what we are thinking about on the screen, but instead how we are thinking about it. By tracking eye movements over time during a cognitive activity, a computer may be able to key in on elements of the thought process as it unfolds.
It's true that the notion of computers reading our thoughts is somewhat vague, so I pose it as only a suggestion. Much must be learned before anything of the sort could be fleshed out. Nonetheless, it seems, in principle at least, that keying in on eye movements will, in a manner of speaking, bring the computer to our cognitive level and allow it to tightly embed itself in our cognitive loops. To put it differently, we should hope for a day when our cognitive loops are more than just screen-deep.
I'll add as an aside another implication of extended mind for HCI. If we take the notion of feedback loops comprising cognitive activities seriously, then we have the basis for a new characterization of interface design. The traditional view is that interfaces mediates the interaction of human and computer. While this is no doubt true, the notion of extended mind begs for an amendment. The design of computer interfaces could be seen as the engineering of an external component that, when it enters a cognitive loop with the user agent, becomes an active piece of the user's thought stream. This extends as well to all sorts of design - from formatting and layout to product design. It is in this way that design can be understood as the creation of an external module of mind. We don't need neural implants to incorporate computers into our thoughts - active externalism accomplishes that through our senses.
(This one reminds me of a video game, which is obviously cool. And augmented reality applications come to mind.)
---
Clark, A. (1997). Being there: Putting brain, body and world together again. Cambridge, MA: MIT Press.
Clark, A. (2008). Supersizing the Mind: Embodiment, Action, and Cognitive Extension. New York: Oxford UP.
Johnson, S. (1999). Interface culture: How new technology transforms the way we create and communicate. San Francisco: HarperEdge.
