what do video games see? an intervention

philosophical context

For two millennia, philosophers in the west insisted that technology had nothing to do with the “examined life.”  Now, after nearly a century of philosophical “disinterest” in machine guns and mass media, in Taylorization and nuclear proliferation, in Agent Orange and Silent Spring, there is a growing body of philosophers who insist that their discipline must finally become as introspective as it has always claimed to be.

That rigorous soul-searching is long overdue, and technology’s presence has already brought surprising changes to philosophy and its practice. In truth though, maybe things have changed less than it seems: After all, Plato’s philosophy is itself immutably engaged in technological experimentation of its own: The alphabet had existed for hundreds of years, but was really a tool for wine merchants and shepherds.

Perhaps rethinking technology as a proper topic for philosophy will lead us to ask:  Are books really the only way to philosophize?

Philosophers themselves must become builders and makers.  Pace Plato, they must get their hands dirty:  philosophers must tinker.  We see the idea take shape in Alfonso Lingis’ “carpentry of things” and Graham Harman’s “object-oriented philosophy;” it is implied in the litanies of Latour; it is realized lately in the media-philosophical code of Manovich and Bogost and the remixes of DJ Spooky and Mark Amerika.


Screenshot of Super Punch-Out
The game from the player’s hybrid third-person / first-person point of view.

Below, images of three players, all playing Super Punch-Out (Nintendo, 1994). Compiled over three separate sessions. For more information, see method, below.


When we play games, we build worlds:  Tiny, trivial, ephemeral worlds.  These worlds require a lot of effort to sustain, though: The outside always wants in. So players invest those worlds with their full attention, and in so doing, intentionally shut the rest of us out.  Sitting next to them on the couch, or the Metro, we can see only the backs of their heads.  What are they doing in there?


With this modest intervention, we repurposed a small teleprompter to serve as the display for a home-brew Nintendo emulator running on a Raspberry Pi 3B+.  We placed a DSLR immediately behind the teleprompter’s screen, so that the camera’s lens was invisible to the player herself, in spite of the fact that she is staring right at it. While she concentrates on the game, which is displayed on the surface of the teleprompter (as though it were a TV screen), the camera is able to quietly insert itself between the player and the game. At certain moments, a script is triggered on the Raspberry Pi, which grabs an image from the DSLR and stores it, capturing “impossible” photos of a once-secret relationship.


The photographs of these handsome players (students from Arcade Theory S18) are not only pleasing to see, they also suggest that our engagement with video games — even the old-school, unrealistic 16-bit games of the past — is far richer and more humane than we might have imagined.  Game developers, for example, typically assume that a player “interacts” with a game with a limited number of inputs: Up, Down, Left, Right, and Jump. When we imagine a child playing a video game, we are even less charitable, picturing a nearly immobilized kid with a blank stare on her face.  But clearly, our games see much more than this.