A History of Haptics: Electric Eels to an Ultimate Display

At HaptX, we periodically speak to leading experts in haptic technology, its history, and its applications. Greg Bilsland, Sr. Communications Manager at Haptx, recently spoke to David Parisi, an Associate Professor of Emerging Media at College of Charleston. His research “investigates the myriad interfaces between bodies and media technologies, with a particular emphasis on physical interactions with digital media.” He and several colleagues, including Mark Paterson and Jason E. Archer, recently curated a special issue in the journal New Media & Society focused on haptics, which is now available for purchase through Sage Journals. David also helps maintain a website about haptic media studies and has a book about the history of haptics called Archaeologies of Touch: Interfacing with Haptics from Electricity to Computing, coming out in early 2018 with the University of Minnesota Press.

A History of Haptics: The Birth of a New Language

Greg Bilsland. Thanks for taking the time to talk, David. Let’s start with a little about your background. How did you become interested in the history of haptics, as well as haptic technology and media?

black and white portrait of david parisi

David Parisi, Associate Professor of Emerging Media at College of Charleston

David Parisi. I’ve been thinking about this subject for nearly twenty years at this point.

Part of the answer to this question comes from my experience growing up with a paraplegic sibling, which prompted me to think about how the body operates as an information-transmission network—without us being aware of it, our bodies are constantly sending and receiving signals. But sometimes, these networks become damaged, and they fail to relay commands to our limbs, with the senses failing to send their complex data back to the brain.

I was in that headspace when I first held a vibrating video game controller—this was around 1999—and I’d been deeply immersed in the writings of media theorist Marshall McLuhan, who wrote a lot about touch. When I was playing Metal Gear Solid, I felt the controller rumble, and it struck me immediately as a new language, a new way of communicating information through touch. I didn’t quite know what to do with it at the time, but as I started reading more, my philosophical interest matched up with an epistemological question that’s been longstanding in media theory: how do we know the world through our senses? And how do media technologies, by extending and amplifying our senses, participate in that process?

“I was playing Metal Gear Solid, I felt the controller rumble, and it struck me immediately as a new language, a new way of communicating information through touch.”

As I started researching, learned a lot about the efforts during the 1990s to incorporate haptics into VR, but interest in haptics tech seemed to fizzle out as enthusiasm for VR subsided. When I started my doctoral work at NYU in 2002, I was trying to understand the contemporary, real-world applications of haptics technology. It was a hard thing to study in the early 2000s— you couldn’t really witness it in the wild because most of the applications at the time were theoretical, or deployed in very limited, specialist contexts.

I eventually realized we were seeing the birth of a new field, and I began to focus on  historically contextualizing the emergence of what Mandayam Srinivasan and Cagatay Basdogan called “the new discipline of Computer Haptics.” I was trying to uncover its roots, trace the disciplines that it grew out of, and understand its future direction. What’s interesting to me is how we perpetually tell stories about how haptics technology is just about to come into its own—on the verge of totally transforming the way we interact with machines and with each other. These sorts of predictions, often advanced in popular press accounts of haptics, go all the way back to the 1990s. So part of the challenge for me is to separate the hype around the tech from its real-world effects. Haptics is an obscure term for a lot of people, but at the same time, it’s a technology that’s existed in the world for a long time, not just in design labs but also in your pocket—in the vibrating alerts used in cell phones. All these bits and pieces of haptic technology have already arrived. For example, there are over a billion videogame controllers currently in the market, and that’s a big deal—it suggests that haptics tech has already had a major impact on the way that we interact with virtual worlds

image of the novint falcon haptic device

The Novint Falcon provided force feedback but was limited in its versatility. Source: Wired.com

I still maintain a healthy skepticism with this new wave of haptics because these periods of discovery tend to be recurring. I try to understand how people talk about the technology and look for continuities in the stories being told around the devices. It’s one reason I liked Novint’s project, the Falcon, which I wrote about in Reach In and Feel Something: On the Strategic Reconstruction of Touch in Virtual Space.

Haptic Media Studies Explained

Greg Bilsland: Can you define what Haptic Media Studies is or speak a little more to how the field relates to the history of haptics?

David Parisi: This is a fun question to answer because the field right now is the special issue. That article put a call out to people who are doing touch-related research in haptics situated in n the Media Studies tradition. We wanted to know how touch informed their research. We had a sense there was an increased interest among academics in this area.

So at this point, it’s more of an opening gambit to get people talking about what haptic media studies means. The topics in the special issue really cover a lot of potential dimensions of the field. For example, Rachel Plotnick’s excellent piece looks at the relationship between buttons and knobs and examines then how that relationship changes with the move to touchscreen-based interfacing. Other articles, such as Mark Paterson’s manifesto and Gerard Goggin’s piece on haptics and visual impairment, approach haptic media from the perspective of disability. So we conceptualized ‘haptic media’ fairly broadly, enough that it includes devices like Apple Watch along with Q*bert arcade cabinets. But at the same time, due to the constraints of journal publishing, we were limited to six articles, plus the two manifestos that serve as a way of framing the field. On the blog, we’ve identified some areas of research to lay a blueprint for what can be expanded for historical and empirical research, and we hope that this will be an ongoing conversation that attracts a greater diversity of voices as it grows.

A Haptic Moment

Greg Bilsland: Tell me a little more about the idea of the “haptic moment” you describe in the call of papers for the special issue. What do you think it means for our relationship with technology?

David Parisi: The phrase “haptic moment” comes from Mark Paterson, whose book The Senses of Touch: Haptics, Affects, and Technologies (2007) has been a foundational work for those of us interested in the relationship between touch and tech. We had some spirited conversations among the editors (Mark, Jason, and myself) debating whether this is a haptic moment or if this framing succumbs to precisely the sort of technofetishism that we’re trying to push back against. And it’s a question of how we evaluate something that’s truly new. Mark recognizes that something has changed with this recent wave in VR and the haptic work being done in touchscreens, and he also acknowledges the growing interest in haptics within academia, some in literature and some in media technology. Mark is really trying to reconcile the interest across these two areas. The question is whether this is something unique to this moment or whether it’s the accumulation of previous research.

“Mark recognizes that something has changed with this recent wave in VR and the haptic work being done in touchscreens, and he also acknowledges the growing interest in haptics within academia…”

I tend to stress the historicity of making claims like this. Back in 1999, a German journalist cited by the medical historian Robert Jütte asked if we’ve entered a “haptic age.” So this is now almost twenty years that we’ve been grappling with this question. The designer Hiro Iwata wrote a retrospective on haptics research the 1990s, and he termed the “epoch of haptic interface.” In the past, we’ve seen people tend to identify the same wave or moment. Is this a new wave or part of that continuity? Are those seeds just beginning to bear fruit? And what are the similarities and differences between these narratives about haptics?

Greg Bilsland: And related–do you think the technology of touch has the potential to bring us closer together or that it will drive people further apart?

David Parisi: This is such a classic question we ask about new communication technologies. When I’m teaching this sort of thing in class, it’s the kind of question I ask my students. Let’s look at the smartphone. It’s a technology intended to make us feel closer to people who are far away, but when we’re using one in proximity to people we’re with, it can also drive a wedge between us. Haptics is hard to talk about in this context because it’s really application-dependent at this stage—and still very much in a protean phase, where its conventions of use are still very much up in the air.

CUTECIRCUIT’s Hug Shirt, for example, was a technology, like the smartphone, intended to help people who were physically distant erase and overcome that distance through affective haptics. By capturing, storing, and transmitting your hug to someone on the other end of the network, the Hug Shirt offered a substitute for a real hug. And it seems laughable ten years later that these little actuators embedded in the shirt could function as a stand-in for the rich and wonderful complexity of a real, physical hug—but the impulse of bringing people closer together through technology has been fundamental to shaping the development of media. After all, doesn’t everyone want to get hugged?

image of model wearing hug shirt from CUTECIRCUIT

The Hug Shirt™ has “sensors that feel the strength, duration, and location of the touch, the skin warmth and the heartbeat rate of the sender and actuators that recreate the sensation of touch, warmth and emotion of the hug to the Hug Shirt of the distant loved one.”

So with haptics, the effect on our relationship with technology really depends on how it will be implemented and what sectors will it impact. At the conclusion of Archaeologies of Touch, I was asking the “what if” question. What if AxonVR delivers a full-body haptic suit that ends up having near-universal adoption? What industries will it impact? How will that change the way that we relate to each other if virtual touch begins to displace physical touch? What effects will that profoundly important technology have? What new vulnerabilities will an embrace of haptics expose us to? You could eliminate the need for travel if you could send your sensor-enabled avatar to a remote destination as a surrogate for your physical body. It could drastically change the way we work—machine haptics has the possibility to accelerate automation and the replacement of humans by machines. The transformations are potentially quite far-reaching—and ultimately, I think, the enduring consequence of the technology will be in the way it re-shapes the category of touch itself.

Archaeologies of Touch

Greg Bilsland: You have a book called Archaeologies of Touch coming out this February. Who is it for? What does it share about the history of haptics?

David Parisi: Though the book’s primary audience is academics the field of Media Studies, I hope that non-specialists will find it valuable too. There’s some coded language that might be a bit arcane to non-initiated—but there are also a lot of compelling episodes in the history of touch technology going back to the early 1700s, which I found interesting and fun to write about, including some long-neglected stories that prefigure the emergence of today’s digital haptic techs.

The use of the term ‘archaeology’ in the title might be a little strange to people who don’t spend a lot of time in the swamps of media history, so let me unpack that for a second. It’s a direct reference to the academic field of ‘media archaeology’—a way of doing media history that tries to understand how the past continues to actively shape the present, in part by a metaphorical ‘digging up’ of old technologies. Archaeologies of Touch opens by looking back at electrical machines from in the 1700s and drawing a line between the shocks induced by those machines and present-day haptics devices. We tend to forget that there was about a 100-year period where people didn’t really know what to do with electricity. It had no practical application; it was just something to play with. You could do parlor tricks, shocking people for fun and profit. Experimenters would travel around with electric eels or electrostatic batteries—for a small fee, you could touch the eel or the battery and get a really curious shock. So I want to bring forgotten episodes like this into dialogue with our contemporary haptic moment, to open up productive points of contact between past and present.

image of the cover of Archaeologies of Touch

David Parisi’s Archaologies of Touch comes out February 2018. Source: University of Minnesota Press.

Greg Bilsland: Have you read Ready Player One? How do you think the movie next year will inform haptic media studies and consumers perceptions of the technology?

David Parisi: I have to confess I didn’t make it all of the way through—outside of graphic novels, I don’t read a lot of fiction these days. But I came at haptics through the filter of the 80s and 90s cyberpunk fiction—William Gibson and Bruce Sterling and Neal Stephenson and Kathy Acker—which Ready Player One is sort of an homage to. For me, as a child of the 80s and 90s, I’ve been thinking about how the body migrates into virtual space for quite a while now, so I have a preference for other cultural touchstones I prefer. And I’d also include Howard Rheingold’s Virtual Reality from 1991. There’s so much discussion of haptics in that book.

A lot of these ideas re-circulate from earlier texts. It will be interesting to bring the depiction of haptics in the Ready Player One movie into dialogue with its predecessors. For example, if you recall the formulation of the link between the real body and its virtual projection in The Matrix: if your virtual body dies, your real body dies along with it—which is essentially the same principle Ivan Sutherland laid out in the Ultimate Display in 1965. A bullet in the ultimate display would be so realistic it could kill you.

I also think of Aldous Huxley’s “Feelies” from Brave New World—these fictional representations of touch technology are complementary to existing technologies. There’s a line from Jake in your video, where he says “sight and sound, but also touch.” There’s a narrative that we have existing media that is all sight and sound, and now we’re ready to layer touch on to them. It’s additive but also transformative, and it provides a really helpful way to frame the development of haptics.

A New History of Haptics

Greg Bilsland: What applications for haptics are you most interested in seeing in the next five years?

David Parisi: One thing we’re seeing that’s quantifiable is a huge spike in the engineering and robotics related publications on the topic of haptics. I graphed the uptick at one point based on Google Scholar results, and the results are really striking. We’re seeing a lot of resources devoted to figuring out how to use this technology.

What’s interesting right now is that the fate of haptics seems tied to the fate of VR. In Archaeologies of Touch, I open with a discussion of the keynote Oculus Chief Scientist Michael Abrash gave a couple of years ago at Oculus Connect. He’s very upfront that for VR to work, it must have a robust and effective touch feedback system. But Oculus hasn’t provided that system—and Abrash states that it will take “breakthrough research” in order to make it happen, along with an entirely new programming language for rendering haptic sensations.

In Archaeologies of Touch, I talk about this as a “master device”—a fully-body haptic interface capable of replicating a whole range of haptic sensations, which achieves such widespread adoption that it becomes a standardized format for digitalizing touch. This lack of standardization is one of the things that’s held the diffusion of haptic interfaces back—and I’m curious to see what happens once a particular device starts to get widespread uptake. It’s what Sutherland imagined in the Ultimate Display, and it’s exactly the vision Jake lays out in your video. Marshall McLuhan has this great line: “Resenting a new technology will not halt its progress.” My resistance to a technology will not impede it, but on the flipside, my advocacy for a technology will not necessarily hasten its advance. I really try to stay on the sidelines in terms of viewing this one way or the other, but one of the things that make haptics tech so appealing to me as an object of study is its immense potential to bring about widespread changes in a whole range of human activities—from the way we work to the way we love.

Interested in Archaeologies of Touch?

The University of Minnesota Press is offering a limited time discount of 30% off the book’s cover price through its website (www.upress.umn.edu) using code MN82600.