21.9 Commentary by Jeffrey Bardzell

I begin my commentary by expressing my excitement to see an important philosopher directly engaging the HCI community. Up till now, philosophy’s participation in HCI has been mediated by HCI researchers’ interpretations and often (frankly) dumbed-down introductions to philosophical concepts. So this is a special moment for us, but it also introduces a practical challenge, and that is that because Shusterman is outside of our community, he can’t fully participate in our “language games,” which, in the case of HCI refers to the ways we frame and legitimate research, the sorts of questions we ask, the stock of examples we assume everyone knows, the history of the field we all think we hold in common, etc.

For a relationship between somaesthetics and HCI to prosper, linkages will need to be developed. Doing so is a book or at least a lengthy journal article, and I have here only a commentary to work with. So I will pursue the more modest goal of sketching out a position for somaesthetics in HCI, with the hope that others (from HCI, from philosophy) will join those of us who are already exploring those linkages.

My thesis in this commentary is that somaesthetics occupies a unique theoretical position in our field, able to connect pragmatist approaches to HCI (design theory, experience design) and embodied approaches to HCI (affective computing; mobile, pervasive, and ubiquitous computing). This has implications both for users, and in particular, norms for serving users in the deepest and most important ways possible; and also for interaction designers, and in particular, the cultivation of the professional self as an expert subject.

My argument will first sketch out recent calls for a more designerly HCI, exploring what those calls really mean, and specifying the challenges this community faces in responding to them; and second it will explore somaesthetics (and related pragmatist traditions) as a collection of useful concepts that has practical, pedagogical, and normative implications for a somaesthetic HCI. I will conclude by reflecting on some limitations of somaesthetics and HCI, which could guide future work.

21.9.1 The rise of design as an input for HCI

As is well documented elsewhere, since the 1990s there have been increasing calls for a more designerly approach to HCI, calls which have picked up steam since 2000. As computing has moved from the workplace and into everyday life, e.g., with the rise of mobile phones, pervasive computing, and the wild spread of digitally enabled consumer electronics, from DVRs to programmable sex toys, the distinctions between product design, service design, communications design, interaction design, and the emerging area of experience design have never been blurrier. Today design is a thriving subcommunity of HCI, with its own official subcommunity within CHI, a highly successful biannual conference in DIS, and ongoing calls for even more design contributions to the field. For example, McCarthy & Wright (2004) observe that

This turning towards design is often couched in critiques of the limits of social scientific approaches that have dominated HCI for decades. Greenberg & Buxton (2008) write,

There is a lot going on in this quote, including a critique of an HCI dogma that privileges objective measures over other legitimate forms of design knowledge-making, a prioritization of the social sciences over design and engineering innovation, and above all the practical consequences that we are too narrow to innovate. Restating this argument more simply, Greenberg and Buxton imply that innovation depends on subjective expertise and is hard to get at using the objective methods valorized by the HCI community.

Kuutti (2009) agrees: tracing the evolution of HCI as an academic-industry collaboration to deal with the explosion in demand for usable software during the rise of the PC in the 1980s, Kuutti describes how usability became the primary practical achievement to respond to this problem, with the unfortunate side-effect that “this also meant a certain intellectual impoverishment: calls to discuss about HCI theoretical foundations lost the audience, when the somewhat a-theoretical (and originally sometimes even anti-theoretical) usability movement took over” (Kuutti, 2009, p. 7); indeed this anti-theoretic stance is alive and well today, as can be seen in Tractinsky’s account of visual aesthetics in HCI and the portion of my commentary devoted to confronting his anti-theory position.

So the lack of design thinking inhibits innovation, and it is characterized by impoverished theory and a dogmatic fetish for objectivity. What then is a designerly approach to HCI? What sorts of theories, methodological strategies, and goals add up to a designerly HCI? A few choice quotes can at least offer some insights:

Design pedagogy is about “educating such personalities that can filter and crystallize cultural influences into effective and meaning-laden forms.” This characterization is a far cry from the traditional rhetoric of user-centered design. Whereas user-centered design positions the designer in an almost passive position of discovering existing needs using scientific methods and then designing around and for what is discovered in that activity, traditional design activates the designer as a perceptive, insightful, and imaginative meaning-maker, an ability that is individualistic to a certain degree and dependent on judgment rather than data, and offers a radically different view of the foundations of a design problematic.

Cockton offers a more specific account of the sorts of things that designers do as a part of their profession:

This is a different sort of list than one might expect from a traditional textbook on HCI: reflection, theory, a foregrounding of purpose, transdisciplinarity, thoughtfulness, exploration of alternative design means all characterize a designerly approach. Again, the role of a subjective expertise is unmistakeable in this list. It is quite easy to imagine how one designer could be superior at reflection, use of theory, creativity with alternative means, and so on than another. In contrast, user-centered design methods seem to presume that a designer’s strength is the quality of the data informing decisions, not anything internal to the designer herself.

My third quote comes from Greenberg & Buxton again, who offer a different sort of list and yet one that is fundamentally compatible with the vision of design that is emerging here:

Guiding all of this seems to be a holistic interpretation of what the design will be, accompanied by a rationale, and vetted by an intensely iterative and ongoing critical process involving stakeholders and other designers. This situates design in a dialogic and argumentative tradition. What is being argued for and against is the designer’s particular framing of the problem and speculative vision of its solution. Again, the subjective expertise of the designer—as an active meaning-maker and speculative reasoner—is the foundation of the whole activity. Others are brought in on similar terms: their own ability to interpret, frame, re-frame, and speculate determines the quality of critical feedback that they can provide.

In short, design professionals require a cultivated ability to read socio-cultural signs and trends; a creative and reasoned ability to explore alternative futures; a verbal ability to articulate these activities; a receptiveness to alternative framings and a willingness to explore highly variable alternative directions; and above all a personal identity or coherence that holds all of these moving parts together through a given process. Much more is personally demanded of designers than is personally demanded of traditional usability engineers.

So how to we get there?

There are several answers to this question. One is that a small number of individual designers cannot achieve critical mass: we need a design culture in HCI (Nelson & Stolterman, 2003). Creating such a culture has implications both for how interaction designers are trained and also for how the community legitimates certain knowledge practices. But however that happens, one thing is for certain: theory is going to be in the middle of it. Theory has historically been central to design and the humanities, inasmuch as each is concerned with the insightful and imaginative understanding of culture. Inasmuch as HCI now wants such accomplishments to be part of its discipline, HCI will have to get over its aversion to theory and fetish for methods. And this is where a philosophical program such as somaesthetics enters the picture.

21.9.2 Somaesthetics in/for HCI

Shusterman has offered here an accessible introduction to somaesthetics, effectively condensing and synthesizing earlier works, including “Somaesthetics: A Disciplinary Proposal,” which appeared as Chapter 10 in the second edition of Pragmatist Aesthetics: Living Beauty, Rethinking Art (2000) and his more recent book Body Consciousness (2008). I reference these books partly because they much more context for somaesthetics and Shusterman’s whole project. The first chapter of Pragmatist Aesthetics carefully situates somaesthetics within the context of Dewey’s pragmatism and in opposition to analytic aesthetics. This context translates to some core positions.

One core position is a formulation of somaesthetics as comprising a holistic and even organic perspective on life, work, and the self. Shusterman’s holism is evident in many places, not least in his efforts to undermine rigid distinctions between aesthetics and ethics, as when he writes, “aesthetic considerations are or should be crucial and ultimately perhaps paramount in determining how we choose to lead or shape our lives and how we assess what a good life is” (Shusterman, 2000, p. 237). HCI has long debated about the relative values of function versus aesthetics, with traditional HCI siding with function; but Shusterman follows Dewey in rejecting the distinction and seeing aesthetics as a holistic and inclusive term that encompasses the categories traditionally subsumed under function and form.

Dewey’s notion of an aesthetic life is not mere hedonism, but rather a sophisticated understanding of how the human as an organism purposefully and successfully copes with its environment (Berstein, 1971). Shusterman follows Dewey in rejecting divisions, distinctions, formalisms, and hierarchies. This is not merely an abstract chin-scratching philosophical position, but rather a practical position that has serious methodological implications: it basically rejects atomism or scientific reductionism of experience (Bernstein, 1971). Applied to HCI, such a view would reject the ways that affect researchers decompose affect into mood and emotion, emotion into positive and negative valence, positive valence into n number of positive emotions, and each of those into d degrees of intensity. Rather, a Deweyan view would construe emotion as a part of human’s purposive future-oriented disposition to the world, helping the organism orient itself in the best possible way.

Described holistically, somaesthetics is “a life-improving cognitive discipline that extends far beyond questions of beauty and fine arts and that involves both theory and practical exercise,” which seeks “to end the neglect of the body that [was] disastrously introduced into aesthetics,” with the ultimate goal to “contribute significantly to ...  an art of living” (Shusterman, 2000, pp. 266-67). That’s a nice sounding agenda, but what does all that mean?

Somaesthetics, like any other philosophical position, can be characterized as comprising a system of relating concepts. I began with the context and overall view, because I don’t want an analysis of somaesthetic concepts applied to HCI to lose sight of what somaesthetics is supposed to do, which is to help us lead or shape our lives and to recognize what a good life is. I will consider this in two different directions, both of which are central to HCI:

• The training of interaction designers
• Normative criteria for user experience

21.9.2.1 Aesthetic perception, somatic training, and the interaction designer

In this section I want to argue that two of Shusterman’s key concepts, aesthetic perception and somatic training, contribute to both (a) a substantive epistemological account of the designer as expert subject and also (b) a useful set of norms to orient professionals cultivating designerly ways of doing (with implications for interaction design pedagogy as well).

21.9.2.1.1 Aesthetic perception

Among the most common views of aesthetic perception in HCI is the stimulus-response model. On this view, an object in the world, such as an interface, acts as a stimulus to the human cognitive system, which responds to it, e.g., by perceiving it, storing it in memory, understanding it, deciding to act based on it, etc. In HCI, for example, physiological data, such as breath rate and skin conductance, is collected as ways of measuring a person’s response to an input. Important advances to the field have been made with this model, and I certainly don’t want to suggest that it is somehow “wrong.”

But the stimulus-response model has epistemological limitations, and these have implications for interaction design professionals. The key limitation is that such a model assumes the existence of certain interpretative skill in the first place. It’s not always obvious to us, but understanding is a deeply cultural and learned ability. For instance, when we go to a museum or historical site, a docent not only relates historical backgrounds and contexts, but more importantly tells us what to look at. But if it is right before our eyes all along (i.e., a visual stimulus), then why don’t we respond the right way (i.e., with appreciation)?  Why is it that a professional and amateur photographer standing side-by-side looking at the same thing will take very different pictures of it? How can a professional designer look at a given design material and come up with surprising and expressive new forms, where others simply rehash existing forms? Docents, accomplished photographers, and designers see and understand in richer ways than others do, and this is fundamental to their professional abilities.

Shusterman offers an explanation for these abilities by paraphrasing the work of 18th century philosopher Baumgarten, who first coined the term “aesthetics.” For Baumgarten, in Shusterman’s paraphrase,

In the stimulus-response model, and in most empirical science itself, such qualities have no meaningful place. Visual stimuli, and by extension empirical data, are seen to speak for themselves: the experiencing subject has only to perceive them to understand them. So stimulus-response is basically passive, and the model doesn’t differentiate among responders. Much UX research in this tradition assumes that research subjects are fundamentally interchangeable and simply seeks to average their physiological or reported emotional responses (e.g., using Likert scales). 

But what Shusterman wants to emphasize is—and here he is leveraging phenomenological hermeneutics and reader-response theory as well as pragmatism—that understanding and hermeneutic skill must also exist before perception. Though we often speak commonsensically as if object, lightwaves, visual perception, mental image, understanding, judgment, and decisionmaking all happened in a causal linear sequence, in fact it cannot characterize what actually happens. Meaning-making is an active process; meaning is not a form stamped in our cognition like a seal ring to wax.

Some people can perceive more keenly than others; some have more penetrating insights, some have a greater imaginative capacity. Importantly, these are not static “faculties” that we are born with, but rather “sensory cognition” comprises cultivatable abilities or habits that we practice and can improve over a lifetime. As Dewey writes of intelligence, it is not “the faculty of intellect honored in textbooks” [but rather is] “the sum total impulses, habits, emotions, records, and discoveries which forecast what is desirable and undersirable in future possibilities, and which contrive ingeniously on behalf of imagined good” (Dewey, cited in Bernstein, 1971, p. 211).

Somaesthetics is substantially responsive to the calls for a design sensibility in HCI because it offers an epistemological account of what such a sensibility actually is: a sensitive, imaginative, penetrating, tasteful, poetic, and expressive habit or disposition to design problems, materials, processes, opportunities, and situations. But how does one achieve such habit or disposition?

21.9.2.1.2 Somatic training

One of the signature pieces of Shusterman’s somaesthetics is his call for somatic training: if it’s the case that the body is the “tool of tools,” then philosophers need to get out of their armchairs and cultivate their own somatic competencies. He defines this as “actually engaging in programs of disciplined, reflective, corporeal practice aimed at somatic self-improvement” (21.3.2).

To understand what he means by this, we might consider the practices that he uses to exemplify this: “diverse diets, body piercing and scarification, forms of dance and martial arts, yoga, massage, aerobics, bodybuilding, various erotic arts (including consensual sadomasochism), and such modern psychosomatic therapies as the Alexander Technique, the Feldenkrais Method, Bioenergetics, Rolfing, etc.” (Shusterman, 2000, p. 272). Common to each of these is a long-term commitment to body refinement. This is not a question of mastering the body by feeding and exercising it according to recommendations from the health sciences; rather, this is a practice of self-stylization for which the body is the locus of one’s individual stylized identity. The distinction I’m drawing here is akin to shaving one’s head versus becoming a “skinhead.” One is a mere physical description of a change to the body, whereas the other entails the same physical change but in such a way that it is inscribed in and defining a symbolic identity for the person doing it.

Thus, for Shusterman, the cultivation of somatic sensibility is an outcome of corporeal training. And surely he is right about this. I trained myself to skate well enough to join an amateur hockey league, and I also enjoy watching ice hockey. It is certainly the case that my appreciation for the watching the sport is partly based on a somatic understanding of the sheer skill of good players: people who have themselves tried to skate backwards while turning and accelerating and also while controlling a puck and keeping it from an opposing player can almost physically appreciate the somatic aesthetics of such movements as they are displayed gracefully and effortlessly in a professional game. Surely people who go to the ballet, many of whom have had some dance training themselves, also appreciate somatically, almost “feeling” in their own bodies the somatic near-perfection of a professional dancer’s alignments, body angles, and turnout. Indeed, how many somatic spectacles (e.g., professional sports, dance, rock concerts, etc.) do we enjoy that we haven’t tried out in one form or another, whether it’s backyard football, an attitude en pointe in front of the mirror, air guitars, or temporary tattoos?

Somatic training is also a part of HCI. We don’t just talk about designs: we sketch them, prototype them, try them out, put them in people’s hands and homes and watch what they do. These are all somatic exercises, and all of them require considerable training before anyone becomes good at them. Even among HCI researchers, the rising interest in critical design (e.g., Dunne, 2006; Dunne & Raby, 2001) and research through design (Zimmerman et al., 2007) extends this trend: such research uses design methodologies not in the hopes of creating new commercial products, but in order to generate knowledge and theory. Critical design is not easy to do, and seems to be an activity that requires iteration, practice, and training (Bardzell et al., 2012).

The other obvious area of HCI that involves somatic training is pedagogy. In our HCI/Design Master’s program at Indiana University, students work in groups and individually to generate sketches, prototypes, workbooks, and portfolios within a studio culture in which they are frequently presenting their work for critique by faculty and peers. I don’t think we’re unique in that: such pedagogy is a part of our field. Additionally, a majority of our students’ Master’s projects involve a domain with which—external to their participation in our program—they already have somatic training: we’ve seen projects building on prior experience in political activism, hardcore World of Warcraft play, fashion illustration, senior health care provision, and film production, among others.

Somatic training is substantially responsive to the call for a design sensibility because it relates design processes and practices to the underlying epistemology of an expert subject. Design processes are a form of somatic training: they entail disciplined embodied practices, and these practices eventually heighten perceptual and expressive sensitivities towards human needs, visual forms, problem reframings, socio-cultural meanings. It is by such mechanisms that designers train to become the kind of people who can, re-quoting Kuutti, “filter and crystallize cultural influences into effective and meaning-laden forms.”

21.9.2.2 Somaesthetic Experiences

Much of my commentary thus far has focused on the suitability of somaesthetics as a theory that offers a rich and useful account for training the specialized sensibilities expected of design professionals. However, neither Shusterman nor the other pragmatists were seeking to support specifically the design profession: the pragmatist project is fundamentally geared to improving all human quality of life by reminding us that humans are organisms purposefully engaged in their lived environments, and not information processors or “disembodied ratiocinators” (in the memorable phrasing of Bannon & Bødker, 1991). And that means that somaesthetics also provides normative criteria for the design of aesthetic experiences for users.

That is, if we want to reframe UX away from usability and towards something more robustly aesthetic, then we need to replace existing UX goals with new ones. Traditional ones include low task completion time, low error rates, high learnability, etc. Kutti (2001) proposes three alternative criteria that seem in the spirit of Deweyan pragmatism:

• Users as learners
• Users as shapers of their environment
• Users as becoming something else by using a device or a system

These sound like good goals, but their abstractness creates a practical challenge for designers. Here somaesthetic and pragmatist theory can begin to unpack some of these concepts. From a pragmatist perspective, aesthetic interaction should contribute to some combination of the following user experience-abilities. I combine “experience-abilities” into a single concept, because pragmatists stress how skills emerge in and from experiences; it is only by being challenged—not dumbed down with ease of use, transparency, and simplicity—that skills and aesthetic pleasure emerge.

The following are normative goods valorized by pragmatism in general and somaesthetics in particular. That is, an aesthetic interaction is one that adheres or contributes to some critical mass of the following:

• The experience and cultivation of Baumgarten’s “perfection of sensory cognition,” that is:
• keenness of sensation
• imaginative capacity
• penetrating insight
• good memory
• poetic disposition
• good taste
• foresight
• expressive talent (from Shusterman, 2000, pp. 264-65)
• The expansion of people’s range of sympathetic identification with others (Guignon & Hiley, 2003, p. 36, paraphrasing American pragmatist Richard Rorty)
• A reformation of objects not as external to and in opposition to individual subjectivity, but rather seeing objects as contiguous with human consciousness, that is, seeing objects as human “activity in an objectified or congealed form” (Bernstein, 1971, p.46)
• An orientation towards the future (hope, intention, disposition), not the past (secure knowledge): “anticipation is ... more primary than recollection; projection than summoning of the past; the projective than the retrospective” (Dewey cited in Bernstein, 1971, 207).
• An appreciation of (and contributions toward) consciousness as dynamic and emergent, rather than static but wanting to collect more information. Consciousness is not a fixed form of mental seeing (i.e., disinterested understanding), but more along the lines of “the craftsman involved in doing and making.... The craftsman perfects his art not by comparing his product to some ‘ideal’ model, but by the cumulative results of experience—experience which benefits from tried and tested procedures, but always involves risk and novelty” (Bernstein, 1971, paraphrasing Dewey, p. 219)
• Ongoing somaesthetic training, self-improvement, and self-stylization. If experiences of art and beauty are distinctive for the powerfully gratifying ways they absorb our attention, unify our consciousness, and engage our emotions, then increasing our powers of awareness, focus, and feeling through better mastery of their somatic source could render more of our experience similarly rewarding in such ways. (Shusterman 21.1)

An aesthetic experience is one in which (a) the aesthetic goods listed above are experienced or felt, and also (b) the experience of them contributes to the long-term somaesthetic skills of insightful perception, imagination, meaningful self-stylization, and a disposition to do good.

All of this is not to suggest that every single interaction design must meet all of the above norms; rather, pragmatist holism would seem to suggest that interactions need only to contribute to and participate in lived ecologies where these qualities are experienced and subsequently cultivated through practice into skills. It is the lived world that ideally needs to have these qualities, not every single thing a person touches within the lived world. But inasmuch as our lived world is artificially designed—buildings, clothing, parks, appliances, furniture, and now interactive technologies—the burden is on us as designers to make that artificial world humane.

Somaesthetics offers normative criteria and a conceptual vocabulary to facilitate the design and evaluation of humane interactive products.

21.9.3 Criticisms and Limitations

It is probably obvious that I am sympathetic to somaesthetics and believe that it can contribute to HCI and interaction design. But one bad habit that we as a field have is that while we are eager to advocate for the introduction of a given theory, we often don’t acknowledge that this theory has confounds or limitations. I have tried to argue some of the specific ways that somaesthetics can contribute to HCI, and I think it’s also important to explore some of the ways it is not particularly well positioned to contribute to HCI. By exploring both strengths and limitations, as a field we can use theories more effectively and have some sense for when alternative theories are needed.

21.9.3.1 Somaesthetics is only loosely coupled with methods

HCI is a field that likes its methods, and it’s not clear how somaesthetics translates into methods. More fundamentally, it’s not even clear whether somaesthetics should translate into methods, at least not in the sense that the term is used in the sciences. At stake is an epistemological disagreement about how best to produce knowledge. A traditional experimental methodology, such as that described by Tractinsky in his interaction-design.org Encyclopedia entry on Visual Aesthetics, isolates variables in controlled experiments—classic methods from experimental psychology. For Kutti at least, such an approach is the antithesis of how designers operate:

Kuutti seems to establishing an exclusive opposition between expert judgment-based approaches and methods, a position also suggested by Greenberg & Buxton (cited earlier). Indeed, the very existence of methods seems to dumb scientists down into “barbarians.” Kuutti’s provocative language aside, it is easy to understand why scientists might view personal judgment as lacking any methodological rigor and thus barely any better than “mere opinion,” and why designers might view scientific methods as replacing human judgment with mechanistic algorithm-like recipes, which would seem to be a form of intellectual “barbarism.” Though it’s easy enough to understand these caricatures, it’s less clear whether they have any validity or in fact if they mainly just exaggerate differences. Dewey harmonized scientific and artistic approaches, but he did so by treating the sciences as an aesthetic practice, a position that might not appeal to practicing HCI scientists (though there is not as much daylight between Dewey’s position and that of post-positivist science, e.g., Quine, as one might expect).

Regardless of how apparent or real the opposition is between expert judgment and scientific method, it’s not clear to me that somaesthetics is going to resonate with interaction designers for whom data and methods are paramount.

21.9.3.2 Somaesthetics says little about the content of actual experiences

In a recent paper, Hassenzahl, Diefenbach, & Göritz (2010), who operate within a cognitive approach to UX, criticize McCarthy & Wright’s pragmatist take on UX as follows:

What Hassenzahl, Diefenbach, and Göritz are getting at is that there is an empirical dimension to experience, that is, users of a given design do have an experience e, it should be possible for research to discover the content of that e, and that McCarthy & Wright’s approach categorically fails to address that question, because it offers few strategies to capture “what these ‘values, needs, desires’ are”—which is an empirical, rather than critical, question.

I think there is a valid point here. I do see value in McCarthy & Wright that Hassenzahl, Diefenbach, and Göritz apparently do not, but I can understand why they read McCarthy & Wright in that way.

And now I will also add that I think a somaesthetic approach to HCI basically has the same shortcoming for basically the same reasons. Again, while somaesthetics is strong at offering an account of how an individual trains or cultivates the self as a perceiver and expresser (not just a thinker), it offers fewer tools to try to understand the content of particular experience x, and yet designers do have reason to want to know that.

21.9.3.3 Shusterman's stock of examples isn't particularly helpful

For an encyclopedia entry on interaction design, Shusterman not only used examples that take a lot of work for HCI readers to understand in the way that he wants them to, but he also missed opportunities to explore somaesthetic HCI with appropriate examples from the field. It is, of course, easy to explain this problem as a result of Shusterman’s outside status. Nonetheless, it is a huge missed opportunity, not just rhetorically (in terms of his ability to persuade HCI readers to engage with somaesthetics) but also substantively (some examples from HCI surely would help everyone think more deeply about somaesthetic HCI).

An obvious starting point is the field of robotics, in particular robotic work for domestic settings and everyday contexts, where the robots are designed to be appropriately meaningful as computational bodies in everyday life. Robot designer Tatsuya Matsui, for example, “believes that robots are like flowers. They can be delicate and beautiful. They are endearing and need nurturing” (Hornyak, 2007), a somaesthetic concept that is undeniably visible in his work:

Copyright © Tim Hornyak Photos. All Rights Reserved. Used without permission under the Fair Use Doctrine (as permission could not be obtained). See the "Exceptions" section (and subsection "allRightsReserved-UsedWithoutPermission") on the page copyright notice.

Copyright status: Unknown (pending investigation). See section "Exceptions" in the copyright terms below.

Figure 21.1 A-B: Robots designed by Tatsuya Matsui

 

Beyond robotics, several areas of HCI have explored embodied computational artifacts in relation to human embodiment. Another work is “Soft-Spikey Mouse,” created by artist Youngsuk Altieri working with Heekyoung Jung and myself:

So when Shusterman writes, “At this stage in somaesthetics research, we have only been concerned with somatic feelings of human bodies and thus with only one side of the HCI interaction. But, in principle, it may be possible to consider the somaesthetics of nonhuman somas, including computer bodies” it is clear that he simply has not yet engaged with the considerable amount of work in our field that has already been doing precisely that for decades.

As I imagine how such examples might influence Shusterman’s thinking, and how much I want to hear what he has to say about such work, it becomes clear to me that somaesthetics just might benefit as much from HCI as the other way around.

And I wouldn’t have it any other way.

21.9.4 References

• Bannon, L., J. and Bødker, S. Beyond the interface: En-countering artifacts in use. In Carroll, J., M. (ed), Designing Interaction. Cambridge University Press, 1991, 227-250.
• Bardzell, J. (2012). Commentary on: Tractinsky, Noam (2012): Visual Aesthetics: in human-computer interaction and interaction design. In: Soegaard, Mads and Dam, Rikke Friis (eds.). "Encyclopedia of Human-Computer Interaction". Aarhus, Denmark: The Interaction-Design.org Foundation. Available online at https://www.interaction-design.org/encyclopedia/visual_aesthetics.html
• Bardzell, S., Bardzell, J., Forlizzi, J., Zimmerman, J., and Antanitis, J. (2012). Critical design and critical theory: The challenge of designing for provocation. Proceedings of Designing Interactive System DIS'12. New York: ACM.
• Bernstein, R. Praxis & Action: Contemporary Philosophies of Human Activity. U Penn Press, 1971.
• Cockton, G. 2008. Revisiting usability's three key principles. In CHI '08 Extended Abstracts. ACM, New York, NY, USA, 2473-2484.
• Dunne, A. (2006). Hertzian Tales: Electronic Products, Aesthetic Experience, and Critical Design. Cambridge, MA: MIT Press.
• Dunne, A., and Raby. F. (2001). Design Noir: The Secret Life of Electronic Objects. Birkhäuser, Basel, Switzerland.
• Greenberg, S. & Buxton, B. (2008). Usability considered harmful (some of the time). Proceedings of CHI'2008. New York: ACM.
• Guignon, C., & Hiley, D. (2003). Richard Rorty. Cambridge: Cambridge University Press.
• Hassenzahl, M., Diefenbach, & Göritz, A. Needs, affect, and interactive products — Facets of user experience. Interacting with Computers 22 (2010) 353—362
• Hornyak, T. (2007). Matsui’s graceful robots evoke human emotion. The Japan Times Online. Available online at http://www.japantimes.co.jp/text/nc20071205a1.html
• Jung, H., Altieri, Y., & Bardzell, J. (2010). Computational objects and expressive forms: a design exploration. In Proceedings of the 28th of the international conference extended abstracts on Human factors in computing systems. ACM, New York, NY, USA, 3433-3438.
• Kuutti, Kari. (2009). HCI and design: uncomfortable bedfellows? In Binder, Lowgren & Malmborg (eds.) (Re)searching the Digital Bauhaus. London: Springer, 43-59.
• Kuutti, Kari. (2001). Hunting for the lost user. Cultural Usability. Available online: http://www.mlab.uiah.fi/culturalusability/papers/Kuutti_paper.html
• McCarthy, J. and Wright, P. Technology as Experience. The MIT Press, MA, USA, 2004.
• Nelson H. & Stolterman, E. The Design Way: Intentional Change in an Unpredictable World. New Jersey: Ed-ucational Technology Publications, 2003.
• Shusterman, Richard. (2008). Body Consciousness. Cambridge: Cambridge University Press.
• Shusterman, Richard. (2000). Pragmatist Aesthetics: Living Beauty, Rethinking Art. Second edition. Oxford: Rowman & Littlefield.
• Shusterman, Richard (2011): Somaesthetics: Thinking Through the Body and Designing for Interactive Experience. In: Soegaard, Mads and Dam, Rikke Friis (eds.). "Encyclopedia of Human-Computer Interaction". Aarhus, Denmark: The Interaction-Design.org Foundation. Available online at https://www.interaction-design.org/encyclopedia/somaesthetics.html
• Tractinsky, Noam (2012): Visual Aesthetics: in human-computer interaction and interaction design. In: Soegaard, Mads and Dam, Rikke Friis (eds.). "Encyclopedia of Human-Computer Interaction". Aarhus, Denmark: The Interaction-Design.org Foundation. Available online at https://www.interaction-design.org/encyclopedia/visual_aesthetics.html
• Zimmerman, J., Forlizzi, J. and Evenson, S. Research through design as a method for interaction design research in HCI. In Proc. of CHI’07 ACM Press (2007), 493-502.

21.10 Commentary by Kristina Hook

In designing for bodily experiences, there has been a lack of theories that can provide the underpinnings we need to understand and deepen our design thinking. Despite all the work we have seen on designing for embodiment (Dourish, 2004, and others), the actual corporeal, pulsating, live, felt body has been notably absent from both theory and practical work. At the same time, digital products have become an integral part of the fabric of everyday life, the pleasures (and pains) they give, their contribution to our social identity, or their general aesthetics are now core features of their design. We see more and more attempts to design explicitly for bodily experiences with digital technology, but it is a notably challenging design task.  With the advent of new technologies, such as biosensors worn on your body, interactive clothes, or wearable computers such as mobiles equipped with accelerometers, a whole space of possibilities for gesture-based, physical and body-based interaction is opened.

Some claim that the technologies we wear today treat our bodies in a negative way:

How can we do a better job in interaction design involving our bodies — the sacred guardians of our identity? This is where I think Shusterman’s theories of somaesthetics are relevant.

21.10.1 Three questions: What experiences? Articulation? Experiential qualities?

To design for corporeal, bodily, movement-based interactions, speaking to our senses and aesthetic experiences is difficult. Three questions immediately pops to my mind. First, what kinds of subjective, pleasurable or displeasurable, experiences are we aiming to design for? Glossing them over as all being about designing for flow (Csikszentmihalyi, 1990) or good gameplay experience is too vague (as we argue in Isbister and Höök, 2009). We need to drill deeper and better understand exactly what experiences we are talking about. Are we designing for pleasurable or unpleasurable ones? Are we designing for those that are subjective and unique, or ones that are common and shared? Ones that deliver serendipitous experiences or ones that are evocative and emotional? These are not all the same, even if they all emphasise aspects of bodily experience.

Second, once we know what kind of experience we are aiming to design for, we need to articulate them in a form that makes sense and that we can share within a design team. Ways of knowing can arise from your bodily acts without any language translation in-between. The feel of the muscle tensions, the touch of the skin, the tonicities of the body, balance, posture, rhythm of movement, the symbiotic relationship to objects in our environment — these come together into a unique holistic experience. It is not the ability to fulfil a task, but the experience of the corporeality of doing so that matters here. Those descriptions also need to be shared with the users that we invite to test our designs, or even participate in the design process.

Thirdly, if we try to design interaction that builds on bodily movement, seeking certain experiential qualities, many different aspects of the interaction have to be fine-tuned to enable the experience, as, for example:

• the timing of interaction: movement has to render response in exactly the right moment for exactly the right kind of length of time in order to create for a particular experience (Sundström et al., 2005)
• linking emotion and movement: certain movements and body postures are more likely to coincide with certain emotional experiences (Darwin, 1872, Sheets-Johnstone, 1999, Laban and Lawrence, 1974, Moen, 2006, Paiva et al., 2003)
• harmony of modalities: the modalities of the interaction, such as graphics, haptics or gestures, all have to speak together — harmonize (Ståhl, 2006)

Before turning to what Shusterman and his work on somaesthetics can provide us with here, let me remind just give a few examples of how bodily interactions have been seen in the field of HCI or interaction design.

21.10.2 Strands of interaction design dealing with bodily interactions

21.10.2.1 Ergonomics

In ergonomics (preceding HCI — see Grudin 1990), the actual physical body is the core focus. The body has been measured and designed for in spaces such as airplane cockpits, cars or nuclear plant control rooms. As pointed out by Harper and colleagues, the perspective taken is one where humans are seen as part of a machine. The pilots, car drivers and factory workers are part of a larger machinery. They must be trained to follow certain routines automatically as if they are one part of the machine. The machinery must be fine-tuned so that human error is minimized and this can only be done through designing the machinery to fit with meticulous measurements of our physical capacity. In those situations, we want to see our bodies as machines, able to follow routines and act in error-free ways in the spur of a moment (Harper et al., 2007).

While it may sound negative to take such a narrow view on the body, treating it as a machine, we must remember that sometimes we really do want to see ourselves as machines. It is of key importance to us that risks are minimized with driving a car or controlling a nuclear plant. Ergonomics has also cared for the body, aiming to avoid harming the body. By changing the way a machine works, its users can fit better into the machinery. But in ergonomics, for the most part, we assume the body to be passive — the interface will be sending signals to the human body that the passive body receives, sending onwards to the way more important mind. The body is not a subject, actively perceiving and acting.

21.10.2.2 Cyborgs

Another position towards the body sometime taken in HCI is that of cyborgs. A cyborg consists of both artificial and natural systems, or to phrase it differently, of both human body and designed tools that extends out capacity. In its simplest form the extension can be the stick that a blind man uses to find his way. The stick becomes a part of how he feels the world, an embodied part of his own body. But framing tools as part of our cyborg existence goes beyond this one- way extension of our bodies. The cyborg concept comes with various ethical and moral implications when we regard how the technical tools we extend our bodies with in turn speak back to us. This positive side of being a cyborg is in some sense that we can free ourselves from our bodies — as discussed by the feminist Donna Haraway in her cyborg feminist writings (1991). The focus in this movement is on extending the mind, freeing us from our corporeal reality.

While this body-less cyborg being on the internet was much discussed in the beginning of the virtual reality-era, the pendulum has now swung back and most regard it as bad behavior to not connect your real identity to your virtual identity. In addition, more and more technologies are tying reality and virtuality more strongly together, entering our physical selves into the virtual spaces. For example, in the computer games area, we have new interaction devices, such as WII, fake guitars in Guitar Hero, or mobiles, connecting more strongly with our physical selves.

21.10.2.3 Trimming the body

A growing body of work, focuses on the body itself as the domain or the focus of attention. Here, HCI focuses on interactions for sports, healthy living, or physical activity. These systems often treat our bodies as objects that we can study from the outside, that can be trimmed and controlled. Again, the body becomes subordinate to mind, as an instrument or machine, passively receiving sign and signals, but not actively being part of producing them.

These kinds of systems may have many benefits; relieving our bodies from pain, creating interesting experiences, or making us healthy. At the same time, by making the body into a machine that can be measured and studied as an object, we risk putting ‘goals’ and ‘tasks’ to our bodies without turning to our felt life. Pedometers measure how many steps we take and the goal becomes to walk at least 10.000 steps per day — not matter how we feel about walking that particular day. Again, many of these systems reinforce a dualistic stance where the body is a separate entity that can be measured and dealt with as an object. It is not the sensory-locus of ourselves.

21.10.2.4 Third wave

In the “third wave” of HCI, design for experiences goes beyond those of task completion, efficiency, and tool-based perspectives. This includes designing for bodily experiences. So far, when it comes to involving bodies and creating for bodily experiences, the focus has mainly been on sports and games (e.g. from early work (Ishii et al., 1999) to current (Benford et al., 2012)). The aim is to design for experiential qualities such as flow, immersion or uncomfortable experiences. But there is also a growing body of designs aimed at other experiences. One example is Moen’s Body Bug — a wire that you wrap around your body where a ‘bug’ registers your actions and climbs up and down a wire (Moen, 2006). The bug is a simple robot, moving along the wire. When you strap the wire around your body and start making movements, the bug will move along the wire, in a sense mirroring your movements. The bug makes you want to ‘dance’. The sought experiential quality is that of enjoying your own body movement as we do when we dance.

To reach designs in which such qualities arise, designers and researchers have repeatedly reported that as designers, we need to experience our own bodies in the design process (Hummels et al., 2007). This in turn requires new methods in the design process.

Recently, we have started to see other studies where HCI researchers attempt to observe different cultures or communities of practice, for insight on how to design for novel bodily experiences. There are ethnographic studies about hunting culture (Juhlin and Weilenmann, 2008), skaters and golfers (Tholander and Johansson, 2010), horseback riding (Höök, 2010) to citizens constrained by electronic surveillance bracelets (Troshynski et al., 2008). These studies repeatedly tell us that bodily experiences have been undervalued in ICT design and that there is little knowledge on how to address them.

The study by Tholander and Johansson (2010), on skaters and golfers show that those practices do not distract their users from being in the world together with their skateboards or golf clubs. Tholander and Johansson convincingly argue that interactive technologies that aim for physical interaction too often force users to interact through some type of screen interface, taking away focus from the environment. Instead of interacting with others around us or with the surrounding nature, we focus on the screen.

The study by myself on horseback riding, (2010) I try to provide a rich account of how horseback riding involves all our senses, at moments involving us in centaur-experiences — feeling as one with horse and environment. My point is to show both how impoverished interaction with many of our interfaces are compared to the sensory richness of riding, and also how impoverished our articulations of interactions are, the lack of an agreed upon language for describing interactions.

Troshynksi and colleagues, in their study on paroled sex of- fenders who are required to wear a GPS tracking electronic bracelet on their ankle (2008), show how this technology constrains their bodies in ways beyond that of the original intent of the technology. A considerable amount of work is put into preserving the technology intact during everyday routines like showering, and their mobility in the environment is considerable constrained, among other implications.

All of these studies point to limitations in the ways we think of today’s wearable and mobile technologies and their impact on bodily behaviors and practices.

21.10.3 What Shusterman brings to the table

From this brief walk through some of the work involving our bodies in digital interactions, we can now turn to Shusterman’s work and perhaps see a bit more clearly why his theories on somaesthetics are appealing to some of the interaction design researchers in our field.

When designing for a non-dualistic stance towards body and mind, we need some way of talk about what experiences we strive to engage ourselves and our users into. While most accounts of corporeal involvement will be mainly descriptive, Shusterman’s somaesthetics is also normative. He tells us that by engaging in certain practices, in inward listening and learning, we can know ourselves better, and thereby understand and interact with others more fully. It trains our empathy — both with ourselves and others. While this may all sound mysterious and fluffy, the take away message is, in my view, not religious or mysterious. It simply says that we can train our bodies, our muscles, our nervous system (including the brain), to become more knowledgeable and aware of ourselves. As I am a horseback rider, I know that any predominately movement- and body-based practice requires this kind of training and knowledge.
As mentioned above, I have tried to describe the complexity of knowledge required to ride a horse in an autoethnographic study (Höök, 2010). The interaction with a horse is obviously not word-based. It happens through physical signs and signals: the riders use the muscles in their legs, the placement of their sitting bones, bodily balance, head movement, hand and arm connection to the horse’s mouth and sometimes tone of voice. The horse talks back through its movement, direction, pace, activations of muscles that can be felt throughout the horse’s body, its head movements, tail movements, flipping ears, bend of neck and noises. In order to be a good rider, you need to learn this wordless language. As in any language, understanding and communication arises in interaction over time. When you have experienced a particular bodily schema or concept yourself this understanding may arise.

When designing digital interactions, we should be able to articulate, shape and design for equally detailed descriptions of movement, body and physical signs and signals flowing back and forth between us and the system we are designing.
More importantly, as Shusterman points out, moving your body is not only a matter of performing a function, it is also an aesthetic experience. There is a plenitude of activities that we do for the pleasure of moving — dancing, sports, jogging, cycling. The pleasures of these activities are of course not only soft, flowing movements, since some of the activities involve pain, applying yourself really carefully to make your body do them, adjusting your own body in various ways, even making your body build certain muscles that you normally do not use so much, embarrassment when you do not get it right, and so on.

Interaction design has perhaps been a bit too obsessed with zero-learning time, an issue that will not sit well with some of the movement-based practices Shusterman is advocating. A take-away message from Shusterman is that it takes time to learn. You have to apply yourself. Getting to know yourself, your own body, changing your movements, training yourself, is not “natural” — even if your body is “there for you” all the time. Similar to how you must learn to think and reason, you must learn how to listen to your body, how to improve your body knowledge.

21.10.4 Turning to design

As pointed out by Bardzell in his comment to this chapter, the translation from theories of somaesthetics into HCI and interaction design is non-trivial.
Obviously, any body will have different parts (legs, arms, brain, nervous system) and different processes in that body will have different characteristics, but they are intimately linked. Likewise, bodies move through difference spaces, social and physical, shaping interactions. Or as put by Suchman picking up on Latour, unavoidably a part of complex temporal, material and social assemblies, the body is unceasingly (re)configured in relational terms. We perceive, act and understand the world as unities of mind, body, routine, culture, social settings and with machines as part of our ways of being in the world. The design process needs to consider the connections between these processes, moving beyond a narrow focus on cognition as it happens in our brains.

In particular, I have been interested in the linking from movement to emotion and back. Early on Darwin made a strong coupling between emotion and bodily movement (Darwin 1872). Since then, researchers in areas as diverse as neurology (LeDoux 1996; Davidson et al. 2003) to philosophy and dance (Laban & Lawrence 1974; Sheets-Johnstone 1999) describe the close coupling between readiness for action, muscular activity and the co-occurrence of emotion. Sheets-Johnstone makes the case that:

Or as Dewey puts it:

If we attempt to define a lived emotional experience in dualistic terms, we will surely fail, but with a perspective where man is seen as a whole, both body and mind, both individual and part of the world, the gulf between our interpretative experiences and what can/cannot be studied will not be as problematic.

Just to make it slightly clearer, here is an example of such a co-occurrence of emotion and movement from my horseback riding account:

Translating from this experience in horseback riding, we can see many possible digital interactions picking up on rhythm. Take for example the work by Danielle Wilde named hipDisks recently exhibited at CHI 2012 (Wilde, 2012):

Copyright status: Unknown (pending investigation). See section "Exceptions" in the copyright terms below.

Copyright status: Unknown (pending investigation). See section "Exceptions" in the copyright terms below.

Copyright status: Unknown (pending investigation). See section "Exceptions" in the copyright terms below.

Figure 21.1 A-B-C: Thecla Schiphorst dancing with Danielle Wildes' Hipdisk

 

A designer that has picked up more directly on somaesthetics (and who is also commenting on this chapter) is Thecla Schiphorst. She suggests interactions and design methods that require particular movements, such as moving very, very slowly in order to listen to your own bodily state in interaction or attaching users by velcro asking them to move and interact together in order to explore extensions of the body and their meaning in terms of privacy (Schiphorst, 2007). She has also built a couple of systems, like soft(n), that explores the somaesthetics of touch and interaction through interactive artifacts:

Another compelling example is the work by Høbye and Löwgren on the system named Mediated Body (2011). A performer and a participant both wear earphones and through turning them into human antennas, they can generate evocative music when they touch each-others’ bare-skin or “auras”. Again, this invites a somaesthetic, in this case, social experience.

Copyright status: Unknown (pending investigation). See section "Exceptions" in the copyright terms below.

Figure 21.2: Richard Shusterman and Kia Höök using the 'Mediated Body' system at the CHI 2012 conference.

In my view, apart from using Shusterman’s somaesthetic theories to train ourselves as designers or train our users to express what they experience (as discussed by Bardzell in his commentary), I believe that we can build some of the ideas into the actual designs of interactive systems.

21.10.5 In summary

In a sense, the interest in emotional experiences and third-wave HCI has served as a bridge for the whole field of HCI to turn from symbolic, analytical ways of doing task analysis and designing for efficient ways of supporting tasks, to caring more about experiences in general. It has also, to some researchers in HCI, served as a bridge to start addressing our physical, corporeal bodies in interaction and to attempt to bridge the dualism chasm.

This has, in turn, created a huge space of opportunities for design that puts our bodily ways of being in the world first and attempt to address our corporeal experiences. The systems we have been designing in my group (eMoto, Affective Health and others — turn to the chapter on Affective Interaction in the Encyclopedia of interaction-design.org for a longer description) have all been attempts to address the interaction between emotion and movement. While each of these systems has its deficiencies, none of them is trying to reduce human experience to something that can be measured and modeled, and then packaged as an information piece to be sent to others. They are “non-reductionist” (Höök et al., 2008). The experience of using them emotionally and corporeally is shaped by the participants. In a sense this becomes the “participatory design”-movement of the third wave of HCI (Höök, 2006).

It remains to be seen how we can translate the insights from Shusterman’s work on somaesthetics into design. And perhaps, we will have to look for other, complementing theories of bodily interactions, sometimes with less normative perspectives on what is good and what is bad, and perhaps with a stronger orientation towards our socio-bodily practices. Most of all, if we continue to create interactions that come closer and closer to our bodies, our “sacred selves”, we need to be guided by some values or ideas of why and how to do so. The world is flooded with crappy technologies that harm our bodies and our means to be corporeally present in the world, together with others. Addressing aesthetics of the soma, also means addressing important values in design.

21.10.6 References

• Benford, S., Greenhalgh, C., Giannachi, C., Walker, B., Marshall, J., and Rodden, T. (2012) Uncomfortable Interactions, in Proceedings of 30th CHI Conference on Human Factors in Computing Systems, Austin, Texas, ACM Press.
• Csikszentmihalyi, M. (1990) Flow: The Psychology of Optimal Experience. Harper Perennial, London.
• Darwin, C. (1872/1998) The expression of emotions in man and animals, Third ed. by Paul Ekman. Oxford University Press.
• Dourish, P. 2004 Where the action is: the foundations of embodied interaction. Cambridge, MA: MIT Press.
• Grudin, J. (1990). The computer reaches out: The historical continuity of interface design. Proc. CHI ‘90, 261-268.
• Haraway, D. (1991) A Cyborg Manifesto: Science, Technology, and Socialist-Feminism in the Late Twentieth Century, in Simians, Cyborgs and Women: The Reinvention of Nature (New York; Routledge, 1991), pp.149-181.
• Harper, R. Randall, D. Smyth, N. Evans, C. Heledd, L. And Moore, R. (2007). Thanks for the memory, In Proceedings of Interact: HCI 2007, Lancaster, Sept.
• Hummels, C., Overbeeke, K. C., and Klooster, S. (2007) Move to get moved: a search for methods, tools and knowledge to design for expressive and rich movement-based interaction, In Personal and Ubiquitous Computing, Volume 11, Issue 8, Pages: 677 — 690, Springer-Verlag London, UK
• Høbye, M., & Löwgren, J. (2011). Touching a stranger: Designing for engaging experience in embodied interaction. International Journal of Design, 5(3), 31-48.
• Höök, K. (2006). Designing Familiar Open Surfaces, In Proceedings of NordiCHI 2006 Oslo, Norway, October 2006, pp. 242 — 251, ACM press.
• Ho?o?k, K. (2010) Transferring Qualities from Horseback Riding to Design. Proceeding of the 2010 Nordic conference on Human factors in computing systems, NordiCHI 2010, Reykjavik, Iceland.
• Höök, K., Ståhl, A., Sundström, P., Laaksolahti, J. (2008) Interactional Empowerment, In Proceedings of ACM SIGCHI conference Computer-Human Interaction (CHI2008), Florence, Italy, ACM Press.
• Ishii , H., Wisneski , C., Orbanes , J., Chun, B., Paradiso, J. (1999) PingPongPlus: design of an athletic-tangible interface for computer-supported cooperative play, Proceedings of the SIGCHI conference on Human factors in computing systems, p. 394-401, ACM Press.
• Juhlin, O. and Weilenmann, A. Hunting for fun (2008) In Proceedings of the 2008 ACM conference on Computer supported cooperative work, pp. 57-66.
• Laban, R. and Lawrence, F.C., (1974). Effort Economy of Human Effort, Second ed. Macdonald & Evans Ltd., London, UK.
• Longo, G. O. (2003) Body and Technology: Continuity or Discontinuity? In Fortunati, L., Katz, J. E., & Riccini, R. (eds.) Mediating the human body: Technology, communication and fashion. Mahwah, NJ : Lawrence Erlbaum & Associates.
• Löwgren J. (2001). From HCI to Interaction Design. In Chen, Qiyang (ed.), Human-Computer Interaction: Issues and Challenges, Hershey, PA, USA: Idea Group Inc., 2001.
• Moen, J., (2006). KinAesthetic movement interaction: designing for the pleasure of motion. Unpublished Doctoral Thesis, KTH, Numerical Analysis and Computer Science, Stockholm, Sweden.
• Paiva, A., Costa, M., Chaves, R., Piedade, M., Mourão, D., Sobral, D., Höök, K., Andersson, G., and Bullock, A. (2003) SenToy: an Affective Sympathetic Interface, International Journal of Human Computer Studies, 59:1-2, pp. 227-235, Elsevier.
• Schiphorst, T. (2007). Really, really small: the palpability of the invisible, C&C ‘07: Proceedings of the 6th ACM SIGCHI conference on Creativity & cognition, ACM Press.
• Schiphorst, T. (2009) soft(n): toward a somaesthetics of touch. CHI '09: Proceedings of the 27th international conference extended abstracts on Human factors in computing systems, ACM Press, pp. 2427—2438.
• Sheets-Johnstone, M. (1999) Emotion and Movement: A beginning Empirical-Phenomenological Analysis of Their Relationship, Journal of Consciousness Studies, 6, No. 11-12, pp. 259-277.
• Ståhl, A., (2006). Designing for emotional expressivity. Unpublished Licentiate Thesis in Design, Umeå Design Institute. Umeå University, Sweden.
• Sundström, P., Ståhl, A., and Höök, K. (2005) eMoto - Affectively Involving both Body and Mind, In Extended abstract CHI2005, Portland, Oregon, USA.
• Tholander, J., and Johansson, C. (2010) Design qualities for whole body interaction: learning from golf, skateboarding and BodyBugging. In Proceeding of the 2010 Nordic conference on Human factors in computing systems, NordiCHI 2010, Reykjavik, Iceland, pp. 493-502.
• Troshynski, E., Lee, C., and Dourish, P. (2008) Accountabilities of presence: reframing location-based systems. Proceeding of the twenty-sixth annual SIGCHI conference on Human factors in computing systems, pp. 487-496.
• Wilde, D. (2012) hipDisk: Experiencing the Value of Ungainly, Embodied, Performative, Fun,  Interactivity exhibition at CHI 2012, Austin, Texas, ACM Press.

21.11 Commentary by Erik A. Stolterman

Humans have bodies. Bodies are not only the physical mechanical carriers of who we “really” are. Humans interact with and through their bodies. Users are humans and have bodies. Human computer interaction is therefore about bodies as well as cognitive and emotional minds. All this has become a concrete practical reality over the last decade to anyone involved in interaction design. New interactive technology has changed interaction away from being purely representational (commands and text) to direct and bodily interactions (touch, haptic and gesture) and away from being purely screen based to being embedded in and parts of our physical artifacts and environments.

This development means that interaction designers are more than ever before challenged by bodily aspects of interaction. This has of course been addressed both empirically and theoretically within our field through notions such as, ubiquitous computing, embodied interaction, tangible interaction, haptics, gestures, etc.

In contrast to what is common in our field, Shusterman delivers a more profound approach to the challenge of bodily engagements with technology. As a philosopher and as the founder of the term somaesthetics, Shusterman has over the years, in a serious and foundational way, developed an “integrative conceptual framework” for a better understanding of “somatic experiences” (quotes from his article).

What Shusterman can do, that few in our field can do, is to ground the phenomena and ideas related to bodily interactions in a historical and philosophical context and scholarship. This is extremely helpful for our field and it pushes HCI research forward to a more developed understanding.

It is hard to argue against the basic position that Shusterman present. He writes: “By exploring the fundamental features of our embodied ways of engaging the world and transforming it through action and construction, somaesthetics can provide useful insights and experiential skills to help designers produce products and situations that provide more rewarding and pleasurable experience.” Few would argue against the idea that we can obtain other insights about our reality when we “use” our body as a sensory “tool” than if we solely approach it through intellectual and cognitive means.

Reading Shusterman’s article is interesting and potentially useful for anyone involved in interaction design, especially when engaged with any kind of physical aspects of interactive artifacts. However, while Shusterman delivers an excellent account of the theoretical and philosophical aspects of embodiment and somaesthetics, I am somewhat disappointed when he more directly tries to describe how this can be applied or used in interaction design.

Shusterman does this by bringing somaesthetics aspects not only to the relationship between a user and an artifact but into the design process and to the thinking and doing of a designer. He writes for instance “... the body is our indispensable tool of tools“ in a design process. He continues “... designers could improve their design skills by becoming more aware of how they use themselves and how they feel when using particular products rather than merely thinking of how the product is conventionally used.” Of course, it can be argued that this is already common practice in interaction design, since prototyping already is a core activity in our field. Working with material prototypes at any level of fidelity means that the designer or user engages in bodily explorations of ideas. Prototypes are in most cases physical manifestations of design concepts that are developed explicitly with the purpose to explore precisely what Shusterman suggests, namely how a designer or user “feel when using a particular product rather than merely thinking of how the product is conventionally used”.

It might be fair to acknowledge that when Shusterman talks about using the body as a tool he is in many ways more bodily oriented than what is the case when exploring prototypes. For instance, he also sees bodily engagements as a way to explore more abstract ideas concerning potential design directions. However, the idea of using the body as a tool in design activities has also been more directly explored in our field through techniques such as “body storming” and similar methods. So, while Shusterman provide a philosophical foundation for many of these bodily oriented design process activities, it is also clear that designers are already engaged in somaesthetic activities as a way to better understand design ideas and user experience.

Shusterman mentions a few examples of what designers can do to develop a more somaesthetic understanding of their prototypes and designs. But again, for many interaction designers these examples are quite similar to what they already do during prototyping, design sessions, evaluations, etc. For instance, Shusterman writes that “A comparative study of how different shapes of cup handles affect one’s feelings towards drinking, for example, could be used to improve cup design. Does the handle make your forefinger grip tightly, and how does this affect the rest of your arm, the rest of your body, and by extension, the feeling of drinking?” This example could be found in any interaction design textbook on the notion and evaluation of user experience design. What Shusterman proposes is if phrased in the everyday language of interaction design practice a matter of prototyping and contextual evaluation of artifacts in use. The basic belief in interaction design is that since these artifacts are objects and have physical properties, their properties influence the whole user experience and to develop an understanding of that experience, of course the artifact has to be examined in a way that includes the embodied aspects of interaction. So, with this example Shusterman is not really opening up anything new to most interaction designers, instead the example can probably backfire and make the theoretical contribution to appear less interesting.

Another example that Shusterman mentions is HRI (human robot interaction). This is an area where a lot of research is already focused on how people experience the way robots move and in particular how their “bodies” are designed and how humans can, want to, or refuse to (bodily or otherwise) interact with these artificial bodies. For instance, a lot of research has been devoted to the bodily aspect of facial expressions, arm movements, even the embodiment of free roaming “smart” robot vacuum cleaners. In HRI the somatic aspects of design becomes almost unavoidable. There is no traditional interface, there is very little interaction related to disembodied intellectual and cognitive aspects through representational interaction. Instead, robots are themselves embodied and the interaction with humans is embodied from both sides. So, it can be argued that HRI is not, as Shusterman suggests, a potential area for somaesthtics, instead it is a field where somaesthetic approaches are unavoidable and already in practice, even if not theoretically refined.

The article of Shusterman exemplifies something that is quite common when it comes to design (research) and that is the difficulty of transforming advanced theoretical constructs into relevant and practical support for actual design work. We have over the years seen several examples of theoretical and philosophical approaches that have been both recognized and influential in academia as important scholarly contributions. However, many of these, such as Activity Theory, Distributed Cognition, and others, have proven to be difficult to translate into recommendations for practice and few have successfully reached a broader audience among practitioners. Yvonne Rogers (2004) offers an excellent account of this situation, with numerous examples and explanations of why this is a challenge to the field. Another treatise of the same topic can be found in my article (Stolterman, 2008) where I introduce the notion of “rationality resonance”. This concept manifests the idea that any theory, to be practically relevant, has to be based on a deep understanding of existing practice. There has to be a resonance between the existing rationality in practice and the “new” rationality manifested in the proposed theory. This is of course not an issue for theories that only claim to contribute to our understanding of the field, but as soon as a claim is made that a theoretical construct is “useful” in practice this becomes a critical issue. The rationality embedded in a theory with such a claim must resonate with existing rationality in practice, that is, the theory needs to be based on a deep understanding of practice in all its richness and complexity. This means that the proposed rationality has to resonate with or at least pay respect to every aspect relevant in practice, such as, management, resources, time, skill and competence, even if these aspects are not core to the theory.

The theoretical and philosophical foundation that Shusterman offers is, in my view, excellent and should be required readings for any designer engaged with embodied interaction. But I find that the proposed theory or more precisely the suggested practical applications does not yet show enough resonance with existing practice or pay enough insight and respect to the complexity of existing practice. At the same time, it is unclear to what extent the practical design side of somaesthetics isn’t already practiced. Maybe what many interaction designers are already doing in their serious attempts to capture people’s overall experience of interacting with artifacts are already examples of a somaesthetic approach. If so, then Shusterman’s contribution is not to be evaluated in relation to how “useful” it is but to how well it establishes a scholarly and philosophical foundation that existing practice may relate to and rest upon. If that is the case then instead of reading somaesthetics as an approach for design it could be used as a suitable tool for analyzing and understanding existing practice. But, if somaesthetics is actually meant to be seen as something radically different when transformed into practice then we are still looking forward to that to be developed and explained.

21.11.1 References

• Rogers, Y. (2004) New Theoretical Approaches for Human-Computer Interaction, Annual Review of Information, Science and Technology, vol. 38 pp. 87-143.
• Stolterman, E. (2008). The nature of design practice and implications for interaction design research. International Journal of Design, 2,1 55-65.

21.12 Commentary by Youn-kyung Lim

It was around 2006 when I first encountered the term, somaesthetics. As a part of my research activities, I was searching for the most appropriate philosophical discourse for human dynamic bodily experience. At that time, I was developing a new approach for understanding and designing the invisible but tangibly experiential qualities of interactive artifacts. When I came across the concept of somaesthetics, developed by the Richard Shusterman, I became fully convinced that the idea of designing “interactivity qualities” is valuable and furthermore, that somaesthetics may form the core for further developing this idea, more specifically through interactivity attributes (Lim, et al., 2007; Lim, et al., 2009; Lim, et al., 2011).

For a HCI design researcher like myself – who very much appreciates the perspective of aesthetics of interaction – the most intriguing part of the concept of somaesthetics is that it provides the conceptual framework for consciously explaining the “sense of quality” that is sourced and experienced from our bodily senses when we interact with interactive artifacts. Somaesthetics is not merely about bodily experience but more about the articulation of such experience. It is about body consciousness (Shusterman, 2008; Shusterman, 2011). The excerpt from one of my papers (Lim et al. 2011) below may describe the primary effect of such consciousness in the perspective of quality-centered design.

Somaesthetics is often applied in theatrical contexts to analyze performers’ somatic styles of movement and postures, as Richard Shusterman mentions. Shusterman also notes that somaesthetics is not only about the performers but also about the observers of the performers. As a thought experiment, let us substitute the performers with interactive artifacts and the observers with users, or vice versa. Such mapping provides a new way of thinking about the relationship between a user and an interactive artifact. More specifically, such mapping allows us to focus on the somatic styles of movements and behaviors of the interactive artifact while interacted with by the user. Conversely, the interactive artifact may also interpret the user’s movements and behaviors based on his or her somatic styles and respond to these in an appropriate way. As interaction designers, we can become conscious about this somatic relationship in our designs, and thus open a new space to be explored. It may also be appropriate in the context of robotics design as well.

In this way, the philosophical concept of somaesthetics extends the discourses around - and understandings of - our current concepts in Human-Computer Interaction. Somaesthetics may move our concepts in new directions. To me, somaesthetics is a very strong and effective philosophical concept to be learned and applied in interaction design especially when we are interested in experienced qualities of interaction. When we face emerging interaction technologies such as touch interfaces and gestural interfaces, this philosophical concept may, in my opinion, assume an even stronger role in Human-Computer Interaction.

21.12.1 References

• Lim, Y., Stolterman, E., Jung, H., & Donaldson, J. (2007). Interaction gestalt and the design of aesthetic interactions. Proceedings of the 3rd Conference on Designing Pleasurable Products and Interfaces (pp. 239-254). New York: ACM Press.
• Lim, Y., Lee, S., & Lee, K. (2009). Interactivity attributes: A new way of thinking and describing interactivity. Proceedings of the 27th International Conference on Human Factors in Computing Systems (pp. 105-108). New York: ACM Press.
• Lim, Y., Lee, S., & Kim, D. (2011). Interactivity attributes for expression-oriented interaction design. International Journal of Design, 5(3), 113-128.
• Shusterman, R. (2008). Body consciousness: A philosophy of mindfulness and somaesthetics. Cambridge, UK: Cambridge University Press.
• Shusterman, R. (2011). Somatic Style. The Journal of Aesthetics and Art Criticism, 69 (3), 147–159.