Ubiquity Editorial – Interaction Designs for Ubicomp Cultures

I. Ubicomp Cultures: Hyperbolic Vision, Factual Developments

This is a journal issue invested in remarking more than once upon the undecidability hovering today around our getting into contact with ‘ubiquity’ or ‘pervasiveness’ as a potential to be further actualized in the fields of human-computer interaction (HCI), interaction design, and the cultural life worlds of information societies more generally. It could well be that you have not yet heard of ubiquitous or pervasive computing, or that you have heard of these but still remain in doubt whether there actually is or will be such a thing, in interaction designs or elsewhere. It could also very well be the case, however, that you both know a great deal about this as a rather momentous shift, qua a third wave in computing and associated disciplines, and find yourself engaging with it all around you in your practical life: at work, at home, in leisure activities and games, in the media art at the museum, or in the everyday culture of the public sphere. Affirming this undecidability is a necessity – since both of these alternatives are currently at stake, and since ‘ubiquity’ and ubicomp remain potentialities of whose actualization we are not yet sure. This undecidability may be a matter of the explicit articulation of principal ideas. At the same time, it may concern the concrete lines of development and research that make of this so many hands-on facts inherent in the interactions in our contemporary life worlds. In other words, the focus and special merit of this issue is not least to enter into the set of questions surrounding the notion of ‘interaction designs for ubicomp cultures’ – as something partaking of that which Michel Foucault would have called ‘a history of the present.’ This issue engages with an altogether contemporary field of research in order to make a difference that makes a difference while the cultural and technical developments at stake are still undecidable, multiple, and emergent – at a fast pace, too.

Even now, 20 years after Mark Weiser’s initial coinage of ‘ubiquitous computing’ as a term (1988), and 15 years or so after his seminal papers (Weiser, 1991, 1993, 1994, 1995, 1996, 1997, 1999; Weiser and Brown; Weiser, Gold and Brown), there is not yet a general consensus as regards the definition of ‘ubicomp.’ However, perhaps one might, as do the contributors here, proceed on the assumption that the field of inquiry involves in particular a third epoch of computing (after the mainframe and the personal computer), one preoccupied with the question whether and how computing is, should be, or could be moving on from existing primarily as distinctly recognizable units so as to be multiplicitously and pervasively integrated into our living and working environments and perhaps altogether invisibly embedded in our life world and life form. In that case, a working definition of ‘ubiquitous computing’ would be a socio-cultural and technical thrust to integrate and/or embed computing pervasively, to have information processing thoroughly integrated with or embedded into everyday objects and activities, including those pertaining to human bodies and their bodily parts. Thus, if somebody is living with ubiquitous computing, it does not concern or only marginally concerns engaging consciously with a single device or application for some definite purpose. Rather, it concerns engaging with multiple computational devices and systems simultaneously during more or less ordinary activities, without necessarily being aware of doing so. This is also to say that the models and practical implementations of ubiquitous computing investigated here largely adhere to something like Weiser’s vision of a myriad of small, inexpensive, robust, networked information processing devices, perhaps mobile but certainly distributed at all scales throughout everyday life and culture, most often turned towards distinctly mundane, commonsensical, and commonplace ends. When this sort of extensive distribution is referred to as a development of ‘out-of-the-box computing,’ it is due not least to Weiser’s suggestive notion of a process of drawing computers out of their electronic shells so as to have the ‘virtuality’ of computer-readable data and ‘all the different ways in which it can be altered, processed and analyzed… brought into the physical world’ (Weiser, 1991: 98).

During these last two decades, we have had a number of conferences concerning the technics of ubiquitous and pervasive computing, many of which have resulted in the publication of conference proceedings, and these conferences now continue on a regular basis (Dourish and Friday; Krumm; Indulska; LaMarca, Langheinrich and Truong). Likewise, the interested reader will today be able to find at least a dozen books treating of quite a few of the pertinent issues in the hardware-engineering and the software- or middleware-development for ubicomp (Adelstein; Cook and Das; Szymanski and Yener). In addition, the first more substantial and useful anthologies in computer science studies have now appeared (Symonds, 2010, 2011). Nonetheless, a major part of the technical research issues are far from resolved. Device components, networks, as well as the different layers of protocols and applications remain in multiple strands of basic development rather than already being involved in undertaking a broader and higher level abstraction from a shared consensus or standard. Non-resolved computer science issues that are still common to most research projects at work on ubiquitous and pervasive computing include, among other things: the sensation and collection of meaningful data on ‘human’ activities; building models for real-world ‘human’ activity; application of software agent technology, many of them only loosely integrated; appropriate unobtrusive interfaces; taking the right kind of care of security, privacy, ownership, and trust; appraising ‘human’ factors and social impact; implementing, maintaining, and developing dynamic communications networks; managing the scales and heterogeneities of ad hoc networks in non-hierarchical ways; modeling collective failure modes; appropriate consideration and design of energy consumption when many of the systems depend on batteries (Steventon and Wright: 12; Crowcroft).

In the human and social sciences, moreover, the field of research is characterized by a very noticeable delay in the development of cultural theoretical, sociological, psychological, and aesthetic approaches to ubicomp and its implications for our form of life. Some conferences have now been held, however, in Yokohama, New York, Weimar, London, and Copenhagen, mostly by culture and art organizations, a few by universities. In addition, an initial set of interesting book-length studies have begun to emerge. Malcolm McCullough and Adam Greenfield’s individual studies provide a quite detailed account of what is at stake culturally and architecturally in the emergence of ubiquitous and pervasive computing, while drawing each in their own way on a sound, vocal skepticism so as to point towards a first set of critical evaluations (McCullough; Greenfield, 2006). The Throughout volume edited by Ulrik Ekman presents the first relatively comprehensive anthology engaging in an explicit treatment of a considerable subset of the socio-cultural, ethico-political, media-specific, aesthetic, and philosophical aspects and implications of the contemporary development of ubiquitous computing (Ekman).

Something analogous to this delay is at stake for the disciplines and fields more sharply in focus in this journal issue, i.e., interaction design, HCI, CHI, and human factors. In spite of the undeniable general rise of interaction design since the early 1990s (after Bill Moggridge and Bill Verplank’s introduction of the term, and after the emergence of the first corresponding disciplinary institutes), one cannot but notice a certain slowness and several lacunae in the engagement with an ever more extensive actualization of ubicomp cultures and their associated technics, software, as well as medial interfaces.

As recent a book as About Face 3: The Essentials of Interaction Design (Cooper, Reimann and Cronin) illustrates with abundant clarity that the very vast majority of practitioners today still generally subscribe to a view of this discipline as a matter of defining the behavior of products and systems that users can interact with visually, specifically via a graphical user interface (GUI) appearing on a screen – and this typically by engaging peripherally via a mouse and a keyboard belonging to a very recognizable personal computer with Internet access. Along that path, the keystones of work on interaction design notably include a distinct (personal) computer, a screen, a GUI, and WWW networking. If questions of ‘ubiquity’ and ‘pervasiveness’ present themselves at all, these appear matters of marginal import.

However, one would also want to argue that this state of affairs is to be approached as a question of a hegemonic and predominant disciplinary paradigm for the second wave of computing (personal computing and distinct PCs overtly demanding our attention) that is currently very much in the process of being revised. For in parallel, and simultaneously, you also find, for example, Jay David Bolter and Diane Gromala participating in the SIGGRAPH 2000 digital art show only to write a book that argues in favor of establishing as the goal of interaction design the appropriate rhythmic oscillation between transparency and reflection, and between the invisible computer and the visible computer (Bolter and Gromala: 2-6). Notably, Bolter and Gromala wish to see this oscillation unfolding as a matter of a contextual design that is informed by the new paradigm of relations between the virtual and physical. That is rather precisely what one finds in the current developments of augmentation (AR and AV), mixed reality (MR) (Milgram and Kishino), and a kind of ubicomp that includes not least an embodied experience of decidedly social and cultural settings in the real environment (Bolter and Gromala: 114-40). From this other perspective, more traditional approaches in interaction design could well be supplemented not only by such efforts found in the early T-Garden project examined by Bolter and Gromala, but also, for example, by the many later interaction designs that have been evolving for tangible AR applications and for embodied sociocultural MR interactivity.

An analogous set of more traditional conventions can be observed at play in the field of HCI, whenever computer science and the behavioral sciences are brought to meet in visual design, or whenever computers, operating systems, plus programming languages are brought to meet social sciences, cognitive psychology, plus linguistics in a relation of computer graphics and communication theory. The enormous impact on interaction designs for personal computing made by ocularcentrism, transparency, as well as GUI and WIMP paradigms is evidently still with us, hegemonically too. Nonetheless, the pursuit of actual developments of ubicomp qua an out-of-the-box ‘calm’ computing (Weiser and Brown, 1996) has also had its effects on HCI during the last period. Witness the responses to the remarkable de facto expansion of embedded computing found in the EEC ‘disappearing computer’ research initiative (Streitz, Kameas and Mavrommati), Donald Norman’s work on the invisible computer (Norman, 1998), as well as Paul Dourish’s still seminal work on a notion of interaction design for ubicomp cultures that would draw upon key insights from HCI, an existential phenomenology of the body, and a phenomenology of the social world (Dourish, 2001).

Given such developments, perhaps it is not altogether surprising to find that already the first part of the massive work from HCI International edited by Julie Jacko and Andrew Sears, deriving from conferences in 2003, came to include a substantial prospective section on virtual, mixed, and augmented environments which very evidently involve, at least to some extent, innovative post-PC, post-desktop, and post-GUI notions of interactivity and interaction designs which might draw upon invisible processes of computation (Jacko and Sears: 1103-1308). Two years later, the ubicomp conference in Tokyo already bespeaks a deep engagement with the challenges inherent in developing fast and robust interfaces for ubiquitous applications operating in context-aware mixed-reality systems (Gajos et al.). Here, shifts from interaction to participation become much more explicit, along with a reconfiguration of space as partaking of mixed reality, not least as a matter of a mobile, embodied interaction rather than a more or less virtually abstract, primarily sedentary, and positionally fixed immersion.

Perhaps the year of 2005 can be considered a minor turning point, as Mike Kuniavsky has also observed in his study concerning experience design for ubicomp (Kuniavsky), for the later editions of the key disciplinary works from HCI International (Jacko, 2007, 2009, 2011; Sears and Jacko) bespeak a growing concern with HCI for ubicomp cultures. Witness the expanded general treatment of multimodal interfaces, adaptive interfaces and agents, mobile interaction design, tangible user interfaces, and information-intensive environments – and, more specifically, the devotion of book-length studies to issues of ambient, ubiquitous, and intelligent interaction (Jacko, 2009).

Evidently, the kinds and degrees of actualization of interaction designs for ubicomp cultures are not least affected by the ideas and forces which shape computing during the final years of the 1990s and the first decade of the new millennium. Interaction designs for ubicomp cultures become more than a potential and a much more pressing factual concern in tandem with: decreasing general hardware costs, reduction in power requirements, implementation of ubiquitous ad hoc networking (including high speed and/or wireless LAN and WAN), increasing development of mobile and distributed computing, widening of the ongoing deployment of embedded computation to include networked communications among units, deployment of materials for miniaturization and further specialization of sensors and actuators, increased portability of computational devices, thin and large new display technologies, pursuit of high-bandwidth interaction and innovative multimodal input techniques, presentation of group interfaces on organizational and socio-cultural levels, as well as extensions of user-tailorability to include user-innovation in more domains. However, these interaction designs concern a co-evolution of culture and technics. Thus, such interaction designs develop in tandem with the broad cultural integration in everyday practices of digital identity systems, social media and web 2.0, mobile communications, GPS and locative media, things that think (RFIDs), tagging of the life world, information intensive environments, context-aware installations, responsive architectures and smart homes, security systems, surveillance, and more.

Since projects are currently unfolding along such multiple lines of innovation, ubicomp cultures cannot simply be termed an actual fact or process, but necessarily retain a considerable potential dimension, still undecidable and in emergence. Nonetheless, already in 2005 this third wave in computing has established enough of a cultural presence empirically as well as in research and development that educational textbooks for designers operating with and alongside the computation of the 21st century take as a point of departure a number of the traits and challenges of ubicomp.

In this special issue of Fibreculture and in the contributors’ research articles it is taken for granted that the relation between the potential of ubicomp culture and its actualization is today still undecided but also a matter of a very dynamic and energetic set of ongoing research projects and concrete technocultural developments which call for quite some descriptive, analytical, and critically evaluative efforts. The articles in this issue thus proceed after having acknowledged what one might wish to call a certain ambiguity. First, the contributors acknowledge that a fully developed, robust, pervasively distributed, relatively smart, context-aware, and innovatively ad-hoc networked ubiquitous computing has yet to emerge as a cultural and technical fact, whether in an invisibly embedded infrastructural variant or an overtly attention-getting personalized portable one. [1] Secondly, the contributors affirm, at the same time, that the actual technical developments as well as our modes of interactivity (socio-culturally medial and communicational, psychological, and aesthetic) have already changed enough to warrant the recognition that in a number of ways we are living in a ubicomp epoch and world.

However, this ambiguity is not simply acknowledged in a neutral fashion. The contributions can be read as a constellation of statements to the effect that at this point in time the discourses and practices relating to interaction designs for ‘ubiquity’ call for a first set of critical distinctions. Preferably, one would distinguish between ‘ubiquitous computing,’ as a historically specific term denoting certain actual socio-cultural and technological developments during the last two decades relating in the main to computer science, software engineering, interaction design, media studies, media art, and their supports in the human and social sciences, and, on the other hand, the more metaphorically slanted terms ‘ubiquity’ and ‘pervasiveness’ which appear consistently as idealities, not least in the hyperbolic form of philosophical tropes with metaphysical and/or ontological remainders that display quite some traditional capacity to survive.

Efforts to begin delimiting the latter are evident, for example, in Anders Michelsen’s insistence – in ‘Pervasive Computing and Prosopopoietic Modeling’ – that we must recognize a certain ‘cybernetic metaphoricity’ within existing claims, more or less explicit, that computing and its mediations can ‘pervade’ realms of the real so as to really be or become ‘ubiquitous.’ Michelsen suggests that we first undertake a more detailed rehistorization to see how this metaphoricity displays intrinsic relations to the heritage of cybernetics and systems theory from the mid-20th century onward. On that background Michelsen suggests that we address ‘ubiquitous computing’ as partaking of more than half a century of computational imagination, specifically engaged in a reworking of what one might call, echoing the thoughts of Herbert Simon and Ezio Manzini, ‘an imaginary of the artificial’ (Simon; Manzini). At this altogether general level, interaction designs for ubiquity would then best be approached as a matter of a creative human articulation, Michelsen argues, in which one may distinguish between a novel form of our computational modeling of the real via the artificial (approximating the world as design) and, on the other hand, the more radical address of the artificial as being computational per se (approximating a radical computational imagination or a process of artificialization).

Whether in cultural theoretical or technical discourses, the terms of ‘ubiquity,’ ‘pervasiveness,’ and ‘ambience’ come silently freighted with a notion of totalizing universality or even certain ontological and metaphysical remainders (altogether abstract idealizations and/or excessively essential or substantial extensions). Both the editors and the authors contributing to this special issue approach this as a call for ongoing deconstruction and reconstruction, not least in the sense that remainders and implications of onto-theological and sovereign ideological notions must be questioned reasonably so as to be put under critical erasure in one or more ways. The articles thus include an implicit orientation towards rather unconditional critique of the idea that ubicomp is, should, or could be ‘ubiquitous,’ that pervasive computing is, should, or could be ‘pervasive,’ that ambience is, should, or could be ‘all around,’ or that the discourses, practices, and inventions involved extend, penetrate, and invade ‘throughout,’ or are always already at stake all over as an omnipresence. Instead, one would like to put the emphasis on the multiple actual ways in which interaction designs for ‘ubiquity’ partake of infinite finitude. Perhaps a reminder of our myriad on-off relations of interactivity with and within mobile cultures is one of the easiest and best ways to illustrate that the problematics of cultures of ubiquitous information exist not as a totality or infinity but rather as so many matters of immanent complexity. In this actual but still emergent third wave of computing, its mobile devices and co-developing cultural practices might be one of the best foci. This because all our everyday engagements with mobile phones, handhelds, and small tech make felt a culture of ubiquitous information qua the dynamics and energies of ad hoc network theories and practices – live as organized inorganicity, inorganically and organically live. Mobile computational entities and their cultural enfoldment are such good foci because they make felt the ways in which complexity arises from a vast number of distinguishable relational regimes and their associated state spaces, promising a defined system of interactivity for ‘ubiquity’ (to come).

Continue reading Ulrik Ekman: Interaction Designs for Ubicomp Cultures.

Notes

[1]For one good, short, and relatively early account of the status quo of ubicomp research, including its limits and unresolved issues, see the article written by two computer scientists from Intel (Want and Pering).