Interaction & Virtual Environments

2.1. Introduction

Human-Computer Interaction is a relatively new domain in the science of Informatics. Its elements have been present since the beginning of the computer age, but hadn’t been considered to be important or primary at that time and thus, a distinct domain wasn’t formed. The fact that the domain emerged recently, along with the dominant institution of techno-science, often lead to misunderstandings. An examination of the terms Interaction, Human, Computer, on their own may elucidate the situation.

/> Interaction: A universal definition is impossible. It can certainly be told, however, that not all forms of communication are to be classified under this term (e.g. between two routers), even if there is a request-and-respond action from the participants.

There has to be a relationship between the n-th respond and the (n-1)-th request. This relationship has to be more than a functional or, in general, an ensidic one—there has to be a relationship of content, a relationship of presentations, a relationship of meaning.

/> Human: The most usual misunderstanding of the domain until now is that it tries to explain humans through the external characteristics of their behaviour (behaviourism). So, we have loads of models of the brain as teukhein, as a deterministic machine (with some random noise, but without changing the general concept). Humans are represented in analogy to the computer, as two different sources of “objective reality”, which they communicate in the form of bits (reminding us Kant’s for itself), but simply having their own special way to express this “reality”.

This viewpoint cancels human psyche; humans and computers are alien/other to each other, not different; humans are capable of representing—of making meaning emerge in a magmatic relationship of the ensidic, imaginary, symbolic, phantasmatic; it cancels human creation, the social self-institution which happens through and with the historical time.

Recent research [2] [3] [4] [5] that surpass a mechanistic model and discuss about the content of interaction, has encountered with astonishment the fact that, apart from emotions and easy learning, participants in interactive experiences show enhanced sociality (this was successfully realized in the Bauhaus School) [6].

/> Computer: Interaction is usually considered to be “external” action. The metaphor of “navigation” used in Interface Design shows the contradiction. When we “navigate” in an application we perceive it in two opposite ways. The Cartesian view of the digital landscape is external to us, as a map both independent and spontaneous. On the contrary, our subjective presence is an undoubted part of the virtual digital landscape. This contradiction is quite old [2].

The computer, however, bears within itself interactivity—it has been constructed for this purpose. Its application
and its hardware are thinkable as interactive because they are organized as such. Anyway, the computer processes data and not information—meaning-making, due to lack of radical imaginary, is impossible. This is often omitted during the development of interactive applications. Inevitably then, we have to take Humans into consideration even more, if we take into account this alterity. This distinction is essential to us, and we view it as Human-Computer Interaction’s principle and not as a neo-humanistic ideal.

Progress in Human-Computer Interaction (presented in 2.2) is technically undoubted, because of research on and understanding of some basic functions of interactivity. Nevertheless, one has to keep in mind progress (or rather evolution?) in Human Interaction, like with Freudian psychoanalysis initially, group therapy afterwards, etc.

2.2. Interactivity

People tend to connect interactivity to computers, especially after 1990 when Bill Moggridge came up with the term interaction design, but this is totally insufficient. Interactivity has been in different regions and eras of social-historical institution, in the fires of Homer, in the smoke signals of Native American, or in the stone cairns of the Celts.

Several centuries later, in the 1830s, Samuel Morse constructed and designed not only his code, but also the whole telegraph system: the electric circuits, the mechanism to produce the code and user’s training. Of course, it didn’t happen in one day, but it is the first communication system too complex for the average user. It required an entire system to be specially designed.

Similarly, for other communication systems such as the radio, the telephone or television, interfaces had to be designed by the engineers. For the very first time the interface
was not a human being (the telegrapher, the typographer, the post-man), but a mechanical, and often automatic, device. We can define, for the first time, a virtual interface. These interfaces were also used for the communication between devices (telephone routers etc.). This forms an entire virtual environment. Interaction design, at that time, was not a priority.

Machinery used for these technological achievements had human input, but in a simple, linear, non-adoptable way. That changed with the use of computers.

Other advancements concerning input and data control in computers, came from medical and film industry and have many similarities. In motion capture usually sensors are used, reflective or gyroscopic to construct and control virtual environments. State-of-the-art implementations combine different sensor types (U70 computer, XSens MT9, ARTTrack, Flock of Birds, ARToolKit, Garmin GPS18 USB). [8] [9]

Usual navigation in virtual environments with touch screens derives from 2D and 3D computer desktops and its usability is questioned. The iOrb device was specially designed for command and data input in virtual environments. The user can run application commands selecting from 1D and 2D pie menus. Spatial selection include both selection rays and cones. Interaction uses relative gyroscopic distances, so measuring is quite sensitive to errors. [10]

Concerning the development of interactive applications targeting the expression of users’ psychic presentations, haptic senses are used, a rather constraining factor. State-of-the-art application is the reacTable*, while non-haptic virtual environments had been restricted to visualization of voice data—as in the installations Hidden Worlds, Messa di Voce and RE:MARK. [11] [12] There has been a lack of an electronic music instrument that allows the musicians to express psychic representations through their own physical movement, but with a non-haptic interface, controlled completely remotely.


[2] J. Wood, The Virtual Embodied, Routledge, London, 1998.
[3] S. Rafaeli, Interactivity.
[4] M. Merleau-Ponty, Phenomenology of Perception, Humanities Press, New York, 1962.
[5] P. Light, K. Littleton, “Situational Effects in Computer-Based Problem Solving”, Discourse, Tools, and Reasoning, Springer-Verlag, 1997.
[6] D. Svanaes, Understanding Interactivity, 1999.
[7] G. Levin, Painterly Interfaces for Audiovisual Performance, Massachusetts Institute of Technology, 1994.
[8] D. Schmalstieg, G. Reitmayr, “The World as a User Interface: Augmented Reality for Ubiquitous Computing”, Central European Multimedia and Virtual Reality Conference, 2005.
[9] N.J. Dellemana, E. den Dekkera, T.K. Tana, I3VR –Intuitive Interactive Immersive Virtual Reality– Technology, France, 2006.
[10] G. Reitmayr, C. Chiu, A. Kusternig, M. Kusternig, H. Witzmann, “iOrb - unifying command and 3d input for mobile
augmented reality”, Proc. IEEE VR Workshop on New Directions in 3D User Interfaces, IEEE, pp. 7–10, 2005.
[11] S. Jordà, Sonigraphical Instruments: From FMOL to the reacTable*, 2003.
[12] G. Levin & Z. Lieberman, In-Situ Speech Visualization in Real-Time Interactive Installation and Performance, 2004.