Fundamentals of Interaction

We are developing a set of design principles that provide the theoretical foundations for our research. These concepts are grounded in current understanding of human cognitive and sensori-motor capabilities and the potential of modern-day interactive computing. Together, they form a novel approach that both describes current interactive technology and also provides a generative approach for designing simpler, yet more powerful interactive systems. Our goal is to create a paradigm shift similar to that of moving from command-line interfaces to graphical user interfaces.

In order to better understand fundamental aspects of interaction, ExSitu will study interaction in extreme situations. We will conduct in-depth observational studies and controlled experiments which contribute to theories and frameworks that unify our findings and help us generate new, advanced interaction techniques. Although we will continue to explore the theory of Instrumental Interaction in the context of multi-surface environments extend it into the wider framework of information substrates. We will also continue to study elementary interaction tasks in large-scale environments, such as pointing and object manipulation.

Creating interactive systems is complex, and the available tools trail far behind the capabilities of the available hardware. Creating the kinds of systems we envision is even more challenging since we must create high-performance distributed systems running in heterogeneous environments using a variety of input and output devices while collecting and analyzing data in real time. In addition, the rapid prototyping cycles required by our Extreme Design approach imply the ability to quickly create and modify such systems.

To address this challenge, we need to better understand the interplay between the design process and the tools to create designs so that we can identify the appropriate tools and software infrastructure. In particular, one critical challenge that hinders extreme users when they want to appropriate and combine existing technologies is the lack of interoperability among tools and the lack of consistency within and across interactive systems, making it often impossible or excessively difficult to, e.g., combine different interaction styles or devices, transfer an interaction technique from a desktop computer to a mobile device, use different programming toolkits or access low-level settings of a device driver at the user level.

By identifying concepts and abstractions that work at multiple levels, we seek to address these challenges and facilitate the exploration, implementation and integration of novel interaction technologies and techniques. More concretely, we will develop concepts, methods and tools at three levels: the developer level, with new architectures and programming languages that promote openness for interaction through interoperability, the designer level, with new prototyping tools for creative and extreme prototyping, and the end-user perspective, with new configurable and co-adaptive systems.