September 5th, 2019

Tangible Computing & Interactive Paper

At EUROPARC (Xerox PARC’s European research lab), I created a research group that introduced the concept of augmented paper interfaces, and explored how to integrate paper with on-line multimedia information. Projects included: Video Mosaic, a digital desk that permits paper storyboards to act as the interface to an on-line video editor, Ariel, to let engineering supervisors use paper engineering drawings as their interface to a media space, to their on-line computer system, and as a method of sharing informal annotations, Digital Drawing Board, to project hand-drawn 2-dimensional sketches as 3-dimensional drawings, and Caméléon, augmented flight strips for air traffic controllers. The latter has been pursued by my colleagues at ENAC in Toulouse, and was released as a product, called Digistrips, that was sold to British air traffic control.

After joining Inria, I developed the A-Book and Prism, hybrid laboratory notebooks that integrate paper-based and electronic information for research biologists at the Institut Pasteur. FamilyNet and Telebeads provided simple-to-control tangible network interfaces for managing access to Communication Appliances. We created a series of innovative interactive paper applications with professional composers at IRCAM: Musink [64] (CHI best paper) lets composers can add meaning to the musical notations they express on paper and link them to powerful tools such as OpenMusic, Knotty Gestures [60] can turn any hand-drawn line into an interactive command or controller, InkSplorer [58] lets composers move fluidly back and forth between paper and OpenMusic, Paper Substrates [55] lets composers layer translucent interactive paper to create diverse effects and PaperTonnetz [50] offers interactive tonnetz for exploring complex musical relationships on paper. This work led to the creation of a piece called Quid Sit Musicus by the internationally acclaimed composer Philip Leroux, with a world premiere in 2014. We are currently exploring the power of print electronics. Stretchis use a simple silk screen technique to create silicon-based wearable, stretchable interfaces that include various forms of input and output, including buttons, touch sensors, and phosphorescent screens.
At Rank Xerox EuroPARC, Wendy created a research group that introduced the concept of augmented paper interfaces and explored how to integrate paper with on-line multimedia information. Pierre Wellner’s influential DigitalDesk came out of this group and together with Rich Gold from PARC, they co-edited the seminal award-winning special issue of CACM on Computer-Augmented Environments, which launched the fields of Augmented and Mixed Reality. Wendy is a leading expert in augmented paper-based interfaces, with many projects developed for targeted users, including video editing, 2D and 3D sketching, air traffic control (Caméléon), laboratory notebooks, communication appliances (FamilyNet), and music composition (Musink). Caméléon led to a product sold to British air traffic controllers; FamilyNet led to a technology transfer with a French startup, and Musink to the creation of a piece called Quid Sit Musicus by the internationally acclaimed composer Philip Leroux.