Since the beginning of digital world, man is in a constant struggle of accessing and manipulating the virtual resources. In 1973, the Xerox PARC developed the Alto personal computer, which was the first one demonstrating the first generation of Graphic User Interface, with a “desktop metaphor”. The Apple Macintosh brought this new conception of HCI into the commercial success in year1984.

In 1991, Mark Weiser (Xerox PARC) published an article “The Computer for the 21st Century“ on his vision of “Ubiquitous Computing”, illustrating a different paradigm of computing and HCI. From that prediction, computers are no longer objects and artifacts we work in front of, but rather installed in the background, connected by network, talking to each other and barely visible.

“Ubiquitous computing names the third wave in computing, just now beginning. First were mainframes, each shared by lots of people. Now we are in the personal computing era, person and machine staring uneasily at each other across the desktop. Next comes ubiquitous computing, or the age of calm technology, when technology recedes into the background of our lives.” ~ Mark Weiser.

In this wide spectrum of Human Computer Interface design, from GUI to Ubiquitous Computing, tangible interaction is an area of exploration in this project.

Tangible Interaction Design


Adjective: Perceptible by touch

According to dictionary definition, the word “tangible” literally means capable of being touched and is real or actual rather than imaginary or visionary. So it usually points to physical objects or at least something you can get hold of. In discussion about tangible Interaction, it will be always referred to Tangible User Interface (TUI), so as to differentiate from Graphic User interface (GUI) under the big umbrella of Human Computer Interface (HCI).

TUIs couple physical representations with digital representations yielding interactive systems that are computationally mediated, but generally not identifiable as “computers”.

~ Brygg Ullmer and Hiroshi Ishii [1]

It is obvious that TUI is an important bridge between physical and digital world, I/O devices such as mouse and keyboard are obvious examples. As the development goes further, we are not satisfied with the limited application, and TUI is expanded as an alternative form of computer interface (tangible user interface) that takes advantage of our natural ability to understand and manipulate physical forms while still harnessing the power of computational simulation to help in our understanding of a model representation. [2]

However broadly speaking, Tangible Interaction encompasses user interfaces and interaction approaches that emphasize

  • Tangibility and materiality of the interface
  • Physical embodiment of data
  • Whole-body interaction
  • The embedding of the interface and the users’ interaction in real spaces and contexts.

It is argued that the term tangible user interface has focused mainly on the design of physical interface rather than on what users actually do with the design. So the emphasis of the term was intentional. It put the focus of design from technological point of view back to human centered design. The difference between Tangible interface and Tangible Interaction is like Interface and Interaction Design. [3]

Tangible Interface >< Tangible Interaction

Interface >< Interaction Design

Tangible Interaction is a very interdisciplinary area. It spans a variety of perspectives, such as HCI and Interaction Design, but specializes on interfaces or systems that are in some way physically embodied – be it in physical artifacts or in environments. Furthermore it has connections with product/industrial design, arts and architecture. [4] Then, related topics in Ubiquitous Computing, Tangible Augmented Reality, Organic interfaces, Interactive surfaces and Robotics contribute to the field of Tangible Interaction.

Beijing Opera: the Subject

Beijing Opera is a comprehensive acting art. It blends singing, reading, acting, fighting, and dancing together by using acting methods to narrate stories and depict characters. The roles in Beijing Opera include the male, female, painted-face, and comedic roles. Besides, there are other supporting roles as well. In addition, the types of facial make-up, especially concerning the color, are the most particular art in Beijing Opera, because they can symbolize the personalities, characteristics, and fates of the roles.

Interaction Models of Beijing Opera

Interaction Model – The concept TWO DOORS

Performers enter the stage through one door and leave the stage through the other. They are significant symbols dividing the reality from virtual, and the self from the fictional character. The transition symbolizes the change of status and role, and the performance of Beijing Opera is a continuous loop of this transformation.

However this model has a different implication in modern world as the role of creator and creation is blurring. Audiences have equal interest towards the fictional character as well as the actor/actress themselves. This is especially true in new social media. The notion of authorship no longer lays on the content created, but viewers are more interested in the author himself. This may be an interesting feature to play with in our project

 Interaction Model – The concept of GONG & DRUM

Gong and drum music lead the tempo and rhythm of the opera as well as enhancing the overall performance. Criteria include Rhythm, Composition, Layering and Atmosphere. The goal is to engage and touch more audiences.

There are specific gestures of performance and music playing with traditional musical instruments. The cultural heritage builds up rich language of gestures, which connects to audience in natural way.


Our Design Direction

Based on the study of Beijing Opera, our project idea is

Storytelling + Gestures

// A combination of motion and image making
// Put forward footages for co-creation

FuniMovi is an interactive platform for image creation and storytelling.

Agent // our target user is young generation who love to create. They are encourage changing their perception of image making, and become part of the tangible user interface.

Agency // image-maker is controlled by direct manipulation gesture input (sign language of Beijing Opera). The camera body is designed to be a tangible interface that is deformable in structure. Menu and buttons will be reduced to minimal.

Act // users capture images to create stories, but allow co-creation with other users. Users can play with imaging during the shooting with direct manipulation of the camera body, and during editing by gesture recognition.

Purpose // FuniMovi aims to facilitate sharing of synergy for creative concepts. Group dynamics between participants enhance the imagination and innovation.

Context // Creation of a creative platform which links people from reality to virtual world and back to reality, similar to the interaction model of Beijing Opera one.


Design philosophy of FuniMovi

In our long history, human has developed skills of manipulating tools and objects around us. Through grasping and manipulating these instruments, users of the past must have developed rich languages and cultures that valued haptic interaction with real physical objects.  However much of this richness has been lost to the rapid flood of digital technologies. [5] We were inspired by the possibility of direct manipulation and rich natural physical affordances [6], most of which have disappeared from our daily life and have been replaced with the most general of appliances: personal computers.

Our vision is not about making our product multi functional with high computing power, but rather about regaining the rich language of haptic interaction with real objects and awakening our ability of using our body as an interface interacting with the world. This gives us a broader understanding to tangible interaction that the tangible nature is on the performance of interaction but not just the physical forms.


Design Concept of FuniMovi

In FuniMovi, We tried to create Gesture Sensing Interfaces, expanding on the fields of Tangible User Interaction and Gesture Recognition. Gesture derived from Beijing opera has its cultural heritage that can communicate with the audiences/users naturally. From the study, music actually plays a more important role in guiding the rhythm and enhancing the overall performance than just an ambient sound.

The project demonstrates an alternative form of computer interface that takes advantage of our natural ability to understand and manipulate hand gestures in the creation of visual record of our activities.

Our Slogan of Funimovi


// Use GESTURES to manipulate music, images and video

There are different specific gesture/movement when creating music in Beijing Opera. These gestures could be mapped to the “shooting” &”editing” process of story telling/creation. Our tangible user interface has the implication of mapping physical objects directly to virtual objects within the computer. [7]


Interaction Model of FuniMovi

The concept of 2 doors is modified and implemented. The two doors on stage are combined as a 2-lens camera, one on each side.

The actor and the fictional character coexist on both sides of the camera.

When the user is taking picture and video, his images will be recorded simultaneously that will contribute as part of the creation. Same concept will be performed during video editing. User is not just the Creator of story but become part of the creation/story (Our new definition of authorship).


Visual Impression of FuniMovi

Visualization of FuniMovi camera

Camera lenses are installed on both sides. Flexible grips on the left and right allow gesture input controlling the image effects and lenses’ actions.

The gesture sensing ability of camera is designed to facilitate users’ natural expression of play while capturing images or videos with FuniMovi. The product has of 2 stages of storytelling: one is to activate image effects and lens actions through gesture manipulation of grips on both sides of the camera. The other is to edit footage with a set of hand gestures through gesture recognition system embedded in the camera. FuniMovi consists of a new form of interaction for users’ storytelling and image capturing with physical artifacts. Being positioned as a device for authorship, the open-endedness of visual creation and edit provides competitive appeal to FuniMovi when compared to professional camera gadget that looks for high-quality images.

The open-endedness of Funimovi not only provides excitement for users but can also nurture the creative ability as well.

There are research works suggesting that the open-endedness of tools can actually help creative development of children at their young age. [8]


This is a fun project of tangible interaction design. The debate about the definition of tangible interaction and TUI provides a wide perspective of exploration, and the connection with haptic language derived from human heritage is interesting too.

The adaptation of interaction model of Beijing Opera for new image making device is a good exercise of concept transition. I believe that this development process and skill learned could be very useful in future design and analytical projects.


Our Project Documentation

Project Blog

Team Members

CHOI Chi Kit, Jackson, LIU Ya Ming, LI Xin, Purple, SU Qin