Breaking the Glass: Multimodal, Malleable Interactive Mobile surfaces for Hands-In Interactions

点击翻译按钮获取中文摘要

Mobile phone and tablet touch screens are flat, dead surfaces. Our work seeks to explore the potential of a far more fluid, 'alive' portable display. It departs radically from existing deformable research by endeavouring to provide truly direct interaction with affordances, controls and content integrated within a visual display. We will be highly ambitious and adventurous, pushing the boundaries of technical possibility and being inspired by perspectives from deformations in the animal world through a Co-I and team from biosciences. Our work will be further grounded and informed by our end-user partner, BBC R&D. The outcome will be scenarios, interaction styles and a range of fully-functional prototypes that will enable us to map out a design space to drive developments in both display materials and a richer, expressive toolkit of gestures and manipulation on touch surfaces.Consider the following scenarios that illustrate the possibilities of the new technology and user interfaces:1. Sam is using the new paint application on her tablet. At the top of the screen is a row of paint pots that dynamically recess into the display. Each contains a colour, and the deeper she dips her paintbrush into the pot, the deeper and thicker she is able to paint onto the remaining screen surface, just as in the real world of watercolour sets.2. Rosie is watching a BBC nature programme on her very large screen HD television set. In her hands she holds a tablet which can display companion content for the programme. While the large screen shows a butterfly dancing around a garden, her tablet screen deforms to provide a sensation of the flapping wings, indicating to Rosie that there is further content on the second screen. However, she is immersed with the main screen display and briskly pushes the deformation away to the right of the mobile. The force of her interaction is used by the system as a gauge to her interruptibility as additional second screen content becomes available.3. Siân is producing a live music event for broadcast over iPlayer. She needs to keep her eyes and ears alive to the pace and spectacle of the band whilst catching the individual performances of each of its members. Normally a sea of controls from an array of mixing desks would surround her, rigged to control every potential shot from the camera crew on and off stage. The new flat panel touch screen mixers enable Siân to customise the controls, reducing the kit required, but without tactile feedback her attention is diverted from the action to the control surface. These micro moments of distraction can be critical and the continual switching of attention increases the stress of the job. The new deformable displays give her best of both worlds: a smaller, responsive mixing desk with tactile feedback that keeps her eyes on the performance she is broadcasting.We propose a number of transformative benefits of the novel surfaces and architectures we will construct:- The three dimensional and multimodal nature of the elements can be used to communicate features of an information space in more efficient and satisfying ways;- The surface can provide sophisticated and nuanced controls with, for instance, the extent to which the user pokes her finger or stylus into the surface adjusting the degree of system response; and,- The tangibility of the elements can enable eyes-free operation, allowing the user to combine their handheld interaction with other objects in their environment in order to deal with a complex content space. The platform can then provide physical affordances and controls, in a dynamic way, as the user interacts with the Internet of Things around them.The team combines world leading researchers with expertise in: user experience, hardware and materials innovation, electronics, design space analysis and biosciences.