Breaking the Glass: Multimodal, Malleable Interactive Mobile surfaces for Hands-In Interactions

移动的手机和平板电脑的触摸屏是平面的，没有任何作用。我们的工作旨在探索一个更流畅，“活”的便携式显示器的潜力。它从根本上脱离现有的变形研究，努力提供真正的直接互动与启示，控制和内容集成在一个视觉显示。我们将雄心勃勃，勇于冒险，推动技术可能性的界限，并通过Co-I和生物科学团队从动物世界的变形观点中获得灵感。我们的最终用户合作伙伴BBC R&D将为我们的工作提供进一步的基础和信息。结果将是场景，交互风格和一系列功能齐全的原型，这将使我们能够绘制出一个设计空间，以推动显示材料和触摸表面上更丰富的手势和操作工具包的发展。考虑以下场景，说明新技术和用户界面的可能性：1.萨姆正在她的平板电脑上使用新的绘画应用程序。在屏幕的顶部是一排油漆罐，动态地凹进显示器。每一个都有一种颜色，她把画笔在罐子里蘸得越深，她就能在剩下的屏幕上画得越深越厚，就像在真实的水彩世界里一样。罗西正在她的大屏幕高清电视机上观看英国广播公司的自然节目。她手中拿着一个平板电脑，可以显示节目的配套内容。当大屏幕显示一只蝴蝶在花园周围跳舞时，她的平板电脑屏幕变形以提供拍打翅膀的感觉，向Rosie表明第二个屏幕上有更多内容。然而，她沉浸在主屏幕显示中，轻快地将变形推到移动的右侧。当额外的第二屏幕内容变得可用时，她的交互力被系统用作她的可中断性的度量。Siân正在制作一个通过iPlayer播放的现场音乐活动。她需要保持她的眼睛和耳朵活着的步伐和乐队的奇观，同时捕捉其每个成员的个人表演。通常情况下，她周围会有一排混音台的控制器，控制着舞台上和舞台下摄像人员的每一个镜头。新的平板触摸屏调音台使Siân能够定制控制，减少所需的套件，但没有触觉反馈，她的注意力从动作转移到控制表面。这些分散注意力的微小时刻可能是至关重要的，注意力的不断转换会增加工作的压力。新的可变形显示器给了她最好的两个世界：一个更小的，反应灵敏的混音台与触觉反馈，让她的眼睛在她正在广播的性能。我们提出了一些变革的好处，我们将构建的新颖的表面和架构：-三维和多模态的元素的性质可以用来传达信息空间的功能，在更有效和令人满意的方式;- 该表面可以提供复杂和细微的控制，例如，用户将她的手指或触笔戳入表面的程度调节系统响应的程度;以及-元件的有形性可以实现裸眼操作，从而允许用户将他们的手持交互与他们的环境中的其他对象组合联合收割机，以便处理复杂的内容空间。当用户与周围的物联网交互时，该平台可以动态地提供物理启示和控制。该团队结合了世界领先的研究人员，他们在用户体验、硬件和材料创新、电子、设计空间分析和生物科学方面具有专业知识。

Mobile phone and tablet touch screens are flat, dead surfaces. Our work seeks to explore the potential of a far more fluid, 'alive' portable display. It departs radically from existing deformable research by endeavouring to provide truly direct interaction with affordances, controls and content integrated within a visual display. We will be highly ambitious and adventurous, pushing the boundaries of technical possibility and being inspired by perspectives from deformations in the animal world through a Co-I and team from biosciences. Our work will be further grounded and informed by our end-user partner, BBC R&D. The outcome will be scenarios, interaction styles and a range of fully-functional prototypes that will enable us to map out a design space to drive developments in both display materials and a richer, expressive toolkit of gestures and manipulation on touch surfaces.Consider the following scenarios that illustrate the possibilities of the new technology and user interfaces:1. Sam is using the new paint application on her tablet. At the top of the screen is a row of paint pots that dynamically recess into the display. Each contains a colour, and the deeper she dips her paintbrush into the pot, the deeper and thicker she is able to paint onto the remaining screen surface, just as in the real world of watercolour sets.2. Rosie is watching a BBC nature programme on her very large screen HD television set. In her hands she holds a tablet which can display companion content for the programme. While the large screen shows a butterfly dancing around a garden, her tablet screen deforms to provide a sensation of the flapping wings, indicating to Rosie that there is further content on the second screen. However, she is immersed with the main screen display and briskly pushes the deformation away to the right of the mobile. The force of her interaction is used by the system as a gauge to her interruptibility as additional second screen content becomes available.3. Siân is producing a live music event for broadcast over iPlayer. She needs to keep her eyes and ears alive to the pace and spectacle of the band whilst catching the individual performances of each of its members. Normally a sea of controls from an array of mixing desks would surround her, rigged to control every potential shot from the camera crew on and off stage. The new flat panel touch screen mixers enable Siân to customise the controls, reducing the kit required, but without tactile feedback her attention is diverted from the action to the control surface. These micro moments of distraction can be critical and the continual switching of attention increases the stress of the job. The new deformable displays give her best of both worlds: a smaller, responsive mixing desk with tactile feedback that keeps her eyes on the performance she is broadcasting.We propose a number of transformative benefits of the novel surfaces and architectures we will construct:- The three dimensional and multimodal nature of the elements can be used to communicate features of an information space in more efficient and satisfying ways;- The surface can provide sophisticated and nuanced controls with, for instance, the extent to which the user pokes her finger or stylus into the surface adjusting the degree of system response; and,- The tangibility of the elements can enable eyes-free operation, allowing the user to combine their handheld interaction with other objects in their environment in order to deal with a complex content space. The platform can then provide physical affordances and controls, in a dynamic way, as the user interacts with the Internet of Things around them.The team combines world leading researchers with expertise in: user experience, hardware and materials innovation, electronics, design space analysis and biosciences.