Breaking the Glass: Multimodal, Malleable Interactive Mobile surfaces for Hands-In Interactions

手机和平板电脑的触摸屏是平面的、死板的。我们的工作旨在探索一种流动性更强、更有活力的便携式显示器的潜力。它从根本上背离了现有的可变形研究，努力提供与可视显示器中集成的负担、控制和内容的真正直接交互。我们将雄心勃勃，敢于冒险，突破技术可能性的界限，通过Co-I和生物科学团队从动物世界的变形角度获得灵感。我们的工作将进一步扎根于我们的最终用户合作伙伴BBC R&D，并由其提供信息。结果将是场景、交互方式和一系列全功能原型，使我们能够规划出一个设计空间，以推动显示材料和更丰富、更具表现力的工具包在触摸表面上的开发。考虑以下场景，说明新技术和用户界面的可能性：1.Sam正在她的平板电脑上使用新的Paint应用程序。在屏幕的顶部是一排油漆罐，它们动态地凹入显示屏。每种颜色都包含一种颜色，她把画笔浸入锅中越深，她就能在剩余的屏幕表面上画得越深越厚，就像在现实世界中的水彩布景一样。罗西正在她的超大屏幕高清电视上观看BBC自然节目。她手里拿着一台平板电脑，可以显示节目的配套内容。虽然大屏幕上显示了一只蝴蝶在花园里跳舞，但她的平板电脑屏幕变形了，提供了一种拍打翅膀的感觉，这向罗西表明，第二个屏幕上有更多内容。然而，她沉浸在主屏幕显示中，轻快地将变形推到手机的右侧。当额外的第二屏幕内容变得可用时，系统使用她的交互的力量作为她的可插入性的量度。西恩正在制作一场现场音乐活动，通过iPlayer进行转播。她需要让自己的眼睛和耳朵保持对乐队的节奏和壮观场面的关注，同时捕捉到每个成员的个人表演。通常情况下，来自一系列调音台的大量控制会包围她，这些调音台被操控着控制摄制组在舞台上和舞台下的每一个潜在镜头。新的平板触摸屏搅拌器使Sián能够定制控制，减少了所需的套件，但如果没有触觉反馈，她的注意力就会从操作转移到控制表面。这些微小的分心时刻可能是至关重要的，持续的注意力转移增加了工作的压力。新的可变形显示器给了她两个世界的最好：一个更小的，反应灵敏的调音台，带触觉反馈，让她的眼睛保持在她正在转播的表演上。我们提出了我们将构建的新颖表面和体系结构的一些变革性的好处：-元素的三维和多通道性质可以用更有效和更令人满意的方式来传达信息空间的特征；-表面可以提供复杂和细微的控制，例如，用户将手指或触控笔伸进表面调整系统响应的程度；而且，元素的有形可以实现无眼睛操作，允许用户将他们的手持交互与其环境中的其他对象相结合，以便处理复杂的内容空间。然后，当用户与周围的物联网互动时，该平台可以以动态的方式提供物理负担和控制。该团队将世界领先的研究人员与以下领域的专业知识结合在一起：用户体验、硬件和材料创新、电子学、设计空间分析和生物科学。

Mobile phone and tablet touch screens are flat, dead surfaces. Our work seeks to explore the potential of a far more fluid, 'alive' portable display. It departs radically from existing deformable research by endeavouring to provide truly direct interaction with affordances, controls and content integrated within a visual display. We will be highly ambitious and adventurous, pushing the boundaries of technical possibility and being inspired by perspectives from deformations in the animal world through a Co-I and team from biosciences. Our work will be further grounded and informed by our end-user partner, BBC R&D. The outcome will be scenarios, interaction styles and a range of fully-functional prototypes that will enable us to map out a design space to drive developments in both display materials and a richer, expressive toolkit of gestures and manipulation on touch surfaces.Consider the following scenarios that illustrate the possibilities of the new technology and user interfaces:1. Sam is using the new paint application on her tablet. At the top of the screen is a row of paint pots that dynamically recess into the display. Each contains a colour, and the deeper she dips her paintbrush into the pot, the deeper and thicker she is able to paint onto the remaining screen surface, just as in the real world of watercolour sets.2. Rosie is watching a BBC nature programme on her very large screen HD television set. In her hands she holds a tablet which can display companion content for the programme. While the large screen shows a butterfly dancing around a garden, her tablet screen deforms to provide a sensation of the flapping wings, indicating to Rosie that there is further content on the second screen. However, she is immersed with the main screen display and briskly pushes the deformation away to the right of the mobile. The force of her interaction is used by the system as a gauge to her interruptibility as additional second screen content becomes available.3. Siân is producing a live music event for broadcast over iPlayer. She needs to keep her eyes and ears alive to the pace and spectacle of the band whilst catching the individual performances of each of its members. Normally a sea of controls from an array of mixing desks would surround her, rigged to control every potential shot from the camera crew on and off stage. The new flat panel touch screen mixers enable Siân to customise the controls, reducing the kit required, but without tactile feedback her attention is diverted from the action to the control surface. These micro moments of distraction can be critical and the continual switching of attention increases the stress of the job. The new deformable displays give her best of both worlds: a smaller, responsive mixing desk with tactile feedback that keeps her eyes on the performance she is broadcasting.We propose a number of transformative benefits of the novel surfaces and architectures we will construct:- The three dimensional and multimodal nature of the elements can be used to communicate features of an information space in more efficient and satisfying ways;- The surface can provide sophisticated and nuanced controls with, for instance, the extent to which the user pokes her finger or stylus into the surface adjusting the degree of system response; and,- The tangibility of the elements can enable eyes-free operation, allowing the user to combine their handheld interaction with other objects in their environment in order to deal with a complex content space. The platform can then provide physical affordances and controls, in a dynamic way, as the user interacts with the Internet of Things around them.The team combines world leading researchers with expertise in: user experience, hardware and materials innovation, electronics, design space analysis and biosciences.