Motion Gestures as an Input Modality for Modern Smartphones

点击翻译按钮获取中文摘要

While Apple Corporation's recent release of the Siri voice recognition assistant has illustrated the potential benefits of speech as input to smartphones, speech recognition is not a panacea for the input limitations of modern smartphones. Social and physical contexts (e.g. being in a meeting, being at a concern) restrict the use of spoken dialog as an input modality for smartphone control. As a result, there is a benefit to exploring a broad set of alternative smartphone input modalities. The goal of this grant is to promote motion gestures, i.e. movement of the entire smartphone device, as an input modality for smartphones. Hand motion -- pointing, gesturing, grasping, shaking, tapping -- is a rich channel of communication. We point and gesture while we talk; we grasp tools to extend our capabilities; we grasp, rotate, and shake items to explore them. Yet, the rich repertoire of hand motion is largely ignored in interfaces to mobile computation: the user of a modern smartphone generally holds the device stationary while tapping or swiping its surface. Why are so many possible affordances ignored? Certainly not for technical reasons, as smartphones contain an evolving set of sensors for recognizing movement of the phone, including accelerometers, gyroscopes and cameras. However, beyond rotating to change screen orientation or shaking to shuffle songs, little has been done to enable rich gestural input through device motion. To develop motion gestures as an input modality for modern smartphones, users must learn the language of motion gestures that a smartphone can recognize; smartphones must recognize motion gestures that users perform; and designers of smartphone software must understand the strengths and weaknesses of motion gestures as an input modality. Research in these areas can open up the rich, effortless language of gesture, enhancing the user's command of the computational potential of mobile devices.