Content-Based Haptic Texture Retrieval

当在物体的表面上敲击刚性工具时，在工具上引起振动。通过加速度计测量，相应的加速度信号携带有关表面纹理的重要信息，可用于分类或识别表面材料。在拟议的研究项目中，我们计划超越纹理识别。我们的目标是开发的方法和算法，允许检索的感知相似的纹理从数据库中可用的纹理。纹理检索的一个有形的未来应用是产品浏览和定制，例如用于汽车内饰或家具材料的选择。基于个人偏好，可以检索具有与之前选择的材料类似的触觉感觉的材料的子集。我们把这个新的概念称为基于内容的触觉纹理检索（CBHTR）CBHTR的基本成分是数学指纹表示的纹理，所谓的功能，是强大的外部影响。当人类在物体表面上敲击刚性工具时，在表面探测期间以及在随后的探测会话之间，所施加的力、扫描速度或工具与表面之间的倾斜度可能会急剧变化。这样的扫描时间参数严重影响所获取的加速度信号的性质。目前缺乏用于工具介导的触觉纹理探索的扫描不变特征。一旦有了扫描不变的特征，就可以建立鲁棒的纹理识别系统。纹理检索系统，这是能够返回感知相似的材料，需要额外捕获纹理之间的感知相似性的功能。受人类感知纹理方式启发的特征是沿着这条线有希望的候选者。总之，发展强大的和感知相关的功能构成了本提案的第一个主要目标。第二个目标是在一个原型纹理识别和纹理检索系统中应用和评估这些功能，后者应该在数据库中找到所有感知相似的纹理。为此，需要确定特征和机器学习方法的适当组合。最后，为了进一步提高检索性能，我们还计划将其他低成本传感器（如摄像头和麦克风）在交互过程中感知到的信息与加速度计收集的信息融合。

While stroking a rigid tool over the surface of an object, vibrations are induced on the tool. Measured with an accelerometer, the corresponding acceleration signals carry important information about the surface texture and can be used to classify or to recognize the surface material. In the proposed research project we plan to go beyond texture recognition. Our objective is to develop methods and algorithms, which allow for the retrieval of perceptually similar textures from a database of available textures. A tangible future application for texture retrieval is product browsing and customization, for example for the selection of materials for car interiors or for furniture. Based on personal preferences, a subset of materials possessing a haptic feel similar to the one chosen before may be retrieved. We refer to this novel concept as Content-Based Haptic Texture Retrieval (CBHTR).Essential ingredients of CBHTR are mathematical fingerprints representing the texture, so-called features, that are robust against external influences. When a human strokes a rigid tool over an object surface, the applied force, the scan-velocity or the inclination between the tool and the surface might vary drastically during the surface exploration and between subsequent exploration sessions. Such scan-time parameters heavily influence the nature of the acquired acceleration signals. Scan-invariant features for tool-mediated haptic texture exploration are currently lacking. Once scan-invariant features are available, robust texture recognition systems can be built. Texture retrieval systems, which are able to return perceptually similar materials, require features that additionally capture the perceptual similarity between textures. Features that are inspired by the way humans perceive textures are promising candidates along this line. Together, the development of robust and perceptually relevant features constitutes the first main objective of this proposal.The second objective is to apply and evaluate these features in a prototypical texture recognition and texture retrieval system, where the latter is supposed to find all perceptually similar textures in a database. To this end, appropriate combinations of features and machine learning approaches need to be identified. A systematic evaluation using subjective similarity scores plays a special role in this context.Finally, in order to further improve the retrieval performance, we additionally plan to fuse the information sensed during the interaction by other low-cost sensors like cameras and microphones with the information gathered with the accelerometer.