Intelligent Image Processing and Signal Processing for wearable computing and AR (augmediated reality)

用于可穿戴计算和 AR（增强现实）的智能图像处理和信号处理

• 批准号: RGPIN-2014-06418
• 负责人: Mann, Steve
• 金额: $ 1.82万
• 依托单位: University of Toronto
• 依托单位国家: 加拿大
• 项目类别: Discovery Grants Program - Individual
• 财政年份: 2018
• 资助国家: 加拿大
• 起止时间: 2018-01-01 至 2019-12-31
• 项目状态: 已结题
• 来源: https://www.nserc-crsng.gc.ca/ase-oro/Details-Detailles_eng.asp?id=658660
• 关键词: Intelligent; Image; Processing; Signal; wearable

With ubiquitous mobile/portable imaging and computing now a reality, and wearable computing (e.g. Digital Eye Glass) just becoming a reality, we are entering an era in which our everyday lives are being or will soon be mediated by vision-based computers.**Many of us already carry a camera and computing device (smartphone or cameraphone) most of the time. These devices will soon take the form of Digital Eye Glass to help many of us see better, and improve the quality of our lives. This technology will be especially welcome to the growing population of older individuals with failing eyesight.**Moreover, as technologies become more personal, the Digital Eye Glass will help us remember names and faces (e.g. the wearable face recognizer), find our way (digital maps overlayed with Augmediated Reality), and provide us with improved safety (e.g. personal security), health (sensing, etc.) and well-being.**From my long-standing personal experience (more than 35 years of inventing, designing, building, and experimenting with wearable computing and Digital Eye Glass since my childhood -- first as an amateur scientist in my youth, and later in my professional life), I have helped this industry evolve.**Much of the industry is addressing immediate business interests, but there is a need for research on the foundational aspects of Intelligent Image Processing specifically related to personal imaging devices (small portable or wearable camera systems).**In particular, a core issue that remains is getting cameras to "see" as well as the human eye does. Already the spatial resolution of cameras has increased from thousands of pixels to megapixels, but image quality (e.g. dynamic range) at each pixel has not kept pace.**The first digital camera was invented (by Steven Sasson of Kodak) in 1975, and had 10,000 pixels (based on a Fairchild 100*100 pixel sensor), and 4 bits-per pixel per channel (1 channel, i.e. greyscale).**Today, mobile and portable camera phones are up to 41 Megapixels (e.g. Nokia PureView 808), and some cameras are in the gigapixel range, i.e. spatial resolutions that are more than 100,000 times what they were in 1975. Thus many cameras can "see" the detail (e.g. "read" small print on a newspaper) thousands of times better now than then.**But they still cannot match the human eye for dynamic range (i.e. being able to simultaneously sense in low-light and really bright light).**Remarkably, the bit depth of modern cameras is typically only twice what the world's first digital camera was 38 years ago (8 bits compared to 4 bits).**Recently I have addressed this problem by inventing and steadily improving something called HDR (High Dynamic Range) Imaging.**Robertson et al. write:**"The first report of digitally combining multiple pictures of the same scene to improve dynamic range appears to be Mann [Mann 1993]"*-- [Estimation-theoretic approach to dynamic range enhancement using multiple exposures, JEI 12(2)].***HDR is now used by many commercially manufactured cameras, including Apple's iPhone (which implements HDR internally), as well as many surveillance cameras and other specialized imaging devices.**My work will involve basic research in creation of an Intelligent Image Procesing chip for widespread use. The work involves development of new branches of mathematics and engineering, combining comparametric equations, superposimetric equations, functional equations, and quantum field theory, with FPGA (Field Programmable Gate Array) architectures. Another goal is to create Actional Systems (a new kind of kinematics) for circuit modeling. This fundamental work will be of great use to other scientists and engineers who work with image processing, Digital Eye Glass, and AR (Augmediated Reality).

随着无处不在的移动/便携式成像和计算已经成为现实，可穿戴计算（例如数字眼镜）刚刚成为现实，我们正在进入一个日常生活正在或即将由基于视觉的计算机调解的时代。**我们中的许多人大部分时间都携带着相机和计算设备（智能手机或照相手机）。这些设备将很快以数字眼镜的形式出现，帮助我们中的许多人看得更清楚，提高我们的生活质量。这项技术将特别受到越来越多的视力衰退的老年人的欢迎。**此外，随着技术变得更加个性化，数字眼镜将帮助我们记住名字和面孔（例如可穿戴面部识别器），找到我们的路（与增强现实叠加的数字地图），并为我们提供更好的安全（例如个人安全），健康（传感等）和福祉。**从我长期的个人经验来看（从我的童年开始，我就从事可穿戴计算和数字眼镜的发明、设计、建造和实验超过35年——首先是在我年轻时作为业余科学家，后来是在我的职业生涯中），我帮助了这个行业的发展。**许多行业正在解决直接的商业利益，但有必要研究智能图像处理的基础方面，特别是与个人成像设备（小型便携式或可穿戴相机系统）。**特别是，仍然存在的一个核心问题是让相机像人眼一样“看到”。相机的空间分辨率已经从数千像素提高到百万像素，但每个像素的图像质量（例如动态范围）却没有跟上。**第一台数码相机是在1975年发明的（由柯达的Steven Sasson），有10,000像素（基于仙童100*100像素传感器），每个通道每像素4位（1通道，即灰度）。**今天，移动和便携式照相手机高达4100万像素（例如诺基亚PureView 808），一些相机在十亿像素范围内，即空间分辨率是1975年的10万多倍。因此，许多摄像机可以“看到”细节(例如：（阅读报纸上的小字）现在比那时好几千倍。**但它们在动态范围上仍然无法与人眼相匹配（即能够同时在低光和强光下感知）。**值得注意的是，现代相机的比特深度通常只有38年前世界上第一台数码相机的两倍（8比特比4比特）。**最近我通过发明和稳步改进HDR（高动态范围）成像来解决这个问题。**Robertson等人写道：**“Mann [Mann 1993]首次报道了对同一场景的多张图片进行数字组合以提高动态范围”*—[使用多次曝光增强动态范围的估计理论方法，JEI 12(2)]。***HDR现在被许多商用相机所使用，包括苹果的iPhone（它在内部实现了HDR），以及许多监控摄像头和其他专业成像设备。**我的工作将涉及基础研究，创造一个智能图像处理芯片的广泛使用。这项工作涉及数学和工程的新分支的发展，将共参数方程、叠加方程、泛函方程和量子场论与FPGA（现场可编程门阵列）架构相结合。另一个目标是为电路建模创建动态系统（一种新的运动学）。这项基础工作将对其他从事图像处理、数字眼镜和AR（增强现实）工作的科学家和工程师有很大的用处。