Research about Realtime Obstruct Eliminating System by using Multi Cameras

在这项研究中，我们提出了一种新的方法来产生新的视角，通过使用多个摄像机来去除障碍物。为了在城市等公共环境中使用，该方法应具有针对摄像机对齐的灵活性和可扩展性。通过从任何角度观察消除障碍的图像，用户可以很好地理解正在进行的事件。因此，我们提出了基于投影变换的算法来实时生成图像。通过对输入图像的特征点进行聚类，并根据聚类点对图像进行细分，该算法可以应用于障碍物位于复杂物体前面的场景。然后，通过基于细分图像的图像变形，我们的方法可以从相机之间的任何视点生成图像。在这项工作中，我们基于该方法实现了一个原型捕获系统。系统由管理PC和多台摄像机、节点PC组成。为了保持系统的性能，摄像机不应该由一台PC来管理，因此我们使用节点PC分别连接到每个摄像机。所有节点PC与管理PC （Pentium 4 XEON 2.8GHz）通过千兆网络（1000BASE-TX）连接。通过该原型系统，我们评估了该方法的实时障碍消除性能和灵活性。在多台摄像机的情况下，消障图像的帧率为7-10帧/秒，延迟时间为160-200帧/秒。在任何相机排列模式中，我们都可以从图像中消除障碍。在消除障碍物的情况下，我们还可以从相机位置之间的任何视点看到场景。结果表明，该方法可以通过随机对齐的摄像机实时消除场景中的障碍物。在未来，通过将该方法应用于公共环境中的安防摄像头，我们可以轻松地了解场景在任何时间、任何地点的空间信息。

In this study, we proposed new approach to generate novel view from which obstacles are removed by using multiple cameras. To be utilized in public environment like city, the approach should have the flexibility and scalability against the alignment of cameras. By observing the obstacle-eliminated images from any point of view, user can understand an exactly ongoing event well.Thus, we proposed projective transformation based algorism to generate images in real time. By clustering feature points of input images and subdividing images depend on the clustered points, our algorism can be applied to scene in which obstacles are located in front of complex objects. Afterward, by morphing images based on the subdivided images, our approach can generate images from any viewpoint between cameras.In this work, we implemented a prototype capturing system based on the approach. The system consists of management PC and many cameras and node PCs. To maintain the performance of the system, the cameras should not be manage by one PC, and therefore we use node PCs which are connected to each camera individually. All node PCs are connected with the management PC (Pentium 4 XEON 2.8GHz) through a gigabit network (1000BASE-TX).By using this prototype system, we evaluated the performance and flexibility of the proposed method for real-time obstacle elimination. In case of several cameras, frame rate of obstacle-eliminated image is 7-10 [fps] and delay time is 160-200 [msec]. And in any patterns of camera alignments, we can eliminate obstacle from images. Also we can see the scene from any viewpoint between the camera positions while eliminating obstacles.These results show that the proposed method can eliminate obstacle from the scene in real-time by using randomly aligned cameras. In the future, by applying the method to security cameras equipped in public environment, we can easily understand the spatial information of the scene in any time in any places.