Exploring structural regularities for robust 3D reconstruction of urban scenes.
详细信息
文摘
Driven by emergent needs in industrial applications such as film production,navigation and virtual reality,the problem of inferring 3D structures of urban scenes from 2D images has recently drawn a lot of interest in the computer vision community. Despite the extremely rich literature in multiple view geometry and structure from motion SFM),reconstructing large-scale high-quality 3D urban models still remains a challenging problem. A key feature of urban scene that differentiates it from other kinds of landscapes is the presence of strong structural regularities,such as planar surfaces,repetitive structures and all types of symmetries. While such regularities are largely ignored by existing SFM systems,in this thesis we demonstrate how they can be used to greatly facilitate 3D urban reconstruction as well as other related vision tasks. In the first part of the thesis,we first look into the problem of structure and motion recovery directly from one or more large planes in the scene. We develop a new SFM method that generates high-quality reconstruction results in a short time,while avoiding several practical difficulties of conventional methods. Then,we show how the recovered planar structures can be seamlessly integrated into the current state-of-the-art video stabilization systems to obtain high-quality jitter-free videos in many challenging cases. In the second part of the thesis,we focus on the structural regularities in visual data which give rise to a low-rank matrix structure,and develop a series of tools to recover them from images and videos. After reviewing the recent developments of convex optimization techniques for low-rank matrix recovery,we propose a novel 3D reconstruction approach based on a new class of global features called transform invariant low-rank textures TILT). We demonstrate the advantage of such global features over traditional local features in handling large-baseline images,occlusions,and repetitive patterns. In addition,we extend the tools from low-rank matrix recovery to harness the redundancy and temporal correlations among a large number of video frames,which leads to a novel method for generating clean textured models for street views. For future work,my focus is on developing new methods for discovering complex structural regularities in urban scenes from large-scale visual data. I believe such methods would have a big impact in many modern industry applications in the near future.