Semi-Dense Mapping

UMN Researchers

Mrinal K. Paul and Stergios Roumeliotis (PI)

Summary

The goal of this project is to generate a semi-dense 3D map of an area, given as prior the camera pose estimates from a sparse visual-inertial batch least squares (BLS) cooperative-mapping (CM) algorithm. Specifically:

  1. To densify the map, we consider all pixels that have a large gradient along the epipolar line

  2. To track features, we perform sub-pixel stereo matching between consecutive keyframes; matches are found using the sum of squared distance (SSD) error measured over a linear patch of five equidistant points on the epipolar line

  3. Outlier rejection is based on the convexity of the photometric cost function, uncertainty of the inverse depth estimate, and depth inconsistency in the neighbourhood of the feature

  4. Inlier feature tracks are processed using BLS to refine the feature estimates. In this step, instead of a constant measurement noise for the dense features, we use the inverse of the information matrix, whose diagonal elements are proportional to the features' local intensity gradients (gx, gy)

Results
 

Sparse scattering of white points on a dark background, with a camera photo of the staircase in the lower-left corner
Input: The sparse map of a portion of the scene (from the first BLS iteration)


 

Dense 3D point cloud of a staircase with railings in a dark hall, with the source camera photo of the stairs in the lower-left corner
Output: A semi-dense map of the same portion of the scene (from the final BLS iteration)

Videos

[video unavailable]

Relevant Publications

  • P1. C.X. Guo, R.C. DuToit, K. Sartipi, G.Georgiou, R. Li, J. O'Leary, E.D. Nerurkar, J.A. Hesch, and S.I. Roumeliotis, "Resource-Aware Large-Scale Cooperative 3D Mapping from Multiple Cell Phones," In ICRA Late Breaking Results Poster, Washington, May. 26-30, 2015.