Cooperative Mapping

UMN Researchers

 

Google Researchers

  • Esha Nerurkar and Joel Hesch

 

Summary

We address the problem of cooperative mapping (CM) using datasets collected by multiple users at different times, when the transformation between the users' starting poses is unknown. Specifically, we formulate CM as a constrained optimization problem, where each user's independently-estimated trajectory and map are merged together by imposing geometric constraints between commonly-observed point and line features.

Diagram of three phones and tablets, each moving along its own path in its own coordinate frame, observing shared points, stars, and lines



The line features are tracked by considering the geometric constraints between the line segments in consecutive image pairs. Additionally, line loop-closures (intra or inter-dataset) are detected by employing geometric tests on the optimized trajectories and maps. Furthermore, we provide an algorithm for efficiently solving the CM problem by taking advantage of its special structure. The proposed solution is proven to be batch least-squares (BLS) optimal over all users' datasets, while it has low memory requirements and lends itself, for the most part, to parallel implementations. Moreover, our algorithm is resource-aware as it is able to trade positioning accuracy for computational cost in a consistent manner, by reducing the number of commonly-observed feature constraints.

Algorithm Overview

Main steps:

  • Compute a BLS estimate for each user's trajectory and map independently using their own visual-inertial data
  • Track lines by using the trajectory estimates and line segments in the images.
  • Generate an initial estimate of the users' relative poses based on their observations of common features
  • Find the optimal BLS solution of all users' trajetories and maps utilizing all available sensor data, and all (or a subset of) the common-feature constraints
  • Find line loop-closure features based on visibility and geometric tests.
Flowchart: maps from three users feed into an initialization step, then cooperative mapping, shown on a computer monitor

Visualization

An interactive visualization of various conference-site and building maps can be found at the Onion Maps website.

The estimated map of Keller Hall at University of Minnesota

Overhead view of a building’s corridors as a white point cloud on a grid, with a pink camera marker at the lower left

The estimated map of Walter library at University of Minnesota

Five views of one 3D point-cloud map; the top-left overhead view on a grid has red, green, blue, and cyan device trajectories overlaid

Video

Relevant Publications

  • P1. C.X. Guo, R.C. DuToit, K. Sartipi, G.Georgiou, R. Li, J. O'Leary, E.D. Nerurkar, J.A. Hesch, and S.I. Roumeliotis, "Resource-Aware Large-Scale Cooperative 3D Mapping from Multiple Cell Phones," In ICRA Late Breaking Results Poster, Washington, May. 26-30, 2015.
  • C1. C.X. Guo, K. Sartipi, R.C. DuToit, G.Georgiou, R. Li, J. O'Leary, E.D. Nerurkar, J.A. Hesch, and S.I. Roumeliotis, "Large-Scale Cooperative 3D Visual-Inertial Mapping in a Manhattan World," ICRA, 2016 (submitted).