Mobile Manipulation Leveraging Multiple Views

Abstract

While both navigation and manipulation are challenging topics in isolation, many tasks require the ability to both navigate and manipulate in concert. To this end, we propose a mobile manipulation system that leverages novel navigation and shape completion methods to manipulate an object with a mobile robot. Our system utilizes uncertainty in the initial estimation of a manipulation target to calculate a predicted next-best-view. Without the need of localization, the robot then uses the predicted panoramic view at the next-best-view location to navigate to the desired location, capture a second view of the object, create a new model that predicts the shape of object more accurately than a single image alone, and uses this model for grasp planning. We show that the system is highly effective for mobile manipulation tasks through simulation experiments using real world data, as well as ablations on each component of our system.

Video

Citation

If you use Mobile Manipulation Leveraging Multiple Views in your work please cite our paper!

@misc{watkinsvalls2021mobile,
      title={Mobile Manipulation Leveraging Multiple Views}, 
      author={David Watkins-Valls and Peter K Allen and Henrique Maia and Madhavan Seshadri and Jonathan Sanabria and Nicholas Waytowich and Jacob Varley},
      year={2021},
      eprint={2110.00717},
      archivePrefix={arXiv},
      primaryClass={cs.RO}
}

Arxiv link here

Acknowledgements

¹Department of Computer Science, Columbia University, New York, NY, USA. Corresponding author: David Watkins {davidwatkins} at cs dot columbia dot edu

²U.S. Army Research Laboratory, Baltimore, MD, USA.

³Robotics at Google, New York, NY, USA.

This research was sponsored by the Army Research Laboratory and was accomplished under Cooperative Agreement Number W911NF-18-2-0244. The views and conclusions contained in this document are those of the authors and should not be interpreted as representing the official policies, either expressed or implied, of the Army Research Laboratory or the U.S. Government. The U.S. Government is authorized to reproduce and distribute reprints for Government purposes notwithstanding any copyright notation herein.