Experience is the Best Teacher: Using Past Performance to Continually Improve Localization, Terrain Assessment, and Path Tracking for Visual Route Following

Tim Barfoot, University of Toronto

For the last several years, we have been working on a vision-only, route-following technique for mobile robots, called Visual Teach & Repeat (VT&R). This approach is successful in practice because it (i) exploits human experience in route definition, (ii) avoids the need to build a global map of the world, and (iii) plays to the strengths of computer vision by keeping the viewpoints the same between teach and repeat. However, to scale up to real-world operations, we need to be able to repeat routes as quickly as possible, in the presence of dynamic obstacles, and of course deal with visual change (lighting, weather, etc.). In this talk, I will describe our latest approach, VT&R 2.0, that leverages past route-driving experience not only to improve visual localization, but also to improve our ability to detect obstacles visually in difficult scenarios (e.g., tall grass), and even track paths more accurately and more quickly over time. I will present field test results using our 1000 kg Grizzly mobile robot in a variety of outdoor, offroad scenarios.