IndoorSim-To-OutdoorReal: Learning to Navigate Outdoors without Any Outdoor Experience

We present IndoorSim-to-OutdoorReal (I2O), an end-to-end learned visual navigation approach, trained solely in simulated short-range indoor environments, and demonstrate zero-shot sim-to-real transfer to the outdoors for long-range navigation on the Spot robot. Our method uses zero real-world experience (indoor or outdoor), and requires the simulator to model no predominantly-outdoor phenomenon (sloped grounds, sidewalks, etc). The key to I2O transfer is in providing the robot with additional context of the environment (i.e. a satellite map, a rough sketch of a map by a human, etc.) to guide the robot’s navigation in the real-world. The provided context-maps do not need to be accurate or complete– real-world obstacles (e.g. trees, bushes, pedestrians, etc.) are not drawn on the map, and openings are not aligned with where they are in the real- world. Crucially, these inaccurate context-maps provide a hint to the robot about a route to take to the goal. We find that our method that leverages Context-Maps is able to successfully navigate over a hundred meters in novel environments, avoiding The Georgia Tech effort was supported in part by NSF, AFRL, DARPA, ONR YIPs, ARO PECASE. JT was supported by an Apple Scholars in AI/ML PhD Fellowship. The views and conclusions are those of the authors and should not be interpreted as representing the U.S. Government, or any sponsor. 1JT, SC, and DB are with Georgia Institute of Technology. truong.j@gatech.edu 2AZ, TZ, JT, and WY are with Google DeepMind. 3DB is with Meta AI. novel obstacles on its path, to a distant goal without a single collision or human intervention. In comparison, policies without the additional context fail completely. We additionally find that the Context-Map policy is surprisingly robust to noise. In the presence of significantly inaccurate maps in simulation (corrupted with 50% noise, or entirely blank maps), the policy gracefully regresses to the behavior of a policy with no context. Videos are available on our project website.