Unreal Robotics Lab: A High-Fidelity Robotics Simulator with Advanced Physics and Rendering
Jonathan Embley-Riches, Jianwei Liu, Simon Julier, Dimitrios Kanoulas
AI summary
Problem
Existing simulators struggle to simultaneously deliver photorealistic rendering and high-precision physics due to architectural and computational trade-offs. This gap hinders the evaluation of robotic perception and navigation algorithms under realistic, adverse environmental conditions.
Approach
The authors developed an open-source framework that natively embeds the MuJoCo physics engine within Unreal Engine using a shared-memory execution model, enabling deterministic physics synchronized with advanced rendering and dynamic environmental effects.
Key results
- Native shared-memory architecture eliminating client-server synchronization overhead
- Automated pipeline converting Unreal assets to MuJoCo-compatible collision geometries via CoACD
- ROS-compatible middleware enabling plug-and-play deployment of planners and SLAM frameworks
- High visual fidelity between simulated and real scenes with robust navigation benchmarking under adverse conditions
Why it matters
Provides robotics researchers with an open-source, high-fidelity simulation environment for safely testing and transferring vision-based algorithms to real-world robotic systems under diverse and adverse conditions.
Abstract
High-fidelity simulation is essential for robotics research, enabling safe and efficient testing of perception, control, and navigation algorithms. However, achieving both photorealistic rendering and accurate physics modeling remains a challenge. This paper presents a novel simulation framework, the Unreal Robotics Lab (URL), that integrates the advanced rendering capabilities of the Unreal Engine with MuJoCo’s high-precision physics simulation. Our approach enables re- alistic robotic perception while maintaining accurate physical interactions, facilitating benchmarking and dataset generation for vision-based robotics applications. The system supports complex environmental effects, such as smoke, fire, and water dynamics, which are critical to evaluating robotic performance under adverse conditions. We benchmark visual navigation and SLAM methods within our framework, demonstrating its utility for testing real-world robustness in controlled yet diverse scenarios. By bridging the gap between physics accu- racy and photorealistic rendering, our framework provides a powerful tool for advancing robotics research and sim-to-real transfer. Our open-source framework is available at https: //unrealroboticslab.github.io/.