RAMPA: Robotic Augmented Reality for Machine Programming by DemonstrAtion
Fatih DOGANGUN, Serdar Bahar, Yigit YILDIRIM, Bora Toprak Temir, Emre Ugur, Mustafa Doga Dogan
AI summary
Problem
Traditional robot programming relies on time-consuming offline simulation and manual coding, while existing Programming by Demonstration methods face safety concerns, high barriers to entry, and inefficient real-world data collection.
Approach
The authors developed RAMPA, an end-to-end AR-driven framework that integrates commercial XR headsets with robotic arms to record in-situ demonstrations, simulate trajectories, and train ML models directly in the user's physical environment.
Key results
- First end-to-end XR-driven PbD framework enabling in-situ demonstration, ML training, and deployment
- Comprehensive evaluation across three tasks with 20 participants showing improved usability and reduced task load versus kinesthetic control
- Seamless integration of real-time hand-following, trajectory playback/adjustment, and ProMP model conditioning within a commercial AR headset
- Open-source release compatible with standard ROS and URDF-based robotic platforms
Why it matters
It lowers the barrier to robotic programming for both novices and experts, improving operational safety, efficiency, and human-robot collaboration in industrial and dynamic settings.
Abstract
This paper introduces Robotic Augmented Reality for Machine Programming by Demonstration (RAMPA), the first ML-integrated, XR-driven end-to-end robotic system, allowing training and deployment of ML models such as ProMPs on the fly, and utilizing the capabilities of state-of-the-art and commercially available AR headsets, e.g., Meta Quest 3, to facilitate the applica- tion of Programming by Demonstration (PbD) approaches on in- dustrial robotic arms, e.g., Universal Robots UR10. Our approach enables in-situ data recording, visualization, and fine-tuning of skill demonstrations directly within the user’s physical environ- ment. RAMPA addresses critical challenges of PbD, such as safety concerns, programming barriers, and the inefficiency of collecting demonstrations on the actual hardware. The performance of our system is evaluated against the traditional method of kinesthetic control in teaching three different robotic manipulation tasks and analyzed with quantitative metrics, measuring task performance and completion time, trajectory smoothness, system usability, user experience, and task load using standardized surveys. Our findings indicate a substantial advancement in how robotic tasks are taught and refined, promising improvements in operational safety, efficiency, and user engagement in robotic programming.