← Back ICRA 2026

Modeling and Reinforcement Learning-Based Control of Simultaneous Positive and Negative Pressure Generation in Pneumatic Systems

Sang Hyeon Park, Myeongyun Doh, Chanyong Park, Tuan Luong, Hyouk Ryeol Choi, Ja Choon Koo, Hugo Rodrigue, Hyungpil Moon

PDF

AI summary

Key figure (auto-extracted from paper)

A reinforcement learning controller with predictive observation and action smoothing accurately regulates simultaneous positive and negative pressures in a compact single-pump system, outperforming traditional methods and maintaining robustness under heavy loads.

Reinforcement learning pneumatic control soft robotics dual-pressure regulation sim-to-real transfer multiphysics simulation

Problem

Compact pneumatic systems using a single pump for simultaneous positive and negative pressures face severe pressure fluctuations due to coupled nonlinear dynamics, which conventional linear or rule-based controllers struggle to manage across wide operating ranges.

Approach

The authors built a multiphysics simulator incorporating pump kinematics and valve electromagnetics to train a reinforcement learning controller that uses Model-Predicted Observation to anticipate system behavior and CAPS-based smoothing to stabilize control inputs during real-world deployment.

Key results

Achieved pressure tracking RMSEs of 0.6935 kPa (positive) and 0.2646 kPa (negative)
Outperformed conventional Disturbance Observer-based control across wide pressure ranges
Validated synergistic effectiveness of MPObs and CAPS through ablation studies
Maintained robustness under 0–20 kg external loads with maximum RMSEs of 0.7906 kPa and 0.1186 kPa

Why it matters

Provides a foundational control framework for compact, stand-alone pneumatic power sources, advancing the practical deployment of mobile and soft robots in field applications.

Abstract

In soft robotics, actuators using both positive and negative pressures are notable for their high payload-to-weight ratios and wide operating ranges, but they require separate power sources. A single-pump system generating dual pressures presents a promising solution, though addressing pressure fluctuations due to coupled dynamics remains a challenge. In this work, we propose a reinforcement learning (RL)-based controller capable of tracking both pressures over a wide range. To facilitate RL training, we built a simulator that models not only airflow dynamics but also the pump’s kinematics and the electromagnetic behavior of pneumatic components. Our controller employs Model-Predicted Observation (MPObs) to predict future input effects and mitigate nonlinearities, and uses a Conditioning for Action Policy Smoothness (CAPS)- based action smoothing to reduce abrupt input changes. Exper- imental results show that the proposed RL controller achieves root-mean-square errors (RMSEs) of 0.6935 kPa (positive) and 0.2646 kPa (negative), outperforming the Disturbance Observer (DOB)-based approach. Ablation studies confirm the synergistic effect of MPObs and CAPS, underscoring their importance in control. Furthermore, robustness tests with external loads from 0 to 20 kg demonstrate a maximum RMSE of 0.7906 kPa (posi- tive) and 0.1186 kPa (negative), indicating strong robustness. This study verifies that our proposed RL-based controller overcomes the nonlinear challenges of pneumatic power sources and highlights its potential for future stand-alone systems in field applications.

Index terms

Reinforcement Learning Modeling Control and Learning for Soft Robots