PhotoAgent: A Robotic Photographer with Spatial and Aesthetic Understanding
to achieve one-shot success. (b) highlights three key capabilities.
AI summary
Problem
Robotic photography lacks a method to bridge high-level aesthetic language commands with precise geometric camera control, often relying on brittle rules or costly physical trial-and-error.
Approach
The agent converts subjective goals into geometric constraints via chain-of-thought reasoning, solves for an initial pose analytically, and iteratively refines it through visual feedback in a real-time 3D Gaussian Splatting simulator.
Key results
- State-of-the-art performance in language-conditioned spatial reasoning
- Superior image aesthetics and instruction fidelity over baselines
- Novel anchor-point hypothesis for intent parsing
- Rapid convergence via closed-loop 3DGS visual reflection
Why it matters
Provides a scalable, geometry-faithful framework for embodied creative agents, advancing robotics toward human-like compositional photography without expensive real-world interaction.
Abstract
Embodied agents for creative tasks like photogra- phy must bridge the semantic gap between high-level language commands and geometric control. We introduce PhotoAgent, an agent that achieves this by integrating Large Multimodal Models (LMMs) reasoning with a novel control paradigm. PhotoAgent first translates subjective aesthetic goals into solvable geometric constraints via LMM-driven, chain-of-thought (CoT) reasoning, allowing an analytical solver to compute a high-quality initial viewpoint. This initial pose is then iteratively refined through visual reflection within a photorealistic internal world model built with 3D Gaussian Splatting (3DGS). This “mental simulation” replaces costly and slow physical trial-and-error, enabling rapid This work was supported by the Natural Science Foundation of Shenzhen (No. JCYJ20230807111604008, No. JCYJ20240813112007010), the Natural Science Foundation of Guangdong Province (No. 2024A1515010003) and Cross-disciplinary Fund for Research and Innovation (No. JC2024002) of Tsinghua SIGS. *indicates equal contribution. 1Center for Artificial Intelligence and Robotics, Shenzhen International Graduate School, Tsinghua University, Shenzhen 518055, China, {clr24@mails., gzf24@mails., cyb23@mails., tjblql@sz., wang.xq@sz.}tsinghua.edu.cn †Corresponding author: Junbo Tan convergence to aesthetically superior results. Evaluations confirm that PhotoAgent excels in spatial reasoning and achieves superior final image quality.