Death to Proprioception: How Vision-Only Robots are Conquering Spatial Generalization

In a major shift for robotic control, a new study published on September 24th, 2025, by the Spirit AI team reveals that the industry-standard reliance on a robot's internal "proprioceptive" sense—such as joint angles and pose—actually cripples its ability to adapt to new environments. Led by researchers Juntu Zhao and Wenbo Lu, the paper titled "Do You Need Proprioceptive States in Visuomotor Policies?" identifies that these internal state inputs act as a "shortcut" for the AI. Instead of learning to understand the task through visual cues, the robot simply memorizes specific training coordinates, causing it to fail completely when environmental factors like table height or object position are slightly altered.

To solve this, the team introduced the "State-free Policy," which removes internal body data entirely and forces the robot to rely solely on visual observations. By utilizing dual wide-angle wrist cameras and a relative action space, the robot is required to develop a deeper environmental understanding rather than relying on its internal map. This shift has led to a dramatic breakthrough in spatial generalization; in real-world tests involving pick-and-place, shirt-folding, and complex whole-body manipulation, success rates jumped from near zero to as high as 85%. This research proves that "less is more," showing that stripping away internal states allows for more intelligent, adaptable machines across various robot embodiments.

The Rise of the State-free Policy

Researchers propose a radical shift: the State-free Policy, which removes the proprioceptive state input entirely and conditions actions only on visual observations. This forces the policy to develop a deeper, visual-based understanding of the task.

The success of this new vision-first approach rests on two critical technical conditions:

1. Relative EEF Action Space: Actions must be predicted as relative end-effector (EEF) displacements (Δpt​), not absolute positions. This action space naturally supports generalization, as an identical visual observation yields the same relative movement regardless of the robot's absolute starting pose. Absolute position actions, by contrast, fail disastrously in generalization settings.
2. Full Task Observation: Without the proprioceptive safety net, the policy demands a comprehensive view of the entire task and all relevant objects. To ensure this, the researchers employ dual wide-angle wrist cameras (field of view ∼120∘×120∘) mounted on the end-effector.

A Generalization Breakthrough

The results are stark: the State-free Policy achieves an average success rate improvement from 

0% to 85% in height generalization and from 6% to 64% in horizontal generalization across complex real-world tasks like pick-and-place, shirt folding, and whole-body manipulation. Beyond generalization, the state-free approach delivers compelling operational benefits:

• Higher Data Efficiency: Because the policy doesn't overfit to specific trajectories, it requires significantly less fine-tuning data to achieve high performance.
• Better Cross-Embodiment Adaptation: Policies relying only on visual input adapt much faster to new robot hardware (embodiments) since they avoid the issues of aligning different state spaces or reference frames.

Moreover, the research reveals a final, counterintuitive insight: traditional overhead cameras can become a new bottleneck. Changes in the object's location cause distribution shifts in the overhead view, degrading performance in challenging scenarios where the wide-angle wrist cameras alone remain highly successful. The findings make a compelling case for a new era of robot learning systems built solely on comprehensive visual information.

Reference

• Do You Need Proprioceptive States in Visuomotor Policies?