Realman Robotics Open-Sources World's First High-Quality Multimodal Real-Robot Dataset

Generating high-quality, task-relevant training data for robot manipulation remains one of the core bottlenecks in embodied AI research. While simulation-to-real transfer and synthetic data generation have made incremental progress, real-robot (real-machine) data collected from physical hardware continues to represent the highest-fidelity input for training generalizable manipulation policies. 

Addressing this gap, Realman Robotics  — through its wholly-owned subsidiary RealSource — formally announced on November 21 the open-source release of what it claims is the world's first high-quality, modality-complete real-robot dataset, collected across ten real-world operational environments at the Beijing Humanoid Robot Data Training Center.

Project homepage: https://realmanrobot.github.io/real_source_dataset

Data Collection Infrastructure: Ten Real-World Operational Scenarios

All data was collected at the Beijing Humanoid Robot Data Training Center, a 3,000 m² facility structured into two functional zones. The first zone is a dedicated foundational training area focused on high-throughput, batch acquisition of basic pick-and-place manipulation primitives. The second zone replicates ten industry-specific environments designed to reflect real deployment conditions, including: domestic care and assisted living, smart home interaction, agricultural harvesting, new retail, automotive assembly, and food service.

The smart home zone reproduces full residential interiors — living room, bedroom, bathroom, and kitchen — and covers tasks such as opening/closing refrigerator doors, folding laundry, and bathroom cleaning. The industrial zone simulates factory floor workflows including material sorting, packaging, and logistics handling. By collecting data in functionally realistic settings rather than controlled laboratory environments, the dataset is designed to embed real-world variability — including occlusion, surface variation, and lighting inconsistency — directly into the training distribution.

Dataset Quality Metrics

The dataset achieves 100% modality completeness across all recorded episodes. Anti-noise scoring reaches 78%, and a smoothness score of 82.1% (above the defined excellence threshold) indicates low trajectory jitter and high temporal consistency in recorded joint states. These metrics reflect the pipeline's capacity for stable data capture across high-speed manipulation sequences.

Technical Architecture: Six Core Engineering Properties

The dataset covers the full perception-decision-actuation pipeline and integrates the following modalities: RGB imagery, joint position, joint velocity, end-effector six-axis force/torque (F/T), end-effector pose, action commands, hardware timestamps, and calibrated camera intrinsics/extrinsics. Six engineering properties define its technical baseline:

1. Hardware-Level Temporal Synchronization

A global hardware synchronization signal establishes a unified clock source across all visual sensors, enabling precise per-frame timestamping. This ensures all modalities — including camera streams, joint encoders, and F/T sensors — are referenced to a common spatiotemporal coordinate frame, eliminating cross-modal temporal misalignment.

2. Sub-0.5% Frame Drop Rate

A stringent frame drop threshold of less than 0.5% is enforced through an optimized data transmission pipeline with buffered write management. This guarantees stream continuity even during high-velocity motion or peak-throughput write operations.

3. High-Frequency Joint State Logging

The system samples and records joint angle, angular velocity, and angular acceleration at high frequency in real time, enabling millisecond-resolution insight into robot dynamic state. This also supports real-time high-precision joint velocity control, ensuring smooth and accurate trajectory execution.

4. Factory-Level Camera Calibration

Each data collection robot undergoes factory-level calibration prior to deployment, and the complete camera parameter set (intrinsics and extrinsics) is bundled with the dataset. Users can immediately integrate the data into algorithm development or model training pipelines without performing additional calibration steps.

5. Systematic Generalization-Oriented Collection Protocol

For each core manipulation task, data is collected across systematically varied conditions: object material, geometry, and scale; ambient lighting and scene layout; execution trajectory and viewpoint. This "same task, high variance" collection protocol is designed to encode real-world distributional diversity into the training corpus, directly targeting sim-to-real generalization failure modes.

6. Isomorphic Exoskeleton Teleoperation

Teleoperation is performed using an exoskeleton device that is mechanically isomorphic to the data collection robot's arm. This 1:1 structural correspondence allows the operator's joint angles to be directly mapped to the follower arm without inverse kinematics (IK) computation. The approach eliminates kinematic solving latency, reduces control loop delay to below 10 ms, and produces motion trajectories that more faithfully replicate natural human operator intent compared to dissimilar teleoperation interfaces.

Hardware Platform: RS-01, RS-02, RS-03 Data Collection Robots

The dataset was collected using three hardware platforms developed by RealSource specifically for embodied AI data acquisition. All three robots are equipped with large-FOV cameras (90° horizontal / 65° vertical field of view) at the head and wrist positions, and operate under the global hardware temporal synchronization described above. Key arm specifications: rated payload 5 kg (single arm, full extension), maximum payload 9 kg, TCP linear velocity 1.8 m/s. The system is CR L3 certified with a mean time between failures (MTBF) of 50,000 hours. Power consumption remains below 100 W.

RS-01: Wheeled Folding Mobile Manipulation Platform

• 20 degrees of freedom total; multi-modal visual perception plus spatial navigation
• Six-axis F/T sensor at end-effector; interchangeable end-effectors (parallel gripper, five-finger dexterous hand)

• Self-weight optimized for mobile operation in constrained environments

RS-02: Dual-Arm Liftable Standard Platform

• Dual arms, 7 DoF each; head-mounted RGB-D camera for object recognition and depth estimation
• Six-axis F/T sensing at end-effector; fisheye overhead camera for wide-area scene monitoring

• Optimized for multi-scenario, high-throughput data collection workflows

RS-03: Dual-Arm Binocular Enhanced Platform

• Upgrades RS-02 visual system to human-eye-layout binocular stereo vision for finer near-field depth resolution
• Binocular configuration specifically designed to generate stereo visual data closer to human perceptual geometry for embodied AI model training

• Retains 9 kg single-arm payload and full F/T sensing capability

Data collection modalities are extensible: the hardware supports teleoperation via physical exoskeleton, motion capture (MoCap), and VR-based interfaces. Peripheral sensor options include tactile sensors, additional F/T units, and LiDAR-based navigation modules.

Context and Roadmap

This release positions real-robot data as a critical infrastructure layer for scaling embodied AI — analogous to the role large internet text corpora played in large language model development. By open-sourcing the dataset, Realman and RealSource aim to lower barriers to manipulation policy research for both academic and industrial groups without access to large-scale physical data collection infrastructure.

The company has indicated plans for continued iteration and release of additional high-quality datasets spanning broader task domains and additional modalities, with the stated goal of building a structured open-data ecosystem for robotics research and industrial deployment.