Breaking

Stanford and TRI release HoMMI, learning whole-body robot manipulation from human demos

July 1, 2026 at 15:20 EDT

Robotics
Research & Papers
Open Source

Researchers at Stanford University and the Toyota Research Institute have released HoMMI, a framework that learns whole-body mobile manipulation directly from human demonstrations. It collects data without teleoperation and can learn tasks that combine locomotion with object manipulation. The work has been accepted to RSS 2026.

RSS 2026 · Stanford + Toyota Research Institute

HoMMI: teaching robots whole-body mobile manipulation — with zero robots

A new open-source framework learns long-horizon mobile manipulation purely from human demonstrations. Three iPhones capture the motion; no teleoperation, no robot in the loop.

robots needed to collect training data

iPhones capturing RGB, depth, 6-DoF pose & gripper width

60Hz

synchronized capture rate via ARKit

MIT

open license: code, data & hardware CAD

Tracking accuracy vs. Motion Capture

The constraint-aware whole-body controller reproduces hand-eye trajectories tightly against a MoCap reference.

≤5.0mm

position error

≤0.8°

rotation error

MoCap

full-scale reference

Columns scaled to reference — the error bars are a small fraction of the MoCap baseline.

The pipeline — human → policy → robot

Human demo

Wrist cameras + head cam capture natural motion

→

3D visual rep

DINO-ViT features + 3D points, embodiment-agnostic

→

Hand-eye policy

Diffusion Policy, head relaxed to a look-at point

→

Real robot

Bimanual mobile platforms such as RB-Y1

What works

Egocentric head cam adds global context for navigation & active perception
Transfers across operators of different heights
Long-horizon, bimanual, whole-body tasks — no teleoperation, no robot

Open challenges

Egocentric sensing widens the human–robot embodiment gap
Naive full 6-DoF head regression raises error & causes instability
Newly released (July 2026) — large-scale real-world reports still limited

Continue reading

The rest of this article is for AI News Blitz readers. Choose an option below to keep reading.