DreamDojo uses human video to train a robot world model
DreamDojo gives robot models a broader training base: 44k hours of egocentric human video, followed by a smaller round of robot-data training, are used to predict how actions change the world. The paper says its model supports live teleoperation, policy evaluation, and model-based planning, with a distilled version running at 10.81 FPS. It evaluates the approach on multiple out-of-distribution benchmarks for open-world, contact-rich tasks.