OneWM：统一因果训练下世界建模与动作建模能否互惠？

世界建模与动作建模密切相关，但它们在统一因果训练下的互惠收益仍不明确。我们提出 OneWM，这是一个用于统一因果训练的简单框架，它在共享的视频-动作架构中将动作同时视为世界预测的条件和策略学习的目标。利用该框架，我们研究了以指定动作为条件学习前向动力学是否能改善动作学习，以及学习生成动作反过来是否能改善以动作为条件的世界建模。我们使用 LIBERO、LIBERO-Plus 和 RoboCasa365 作为具身测试平台，评估策略学习和世界建模，并辅以真实世界策略实验。学习指定动作的视觉后果可提升策略性能以及在变化场景中的泛化能力。反过来，在同一架构下，动作学习提升了多视图世界预测的保真度。总之，这些结果对两个问题都给出了肯定回答，支持在统一因果训练下世界建模与动作生成之间的互惠收益。

World and action modeling are closely related, yet their reciprocal benefits under unified causal training remain unclear. We introduce OneWM, a simple framework for unified causal training that treats actions as both conditions for world prediction and targets for policy learning within a shared video-action architecture. Using this framework, we investigate whether learning forward dynamics conditioned on specified actions improves action learning, and whether learning to generate actions in turn improves action-conditioned world modeling. We use LIBERO, LIBERO-Plus, and RoboCasa365 as embodied testbeds to evaluate both policy learning and world modeling, complemented by real-world policy experiments. Learning the visual consequences of specified actions improves policy performance and generalization across changing scenes. In turn, action learning improves multi-view world-prediction fidelity under the same architecture. Together, these results answer both questions positively, supporting reciprocal benefits between world modeling and action generation under unified causal training.

注意转载授权 · 阅读原文。中文正文为 AI 翻译，请以原文为准。

注意转载授权 · 阅读原文