- Researchers presented Embodied.cpp, a C++ inference runtime designed to enable efficient deployment of vision-language-action (VLA) and world-action models (WAMs) on heterogeneous edge devices and robots.
- The runtime organizes execution into five modular layers (input adapters, sequence builders, backbone execution, head plugins, deployment adapters) to solve fragmentation across model-specific Python stacks, supporting multi-rate execution and latency-optimized batch-1 inference.
- The infrastructure addresses closed-loop control requirements with real-time responsiveness for embodied AI deployment, a pattern critical to autonomous vehicle perception-action loops.
- 研究人员提出了Embodied.cpp，这是一个C++推理运行时，用于支持视觉-语言-动作（VLA）和世界-动作模型（WAM）在异构边缘设备和机器人上的高效部署。
- 该运行时将执行过程组织为五个模块化层（输入适配器、序列构建器、骨干执行、头部插件、部署适配器），以解决现有模型特定Python栈的碎片化问题，支持多速率执行和延迟优先的批量1推理。
- 该基础设施解决了嵌入式AI模型对闭环控制和实时响应性的需求，这是自动驾驶感知-动作循环的关键需求。
- Researchers introduced the first multiplayer world model for highly dynamic environments governed by complex physical interactions, trained on 10,000 hours of Rocket League gameplay with publicly available bots.
- The 5-billion-parameter latent diffusion model generates four-player matches in real time at 20 frames per second, learning to attribute scene changes to the correct agent while maintaining coherence under arbitrary action combinations.
- Unlike single-player world models that treat other agents as environmental noise, this system conditions on multiple agents' action streams to predict their interdependent effects in real time.
- 研究者发布首个多智能体世界模型，用于处理高度动态环境中的复杂物理交互，基于10,000小时火箭联盟公开机器人对战数据训练。
- 该50亿参数隐空间扩散模型能实时生成四人对战场景，帧率达20fps，学会准确将场景变化归因于对应智能体，在任意动作组合下保持一致性。
- 与将其他智能体视为环境噪声的单智能体模型不同，该系统直接以多个智能体动作流为条件，实时预测其相互作用效应。
- Across robotics, World Models (WMs) are increasingly used to evaluate action policies by simulating the consequences of actions in an imagined world and returning a success or safety verdict.
- The authors argue a verdict is only as trustworthy as the WM that produced it, meaning the WM itself must first be validated, or "certified," before its judgment can be relied upon.
- They flag that today's video-generation WMs are graded on fidelity metrics such as Fréchet Video Distance (FVD), which reward visual realism but do not check whether the simulated world responds correctly to the policy's actions—including actions unseen during training—a gap directly relevant to any autonomous-driving pipeline that leans on WM-based simulation for policy validation.
- 在机器人领域，世界模型（World Models, WM）正被越来越多地用于评估动作策略——通过在想象世界中模拟动作后果，输出成功或安全与否的判定结果。
- 论文作者指出，判定结果的可信度取决于产生它的世界模型本身，因此世界模型必须先经过验证（即“认证”），其判断才能被采信。
- 他们指出，当前的视频生成类世界模型主要以Fréchet Video Distance（FVD）等保真度指标来评判，这类指标只衡量画面逼真度，却未检验模拟世界是否对策略的动作做出了正确响应（尤其是训练中未见过的动作）——这一缺口对任何依赖世界模型仿真来验证策略的自动驾驶测试流程都直接相关。
- AlayaWorld is an open-source full-stack framework for building interactive generative worlds using video world models, replacing traditional labor-intensive game development pipelines.
- The framework enables real-time interaction through autoregressive synthesis conditioned on world state and user actions, trained on both gameplay recordings and real-world videos to capture diverse visual and physical dynamics.
- The technology extends beyond gaming to embodied intelligence applications and other interactive domains.
- AlayaWorld是一个开源全栈框架，使用视频世界模型构建交互式生成世界，替代传统的劳动密集型游戏开发流程。
- 该框架通过自回归合成实现实时交互，以世界状态和用户行为为条件，采用游戏录像和真实世界视频的混合训练来捕捉多样的视觉和物理动态。
- 该技术超越游戏应用，扩展到具身智能及其他交互应用领域。
- ACID is a decision-time planning framework for embodied control that uses action-conditioned world models to search over candidate action sequences.
- It adds an action consistency cost via inverse dynamics that verifies predicted trajectories are actually realizable in the environment, not just goal-optimized.
- Standard planning judges candidates only by terminal state proximity to goal; ACID additionally checks whether intermediate transitions are executable by inferring actions that would explain each predicted transition.
- ACID是用于具身控制的决策时规划框架，采用行动条件化世界模型搜索候选动作序列。
- 它通过逆向动力学添加行动一致性成本，以验证预测轨迹在环境中的实际可实现性，而非仅优化目标。
- 标准规划仅根据终端状态与目标的接近度评判，ACID额外检查中间转移的可执行性，通过推断能解释每个预测转移的动作实现。
- Researchers introduce GigaWorld-1, a systematic study and benchmark (WMBench) using world models to evaluate embodied robot policies, eliminating the need for expensive real-world rollouts.
- WMBench uses real-robot teleoperation data across diverse manipulation tasks, analyzing 7 video world models and 4 action representation schemes to understand which properties make world models reliable for policy assessment.
- World models address a critical bottleneck in evaluating embodied foundation models—unlike LLMs assessed via digital benchmarks, robot policies traditionally require slow, costly real-world testing constrained by hardware and human supervision.
- 研究团队推出GigaWorld-1，这是一项系统性研究和基准测试（WMBench），使用世界模型评估具身机器人策略，消除对昂贵真实世界测试的需求。
- WMBench使用真实机器人遥操作数据，涵盖多样化操纵任务，分析7个视频世界模型和4种动作表示方案，以理解哪些特性使世界模型在策略评估中可靠。
- 世界模型解决了评估具身基础模型的关键瓶颈——与通过数字基准测试评估大语言模型不同，机器人策略传统上需要缓慢、成本高的真实世界测试，受硬件和人类监督限制。
- NAVSIM v2.1.2 was released on April 28, 2025 (commit 937cefc, signed by maintainer DanielDauner), adding the new navhard_two_stage dataset split to the autonomousvision/navsim autonomous-driving simulation benchmark.
- The release also updates the Extended Predictive Driver Model Score (EPDMS) used for the Hugging Face Warmup leaderboard, with implementation details documented on the project's metrics page.
- As an open-source closed-loop benchmark for scoring autonomous-driving planning models, NAVSIM's refreshed dataset and EPDMS metric give researchers and OEM teams a more standardized way to compare driving-policy performance.
- NAVSIM v2.1.2于2025年4月28日发布（提交937cefc，由维护者DanielDauner签名发布），为autonomousvision/navsim自动驾驶仿真基准新增了navhard_two_stage数据集分割。
- 本次更新还升级了用于Hugging Face Warmup排行榜的扩展预测驾驶员模型评分（EPDMS）指标，具体实现细节已在项目metrics页面中说明。
- 作为一个开源闭环基准测试工具，NAVSIM用于评估自动驾驶规划模型，其数据集和EPDMS指标的更新为研究人员及整车厂团队提供了更标准化的驾驶策略性能比较方式。
- DREAMSTEER is a deployment-time steering framework for pretrained vision-language-action policies that achieves robustness without requiring finetuning or parameter modifications.
- The system leverages latent world models and language-conditioned value models to evaluate candidate actions, addressing the distribution shift problem between training and deployment environments.
- Tested on four real-world manipulation benchmarks with unseen objects, DREAMSTEER demonstrates significant improvements in task success rates.
- DREAMSTEER是一个部署时期的视觉-语言-动作策略框架，在无需微调或参数修改的条件下实现鲁棒性。
- 该系统利用潜在世界模型和语言条件的价值模型评估候选动作，解决训练与部署环境间的分布偏移问题。
- 在四个包含未见物体的真实操作基准测试中，DREAMSTEER展示了任务成功率的显著改进。
- Researchers distinguish between "functional reasoning" (improves task performance) and "faithful reasoning" (truly reflects internal decision logic) in Vision-Language-Action models used for autonomous driving.
- Their analysis reveals that current state-of-the-art alignment strategies admit reasoning that masks causal links through confounding factors and lacks environmental grounding, potentially restricting generalization.
- Human evaluation of a leading autonomous driving reasoning model shows inconsistent coupling between reasoning quality and task performance, suggesting interpretability gaps in current VLA systems.
- 研究人员区分了"功能性推理"（改进任务表现）和"真实性推理"（真实反映内部决策逻辑）在自动驾驶视觉语言动作模型中的差异。
- 分析表明当前最先进的对齐策略存在不足——推理可能通过混淆因素掩盖因果关系，且缺乏环境基础，可能限制模型泛化能力。
- 对领先自动驾驶推理模型的人工评估显示推理质量与任务表现之间存在不一致耦合，说明当前VLA系统的可解释性存在缺口。
- Researchers propose DynaVieW, a schema-guided world model optimized for predicting and simulating temporal visual scene evolution in videos and multi-image sequences.
- The model learns interleaved state-transition sequences where states represent visual scenes from keyframes and transitions capture hierarchical dynamic constituents through a mixture-of-experts architecture.
- DynaVieW jointly optimizes transition prediction and state simulation using cross-expert selective attention and schema token re-weighted loss to enable robust visual dynamics understanding.
- 研究人员提出DynaVieW，一个基于模式引导的世界模型，用于预测和模拟视频或多图像序列中的视觉场景时间演变。
- 该模型学习交错的状态转移序列，其中状态来自视频关键帧，转移在分层模式内捕捉动态成分，采用混合专家架构。
- DynaVieW通过跨专家选择注意和模式令牌重权损失联合优化转移预测和状态模拟，实现鲁棒的视觉动态理解。
- Researchers propose Factorized Dense Routing (FDR-Occ) for vision-based 3D occupancy prediction, addressing the Locality Bottleneck of current methods that use explicit physical projection with sparse camera rays.
- FDR abstracts view transformation as unconstrained bipartite routing using hierarchical tensor contractions, achieving fully-global receptive field with sub-quadratic complexity while maintaining robustness when camera extrinsics are unreliable or absent.
- The method introduces a Resolution-Context Decoupled Architecture to balance fundamental trade-offs between spatial resolution and contextual understanding in 3D space representation.
- 研究人员提出因子化密集路由（FDR-Occ）方法用于视觉3D占用预测，解决当前使用显式物理投影和稀疏摄像头射线方法的局部性瓶颈问题。
- FDR将视角变换抽象为无约束二分图路由，通过分层张量收缩实现完全全局感受野与次二次方计算复杂度，在摄像头外参不可靠或缺失时保持鲁棒性。
- 该方法引入分辨率-上下文解耦架构，平衡3D空间表示中空间分辨率与上下文理解之间的根本权衡。
- LingBot-VLA 2.0 advances the Vision-Language-Action model to improve performance in real-world applications beyond laboratory conditions.
- The model was trained on approximately 60,000 hours of data with a revamped processing pipeline and enhances generalization across tasks and different embodiments.
- The work focuses on bridging the gap between foundation models and practical implementation for embodied robotic systems.
- LingBot-VLA 2.0是对前一代模型的改进，旨在提升视觉-语言-动作模型在实际应用中的表现，超越实验室条件的限制。
- 该模型使用约6万小时的数据进行预训练（包括5万+小时），采用改进的数据处理管道，增强了跨任务和不同机体的泛化能力。
- 这项工作致力于缩小基础模型与实际部署之间的差距，推进具身化人工智能系统的实际应用。
- Researchers trained JEPA-based self-supervised world models on nuPlan data from Pittsburgh, Boston, and Singapore, then evaluated zero-shot generalization on held-out Argoverse 2 scenarios from Miami and Austin.
- Geographically diverse training reduced mean surprise score by 16.5% versus single-geography training at equal scale (0.228 vs 0.273 on 63,000 scenarios), demonstrating that diversity significantly improves cross-domain generalization.
- Training on 200,000 scenarios from a single geography (3x more data) still produced higher surprise (0.264) than the geographically mixed 63K model, showing data volume alone cannot compensate for geographic diversity.
- 研究人员基于nuPlan数据（匹兹堡、波士顿、新加坡）训练JEPA自监督世界模型，在Argoverse 2数据集的迈阿密和奥斯汀场景上进行零样本评估。
- 在63,000场景的对等规模下，地理多样性训练将平均惊讶分数降低16.5%（0.228对0.273），表明多样性显著改善跨域泛化能力。
- 即使用单一地区200,000场景（3倍数据量）训练，其惊讶分数（0.264）仍高于地理混合的63K模型，说明数据量无法弥补地理多样性的不足。
- Researchers propose HiMe, a hierarchical embodied memory framework that addresses the "frequency-competence paradox" in Vision-Language-Action (VLA) models, which currently struggle with long-horizon tasks requiring memory and reasoning beyond immediate observations.
- The framework decouples embodied intelligence into three components: a high-frequency Executor for real-time control, a Sentry for working memory, and a Planner for long-term strategy, balancing execution speed with reasoning capability.
- The system introduces dynamic knowledge management with Add, Update, and Delete operations for memory plasticity, demonstrating improved success rates in long-horizon robotic tasks during experiments.
- 研究人员提出HiMe框架，一个分层具身记忆系统，用于解决视觉语言行动（VLA）模型中的"频率-能力悖论"，该悖论使模型在需要超越即时观察的长视距任务中表现困难。
- 该框架将具身智能解耦为三个组件：高频执行器用于实时控制、哨兵模块用于工作记忆、规划器用于长期策略，平衡执行速度与推理能力之间的冲突。
- 该系统引入动态知识管理机制，包括添加、更新和删除操作以维持记忆可塑性，在实验中显示长视距机器人任务的成功率提升。
- Image2Sim is a real-time neural simulation framework that builds interactive embodied navigation environments from RGB-D image sequences, addressing scalability constraints and sim-to-real transfer gaps in training environments.
- The framework enables agents to interpret multimodal goals, reason in 3D space, and navigate to destinations by combining visual realism from real-world scanned data with the scalability of synthetic simulation.
- The complete code and implementation instructions have been released on GitHub.
- Image2Sim是一个实时神经模拟框架,从RGB-D图像序列生成交互式具身导航环境,解决了可扩展性和仿真-真实差距的关键问题。
- 该框架使代理能够理解多模态目标、进行3D空间推理、导航至目标位置,同时整合真实数据的视觉保真度与合成模拟的可扩展性。
- 完整代码和实现说明已在GitHub上开源发布。
- Researchers introduce Optimal Transport Q-Learning (OTQL), a new RL post-training method for fine-tuning and accelerating flow-based policies that accurately capture multimodal robot trajectory distributions in VLA models.
- OTQL achieves policy optimization with an interaction budget of only 50-60 episodes while avoiding computationally expensive distillation in both simulation and real-world robotic tasks.
- The method uses advantage-weighted conditional optimal transport flow matching to address performance gaps caused by suboptimal demonstrations and distribution shifts.
- 研究者提出最优传输Q学习(OTQL)方法，通过强化学习后训练来微调和加速能准确捕捉VLA模型中多模态机器人轨迹分布的流政策。
- OTQL仅需50-60次交互预算即可实现策略优化，规避了模拟和实际机器人任务中计算昂贵的蒸馏过程。
- 该方法采用优势加权条件最优传输流匹配技术，解决由次优演示数据和分布偏移导致的性能问题。
- PointDiT, a new pixel-space Diffusion Transformer, was introduced for monocular 3D geometry estimation and single-image 3D reconstruction, operating directly on raw 3D point map patches conditioned on image tokens.
- The method uses a minimalist architecture based on plain ViT that eliminates the need for complex hybrid architectures and intricate loss formulations required by existing state-of-the-art methods.
- The paper demonstrates that direct pixel-space operation on 3D geometry without architectural overhead and complex loss functions achieves competitive or superior performance compared to prior latent-space diffusion approaches.
- PointDiT是一种新型像素空间扩散变换器，用于单目3D几何估计和单图像3D重建，直接在原始3D点图像补丁上运行。
- 该方法采用基于plain ViT的最小化架构，避免了现有先进方法所需的复杂混合架构和复杂损失函数。
- 论文表明，在不增加架构复杂度和损失函数设计的情况下进行像素空间3D几何处理，相比先前的隐空间扩散方法性能具有竞争力。
- Researchers audited how frozen vision-language-action models handle visual history using layer-resolved linear probing and causal interventions across three VLAs from two architecture families.
- The study reveals that despite encoding past-frame content throughout the network, history-specific information is nearly absent, with stored history functioning as a largely redundant copy of the present frame.
- History becomes causally important only under heavy frame degradation, and different model architectures deploy history differently—one increasingly relies on history as a fallback under occlusion.
- 研究人员通过层级线性探针和因果干预方法，对来自两个架构家族的三个冻结视觉语言行动模型处理视觉历史的方式进行了审计。
- 研究发现，尽管网络始终对过去帧内容进行编码，但历史特有的信息几乎不存在，存储的历史基本上是当前帧的冗余复制。
- 历史信息仅在当前帧严重降质时才成为因果关键，不同架构的历史部署策略存在差异，其中一个在遮挡下作为备选方案增加对历史的依赖。
- Researchers propose SEAM (Smooth Execution of Action-Chunked Motion), a training-free inference-time method to fix multimodal bifurcation in Vision-Language-Action (VLA) policies where independently generated action chunks produce abrupt discontinuities at boundaries.
- Using Velocity-guided Loss Steering (VLS), SEAM derives consistent trajectory targets from previously executed chunks and applies closed-form corrections after each denoising step, reducing boundary jerk by 28% on LIBERO-10 benchmark with pi_0.5 model.
- The method achieves smoother execution without backpropagation through the policy network, rejection sampling, or retraining, enabling efficient real-time robotic control.
- 研究人员提出SEAM（行动分块运动平滑执行）方法，一种推理阶段的训练无关方法，用于修复视觉语言行动（VLA）策略中的多模态分叉问题，即独立生成的行动块在边界处造成的突然不连续。
- SEAM采用速度引导损失转向（VLS）技术从已执行块导出一致的轨迹目标，在每个欧拉步后应用闭式修正，使LIBERO-10基准测试中pi_0.5模型的边界抖动降低28%。
- 该方法无需通过策略网络反向传播、拒绝采样或重新训练即可实现平滑执行，为实时机器人控制提供计算效率。
Research & IP研究与专利
AIAIsingle source单源
- Researchers propose a structured demonstration collection strategy for Vision-Language-Action (VLA) models on a dual-arm robotic platform, addressing how demonstrations are organized for imitation learning.
- The approach organizes data using three principles: decomposing complex tasks into progressively learnable sub-skills, standardizing interaction environments to reduce variability, and ordering demonstrations by increasing task complexity.
- This structured design improves policy learning efficiency, training stability, and generalization by enabling VLA models to acquire fundamental manipulation skills before learning complex behaviors.
- 研究人员提出了一种结构化演示收集策略，应用于双臂机器人平台上的视觉语言行动（VLA）模型，重点解决演示数据的组织方式对模仿学习的影响。
- 该方法通过三个原则组织数据：将复杂任务分解为可渐进学习的子技能、标准化交互环境以减少不必要的变异、按照任务复杂度递增排列演示。
- 这种结构化设计使VLA模型能先掌握基础操作技能再学习复杂行为，从而提升策略学习效率、训练稳定性和泛化能力。
Read more · 3 more in Research & IP →阅读更多 · 研究与专利还有 3 条 →
