Framework Survey

PraxisWorld: Object-Centric World-Action Model
for Composable Robot Manipulation

Yu-song Huang1,† Yu-yan Li1,† Hong-liang Lu2 Hao-tian Wang1 Xiao-yun Qiu1 Wen-peng Xu1 Yang-gang Sheng1 Zhao-nian Kuang1 Yi-jie Chen1 Jin-tao He1 Dong-ling Liu1 Xin-hu Zheng1,*
1Intelligent Transportation Thrust, Systems Hub, and Center of Seamless Connectivity & Connected Intelligence, Hong Kong University of Science and Technology (Guangzhou), Guangzhou 511453, China
2Department of Mechanical and Energy Engineering, Southern University of Science and Technology, Shenzhen 518055, China
†These authors contributed equally to this work.
*Corresponding author: xinhuzheng@hkust-gz.edu.cn
Paper Reference Map Source Demos BibTeX

Overview

PraxisWorld overview diagram
PraxisWorld at a glance: human behavioral organization motivates the object-centric view, the skill descriptor exposes the world interface, the four-stage lifecycle turns world state into callable, verifiable transitions, and recent world models are read as estimators of the missing transition model Δ.

PraxisWorld treats manipulation as a world state → action interface → state transition problem. Instead of reading verbs such as pick, place, and insert as complete skill definitions, it asks which objects, contact modes, tolerances, initiation conditions, effects, and recovery checks make a physical transition executable.

180+ works reviewed across skills, TAMP, affordances, VLAs, tactile manipulation, and world models
4 lifecycle stages: definition, learning, composition, and execution with recovery
K object-centric skill descriptor for callable, verifiable manipulation interfaces

Motivation

Same action verb producing different object-state transitions in robot manipulation
Same verb, different physics: PraxisWorld uses object-state transitions rather than action labels alone as the unit of analysis.

Framework

The survey formalizes an atomic skill as K = (R, S, X, C, I, π, β, Δ, E), connecting object anchors, world state, embodiment, contact constraints, initiation, policy, termination, predicted effect, and verification signals.

Atomic skill descriptor interface
The atomic-skill interface makes object state and transition effects explicit.
World-state transition under skill execution
Skills are evaluated by the object-state transition they produce, not by the verb alone.

Lifecycle

The skill descriptor becomes operational only when tied to the lifecycle of a manipulation system. PraxisWorld organizes that lifecycle into four stages — definition and discovery, learning and anchoring, composition and orchestration, and execution and recovery — treated not as separate topics but as a single pass through the same world-action model, from world state to a verified transition.

Four stages of the object-centric world-action model
A single lifecycle unifies the four stages below; the skill descriptor connects them, and the interface test Δi ⊨ Ij governs both forward chaining and recovery re-entry.
Definition and DiscoveryIdentify reusable object-state transitions.
Learning and AnchoringGround skills in semantic, metric, and tactile state.
Composition and OrchestrationMatch effects to initiation conditions.
Execution and RecoveryVerify transitions and repair boundary failures.
Composition as effect-to-initiation matching
Composition is effect-to-initiation matching: an edge is valid only when one skill's predicted effect Δi satisfies the next skill's initiation interface Ij; on mismatch, recovery re-enters the same graph from the observed abnormal state.

Current Methods

PraxisWorld rereads current systems by asking what part of the world-action loop they expose: observed object state, transition prediction, callable interface, verification, or recovery. JEPA-style and action-conditioned world models are analyzed as transition-interface estimators.

World models as transition-interface estimators
World models read as transition-interface estimators: JEPA-style and action-conditioned predictors target the weakest variable in most skill frameworks — the predicted effect Δ — while the open challenge is aligning their latent futures with verifiable object-state interfaces.

Meta-World Probe

The manuscript includes a compact Meta-World V3 diagnostic over basketball, bin-picking, and shelf-place. The probe checks whether an explicit object-state gate can detect perturbed skill-boundary transitions and support recovery.

Meta-World probe success rates
Per-task success rates for nominal replay, open-loop continuation, semantic gating, verify-only object-state gating, state-gated recovery, and oracle feedback, with Wilson 95% confidence intervals.
Meta-World probe rollout examples
Example rollout: when the object is displaced at the skill boundary, continuing the old action suffix fails, but the same expert policy recovers once the mismatch is detected and control is re-queried in closed loop.

Simulation Demos

We instantiate PraxisWorld's execution-and-recovery loop on an industrial dual-arm mobile manipulator with dexterous hands in NVIDIA Isaac Sim. The flagship warehouse demo makes the framework's core claims visible: an object-state anomaly is observed (a tipped carton arriving from upstream), the task is checkpointed, a recovery skill re-enters from the observed state, the effect is verified, and execution resumes — followed by an effect-to-initiation re-plan (Δ ⊨ I) when a fragile blocker occupies the next placement slot, and material-conditioned grasps (cardboard clamp vs. glassware cradle).

Warehouse flagship: interrupt → observed-state recovery → verified resume, an effect-to-initiation re-plan around a glass blocker, and material-adaptive grasping — with a bilingual operations HUD narrating each state transition.
Factory cell: a fragile blocker violates the place-skill's initiation condition (Δ ⊨ I); the robot stages its payload, relocates the blocker with a two-palm cradle, then restores and completes the original task.

Reference Map

Explore the survey's 191 cited works as an interactive, section-organized knowledge base — switch between a force-directed graph and a collapsible tree, search by title or author, filter by section, and open any reference for its details and BibTeX.

Open the full Reference Map in a new tab ↗

Citation

@misc{huang2026praxisworld,
  title={PraxisWorld: Object-Centric World-Action Model for Composable Robot Manipulation},
  author={Huang, Yu-song and Li, Yu-yan and Lu, Hong-liang and Wang, Hao-tian and Qiu, Xiao-yun and Xu, Wen-peng and Sheng, Yang-gang and Kuang, Zhao-nian and Chen, Yi-jie and He, Jin-tao and Liu, Dong-ling and Zheng, Xin-hu},
  year={2026},
  note={Survey manuscript}
}