Envisioning
Agentic Robotics

A New Era of Robotic Systems

Place the cup on the marker.

Paradigm Shift

Foundation models offer a complementary path to robot generalization alongside advances in action-policy training. Their broad knowledge and reasoning can help robots interpret unfamiliar goals, organize available capabilities, and adapt through feedback. Agentic robotics connects these decisions to physical action, outcome verification, and adaptation around user intent.

From Policy-Centric Robotics to Agentic Robotics

Policy-centric robotics organizes execution around an action policy. Given a task and observations, the policy produces action targets that the robot’s controller carries out, then uses feedback to adjust its next actions. The policy may also reason or predict subtasks; its central role is to turn task inputs into physical behavior.

Agentic robotics organizes work around an agent. It interprets what the user wants, chooses and coordinates skills and tools, checks whether their effects satisfy the goal, and revises its decisions when they do not. This connects deciding what to do with checking what actually happened.

Figure 1. Two Paradigms. Both paradigms use feedback. Agentic robotics centers on an agent that coordinates skills and tools, verifies outcomes, and adapts its decisions.

Three Typical Paths to Execution

A decision can reach the body in several ways. Here, S2 denotes the role that decides and verifies, S1 supplies skills and callable modules, and S0 provides the body interface and local control. Their connections give three common execution paths, which can coexist within one task.

Figure 2. Three Typical Execution Paths. The paths can coexist within one task. Direct control still leaves the local control loop in S0.

Layered Orchestration. S2 invokes skills and callable modules in S1, including action, perception, and motion-planning modules. Action skills supply control targets for S0 to execute.

Direct Control. S2 sends permitted control targets directly to S0 through an action interface. S0 still runs its local real-time control loop.

Code as Policy. S2 writes a program that takes the role of S1 at run time. The program can execute several actions before S2 checks the outcome and decides what to do next.

Reference Framework

These execution paths share a need to connect intent, action, and evidence. Our reference framework makes the responsibilities and interfaces explicit: S2 decides and verifies, S1 supplies skills and callable modules, and S0 maintains local body control. The decomposition is functional: one model may serve several roles, and one role may span several components.

Figure 3. A Reference Framework for Agentic Robotics. S2 decides and verifies; S1 supplies skills and tools; S0 maintains body control. The event monitor remains responsive during deliberation.

From Intent to Verified Outcomes

Execution returns evidence to S2. Bringing a cup to the table requires checking that it was grasped, remains held during transport, and reaches the intended place before reporting completion. The loop below connects intent, decisions, tool calls, execution, and verification; its timeline shows S0 continuing while S2 deliberates or replans.

Figure 4. The Agentic Loop. Acceptance, termination, physical effect, and intent satisfaction are four different judgments. A completed call alone does not prove task success.

Embodied System 2

The framework shows how decisions reach the body and how evidence returns. We now examine the part responsible for using that evidence: Embodied S2, realized by a foundation model and an embodied harness.

Foundation Model and Embodied Harness

The foundation model supplies knowledge, understanding, and reasoning. The embodied harness makes these capabilities usable in a robot: it assembles the state on which decisions depend and connects model outputs to execution that can be observed and checked. Stronger reasoning still needs current observations and well-defined interfaces.

Before a decision, the context manager and memory combine current observations with task progress and relevant history. The planner maintains a plan that can change after each result; the tool interface exposes skills with explicit inputs, limits, and execution states; the verifier checks their physical effects against the goal. The cup task is complete only when evidence shows that the cup is at the intended place.

The eight abilities below connect this implementation to evaluation. Knowledge, understanding, and reasoning reside mainly in the model; tool use, memory, cross-embodiment, safety, and efficiency depend on the model and harness together.

Figure 5. Core Abilities of Embodied S2. Knowledge, understanding, and reasoning reside mainly in the model; the other five abilities arise from the model and harness together.

Whether it succeeds also depends on when it acts and what has changed in the meantime. Physical constraints therefore shape the decision process, and evaluation must test both the model’s abilities and the resulting behavior.

Physical Constraints

Many modern foundation models improve capability by using more compute, often at the cost of longer deliberation. We call this trend monotonic scaling. In the physical world, reaction deadlines and other constraints demand a different objective: greater capability within fixed limits, or capped scaling. This calls for both better models and a redesign of the surrounding system, including how sensing, reasoning, tools, and control work together. In Figure 6, useful progress increases capability while keeping reaction time below the physical deadline.

Figure 6. Capped Scaling. Decision quality under a deadline. These are conceptual curves, not measured results.

The layers therefore work at different timescales. S0 maintains fast body control; S1 control skills update actions; S2 decides when the task needs a new plan or judgment. Figure 7 places them on a logarithmic time axis. Its S0/S1 bands are indicative, and the S2 band is a target for dynamic manipulation. Perception and planning tools run on demand; the reported model latencies come from different tasks.

Figure 7. Decision and Control Timescales. Indicative control timescales and reported S2 latencies. The S2 band is a target; reported points come from different task settings.

Many current robotics benchmarks pause simulated time during model inference. This can mask the physical cost of delayed decisions; evaluation for real-world deployment should also keep the world running during inference. In the paired scenes below, both robots have the same reasoning delay. Pausing keeps the cup at its observed position; in the live world, the moving belt makes that position stale before the grasp.

The world does not wait for reasoning

Same Instruction: Pick up the cup.

Paused world
Live world

Evaluation

These constraints make evaluation a question of both capability and execution. Model-level tests isolate abilities under specified inputs; system-level tests measure what the model, harness, skills, and body achieve together. The framework covers eight abilities and the resources consumed as cost. Report them as a profile: these overlapping dimensions should not be summed into one score.

9Dimensions
53Sub-Dimensions
29Embodiment-Related
Figure 8. S2 Evaluation at a Glance. Eight abilities plus cost. Highlighted embodiment-related sub-dimensions are specific to, or strongly dependent on, embodiment.

Highlighted entries are embodiment-related.

    Control What Changes

    Fix the harness when comparing models; fix the model when comparing harnesses.

    Keep the World Running

    Report deadlines, observation age, and paused versus live execution separately.

    Verify Independently

    Use evidence-based outcome checks and disclose false completion claims.

    Evaluation Conditions and Outcomes

    The studies reviewed in the paper report gains in semantic and spatial capabilities, while physical execution, completion judging, and timely response remain uneven. These comparisons show how interfaces, the world clock, and the move from text to action affect reported performance.

    Selected comparisons from the paper’s results snapshot, with conditions checked against the original studies. Each row uses its own task, metric, and protocol.

    Selected comparisons from the paper’s results snapshot, with conditions checked against the original studies. Each row uses its own task, metric, and protocol.
    Study / MetricReported ComparisonSettingWhat It Shows
    EmbodiedSkillBenchSuccess Rate92% / 61% / 85%Native / Skill-Only / Selective Reuse (ADR)GPT-6 Astra; 100 perturbed LIBERO-PRO tasks.The tool interface can constrain performance.
    RT-SafeSafe Success Rate (SafeSR)19.8% → 0.7%World Paused During Inference → Real-TimeEight VLMs; 36 matched simulation routes under hard conditions; default reasoning; mode-specific timing prompts.Safe completion is lower in the real-time condition.
    RoboHarmRefusal Rate72% → 18%Text → Robot ActionsSix models; real-robot end-effector control.Refusal in text transfers only partly to action.

    These rows use different tasks and metrics and do not form a cross-benchmark ranking. Refusal rate is distinct from safe task completion.

    From Digital Agents to Physical Agents

    The evaluation gaps also expose assumptions that change when an agent moves from software into the physical world. The observe–act–verify loop still applies, but the conditions that make it reliable must be rebuilt.

    Conditions for Agentic Work

    Coding agents benefit from familiar interfaces, verifiable outcomes, abundant data, and inexpensive environments. A robot instead has to infer state from sensors, check physical effects, learn from costly trials, and recover from actions that may be difficult to undo.

    Figure 9. The Four Enabling Conditions of Digital Agents. Different task settings offer different conditions for agentic work; they are not a single ladder of difficulty.

    What Transfers, What Must Be Rebuilt

    The organizing principles still transfer: act on feedback, define tool contracts, maintain durable state, separate execution from judgment, and reuse validated skills. For robots, those principles require interfaces with explicit units and limits, verifiers checked against physical outcomes, embodied data, and simulation validated against reality.

    Figure 10. What Physical Agents Must Rebuild. Grounded interfaces, calibrated verifiers, embodied data, and reality-checked simulation must work under physical deadlines and limits.

    Road Ahead

    Rebuilding these foundations should expand the work people can entrust to a robot. Progress therefore needs to track the responsibility a robot can reliably carry, as well as the environments in which it can work. This leads from individual primitives and tasks toward coordinated workflows and sustained roles.

    From a Task to a Standing Role

    Consider breakfast. Picking up a cup is one skill; making and serving coffee is a complete task. Preparing breakfast for four adds timing and coordination across tasks. Taking care of breakfast every day adds a continuing responsibility: remembering preferences, keeping supplies ready, and adapting as needs change. The scene below illustrates this ambition; it is a vision of the work we want to delegate.

    Figure 11. A Vision for Agentic Robotics. An envisioned breakfast scenario, from picking up a cup to taking responsibility for breakfast day after day.

    Ladder of Delegable Work

    The ladder makes this progression explicit: a primitive, a task, a workflow, and a standing role. Each step adds responsibility for how work is organized and sustained.

    L1 · Primitive: “Pick up the cup.” Execute one local skill and verify its result. Responsibility stays with that single operation.

    L2 · Task: “Make and serve coffee.” Coordinate skills, monitor progress, and recover from failures until the complete task meets its acceptance criteria.

    L3 · Workflow: “Prepare breakfast for four.” Coordinate interdependent tasks, timing, priorities, and shared resources. Responsibility covers the whole workflow, not each task in isolation.

    L4 · Role: “Take care of breakfast each day.” Sustain an agreed responsibility across tasks and workflows until it expires or is withdrawn. Persistent state maintains continuity between completed workflows.

    Figure 12. Ladder of Delegable Work. Primitives, tasks, workflows, and standing roles. The durations illustrate the progression rather than define it.

    In unstructured settings, evidence of unaided work is strongest at the task level. Some company demonstrations extend toward workflows, but their reports do not yet establish all the conditions for reliable workflow-level autonomy. Sustained roles remain a target for development and evaluation.

    Expanding the Operating Scope

    A workcell calls for grounding and precise actions within reach and view. A room adds base positioning and changing reachability; a building adds maps, memory, and search; a site adds long routes, coordination with other people and robots, and handovers between areas. Spatial scope complements the responsibility a robot carries.

    Figure 13. Spatial Scope of Delegated Work. Spatial scope is a separate axis from responsibility: workcell, room, building, and site.

    What Do Today’s Benchmarks Cover?

    These ten representative benchmarks span workcell manipulation, household tasks, and navigation. Spatial scope and time span describe the setting; a larger scene or longer episode does not by itself establish workflow or role-level responsibility.

    Ten representative benchmarks from Table 9 of our paper, with an additional check of inference-time behavior in standard evaluators.

    Ten representative benchmarks from Table 9 of our paper, with an additional check of inference-time behavior in standard evaluators.
    Benchmark Spatial Scope Time Span World Time During Inference
    LIBERO Workcell Single Episodes Paused
    CALVIN Workcell Single Episodes (5-Instruction Chains) Paused
    RoboDojo Workcell Single Episodes (8–76 s Commanded Motion) Sim: paused
    Real world: live
    RoboTwin 2.0 Workcell (Dual-Arm) Single Episodes Paused
    RoboLab Workcell Single Episodes Paused
    VLABench Workcell Single Episodes Paused
    ALFRED Room (Household) Single Episodes Strict pause unverified
    RoboCasa365 Room (Kitchens) Single Episodes Paused (OpenPI)
    R2R-CE Building Single Episodes Paused
    BEHAVIOR-1K Building (Homes, Offices) Single Long Episodes Paused

    None of the benchmarks reviewed in the paper tests a workflow of interdependent tasks that share resources and deadlines, or a role sustained over days. A longer episode or task sequence does not by itself establish either.

    Future Research Directions

    Reason Within Physical Deadlines

    Decide when to think, observe again, act, or reuse a skill within the response budget.

    Keep Control Persistent and Verifiable

    Maintain trustworthy state across tasks and spaces, and coordinate work with shared resources.

    Adapt Across Environments and Bodies

    Reuse verified experience, expose capabilities explicitly, and reduce data and configuration needs.

    Improve from Physical Experience

    Retain changes that help later work without eroding existing abilities.

    Progress in these directions should carry forward to later work. Test improvements on held-out tasks, check that existing abilities are retained, and identify what changed in the model, harness, or skills.

    Applications

    Imagine giving a robot an outcome, then leaving it to organize the work: breakfast served, shelves replenished, a facility inspected. Embodied S2 could make validated robot skills usable across a growing portfolio of jobs, allowing a deployment to become more valuable as it takes on more recurring work.

    Homes

    Daily household service. Prepare meals, reset rooms, and replenish essentials around a household’s routines and preferences. One robot could earn its place through several useful jobs every day.

    Manufacturing & Logistics

    Adaptable material flow. Coordinate picking, kitting, replenishment, and delivery; handle a missing part or an order change without restarting the whole workflow. Reusable skills could make the next production change cheaper to support.

    Inspection

    Inspection that leads to action. Cover a site, investigate unusual readings, and return with checked findings and evidence. Reliable rounds could extend coverage while reducing the time operators spend chasing uncertain alarms.

    Healthcare & Laboratories

    More time for care and discovery. Coordinate supply delivery, sample transport, and workstation preparation with explicit safety checks and traceable handovers. These support services could return staff time to patients and experiments.

    Agriculture & Mobility

    More jobs across a larger site. Move supplies, inspect crops or equipment, and adapt routes as conditions change. Sharing navigation and handling skills across assignments could keep a robot productive through more of the day.

    The commercial opportunity grows with the work a deployment can reliably absorb. More completed jobs per robot, less setup for the next workflow, and fewer human interventions are the measures that turn broader capability into a stronger business case.

    Conclusion

    Four Takeaways

    1. Foundation models make S2 a promising starting point for robot generalization.

      Their broad knowledge and reasoning can help robots understand unfamiliar goals and situations, organize available capabilities, and adapt through feedback, alongside advances in action-policy training.

    2. Physical autonomy requires the model and embodied harness to work together.

      They must support effective, verifiable action under constraints of time, embodiment, uncertainty, and safety. These constraints must shape reasoning, execution, and adaptation.

    3. Evaluation must distinguish model capabilities from system performance.

      Assess both what the model can do and how its capabilities contribute to reliable outcomes in the complete closed-loop system.

    4. The long-term goal is sustained autonomy with broader responsibility.

      Progress extends from individual primitives and tasks to coordinated workflows and standing roles, measured by the breadth and duration of work people can reliably entrust to robots.

    Three Priorities

    1. Develop reliable evaluation frameworks for broader responsibilities and sustained operation.

      Complement simulation with real-robot testing and cover higher levels of responsibility and broader spatial scopes. Assess sustained performance under changing conditions, including recovery from failure.

    2. Develop embodied-native reasoning that integrates perception, reasoning, and action.

      Improve decisions within the time available for action, rather than relying on longer deliberation alone. Work toward reasoning that integrates perception, reasoning, and action in response to the dynamics and urgency of physical tasks.

    3. Standardize reporting for reproducible comparisons and clear attribution of results.

      Make the model, harness, available capabilities, evaluation conditions, inference budgets, response times, and human assistance explicit. Enable reproducible comparisons and distinguish model capabilities from the support provided by the surrounding system.

    Join Us

    Build intelligence that works in the physical world.

    Turning advances in AI into capable robots takes both fundamental research and thoughtful engineering. We welcome researchers and engineers who ask ambitious questions, test ideas rigorously, and care about making systems work in the real world. If you want to help shape the future of agentic robotics, we’d like to hear from you.

    Send a brief introduction and your CV, portfolio, or selected work to:

    Citation

    If you find this project helpful, please consider citing our paper.

    @article{grid2026agentic,
      title = {Envisioning Agentic Robotics: A New Era of Robotic Systems},
      author = {Qingfeng Lan and Duo Li and Siyao Li and Yiheng Lyu and
                Shijian Lu and Zhenghao Qi and Shizun Wang and Xinchao Wang and
                Ming Yang and Zhiqin Yang and Zuhao Yang and Tao Yu},
      journal = {arXiv preprint arXiv:2611.18359},
      year = {2026}
    }