The Next Step for Embodied AI: Deploy First, Evolve as You Go

After AgiBot's LWD, the real world is no longer just a testing ground for robots — it's becoming a training ground.

👦🏻 Author: Jingshan

🥷 Editor: Koji

🧑‍🎨 Layout: NCon

Over the past six months, domestic embodied intelligence has entered a period of rapid acceleration. Ever since the "martial arts bots" appeared on the Spring Festival Gala, the overall pace has picked up even more.

News about humanoid robot mass production, supermarket pilots, and factory floor deployments has become noticeably denser. The entire industry has hit the gas pedal.

Three months ago, we gave a systematic overview of domestic embodied intelligence players: the SOP (Scalable Online Post-training) framework released by Jianlan Luo's team at AgiBot's Embodied Intelligence Research Center. Our core assessment was:

Whether humanoid robots can exist in the real world over the long term depends on whether they can keep learning after deployment — making the real world itself a training ground.

Yesterday, the team published another research paper, titled:

LWD (Learning While Deploying), with real-world reinforcement learning pre-training + post-training at its core.

Full paper here:

https://finch.agibot.com/research/lwd

If SOP addressed "can robots keep learning in the real world," LWD asks "can robots get stronger on their own through use."

🚥

The Crossing team downloaded the paper immediately and gave it a close read.

Below, we walk through the LWD system's logic and break down several points worth noting.

First, back to that old question: what's blocking robots from getting stronger

Embodied intelligence has been a high-potential area that the Crossing team has tracked closely. The technical approaches in this space are complex.

Let's quickly recap.

VLA (Vision-Language-Action) foundation models have given robots generalizable capabilities, but generalization and real-world reliability are two different things.

The more tasks and the more complex the environment, the more a fixed pre-trained policy will break down in certain scenarios. Robots that look stable in demos hit a wall of long-tail problems once they're in stores, homes, and factories.

The industry's current mainstream "fixes" fall roughly into these categories:

[1] Collect more human demonstration data and retrain the model;

[2] Have operators monitor during deployment, take over when issues arise, and correct trajectories one by one;

[3] Manually annotate failure cases after the fact and fine-tune again.

All three work, but they share the same bottleneck:

Every time the robot needs to get stronger, someone has to be watching. The more robots deployed and the more complex the tasks, the heavier the human overhead — while marginal returns diminish quickly.

This has become one of the most genuine obstacles to robots entering offline scenarios.

LWD's essential contribution is a methodology that switches the source of robot improvement from external human supervision to interaction experience generated by the robots themselves.

Humans are still in the loop, but they're no longer a mandatory bottleneck for every model update.

What exactly is AgiBot's LWD doing

As we noted, LWD is a fairly systematic methodology. Rather than re-summarizing the entire paper, we've pulled out several highlights.

First, a simple definition.

LWD (Learning While Deploying) is a fleet-scale reinforcement learning framework for general robot policies.

"Fleet-scale" is an engineering-heavy term, but the idea is simple: get a whole cluster of robots learning together.

In short, a pre-trained VLA (Vision-Language-Action) model goes online to run tasks and autonomously collect online data; this new data is mixed with original offline data to retrain the model; the updated model is pushed back to the robots to keep running; and the cycle repeats.

This loop is the core of the LWD framework. Let's look at several specific points worth noting.

1) The real-world "data flywheel" starts spinning

LWD's first core element is a data flywheel driven by real-world reinforcement learning:

A robot cluster executes real tasks → execution trajectories (successful, failed, human-intervened) are sent back to the cloud → the cloud updates the policy with this data → the updated policy is redeployed to each robot.

This already had a prototype in SOP. SOP established the paradigm skeleton of "online post-training"; LWD is the first to make embodied models continuously improve through autonomous experience via RL pre-training + post-training.

What's the difference?

SOP was closer to "having robots imitate better trajectories." LWD switches the training objective to "maximizing task success." Beyond repeating human demonstrations, robots use their own execution outcomes to judge which actions move closer to completing the task.

The flywheel's speed is directly determined by cluster runtime. More robots and longer operation mean more real-world data and faster model updates.

2) Putting all "imperfect trajectories" to use

The second notable point is how LWD defines "useful data."

In traditional imitation learning, only successful human expert demonstrations make it into the training set. Failed, stuck, and corrected trajectories are treated as noise and discarded.

LWD is a reinforcement learning framework — the logic changes. What it needs is causal signal between actions and outcomes; failures provide this signal just as much as successes.

They go into the same replay buffer and enter the next round of policy updates together. A robot running in a real environment produces mostly imperfect trajectories.

Making use of imperfect trajectories lets the data pool grow alongside deployment scale.

3) Two core algorithms: one for "evaluation," one for "improvement"

Applying reinforcement learning to generalist VLA policies isn't as simple as running off-the-shelf RL algorithms. The paper spends considerable space on two algorithmic components: DIVL and QAM.

Both are fairly complex; we'll explain them as simply as possible.

DIVL (Distributed Implicit Value Learning) handles "evaluation."

In RL, you need to score each action. The traditional approach is predicting a single number. But real-world deployment involves complex data conditions where a single number can easily go off track.

DIVL replaces "predicting a score" with "estimating a distribution" — what range is this action's score most likely to fall into, then taking the needed value from that distribution. This is more stable than predicting a number directly.

QAM (Q-learning with Accompanying Matching) handles "improvement."

Modern VLAs generate actions through multi-step processes (flow or diffusion-based generation). Traditional RL doesn't work well on this structure — either it's intractable or training becomes unstable.

QAM's solution doesn't touch the backward pass through the entire generation process. Instead, it makes a small adjustment at each generation step, gradually nudging actions toward higher scores.

DIVL evaluates, QAM improves — they work together. LWD uses this same algorithm pair in both offline and online phases, avoiding the transition issues that arise when some methods switch algorithms between phases.

4) A unified "offline → online" training pipeline

LWD splits the full training process into two phases but uses the same algorithms and the same learner.

[1] Phase one is offline initialization, with data from three sources: historical expert demonstrations, rollouts from past policies (both successes and failures), and manually collected failure-mode exploration data.

[2] Phase two is online continuous training.

The significance of this design is that the critic learned offline can transfer directly to the online phase, avoiding the instability period that occurs at handoff in some previous methods.

5) Validation on real physical hardware

LWD's validation was done on actual hardware.

The platform is AgiBot's own Agibot G1 dual-arm robot, with two 7-DOF arms and 3 RGB cameras per unit, policies running joint position control at 30 Hz, in a cluster of 16 units.

Tasks totaled 8, split into two groups:

Group 1: Dynamic supermarket restocking (4 tasks): flat shelf restocking, misplaced item correction, cooler restocking (door-opening), and open cooler restocking (handling cardboard boxes). These mainly test semantic recognition and instruction understanding.

Group 2: Minute-scale long-horizon tasks (4 tasks): cocktail mixing (8 sequential sub-steps), kung fu tea (5 steps), fresh juice (6 steps), and shoebox packing (5 steps).

Each episode runs 3–5 minutes with a dozen or so contact-rich physical operations, demanding both intermediate state recovery and long-span coherence.

6) The results: average success rates across 8 tasks:

It's worth emphasizing that these results come from a single generalist policy, not a separate model trained for each task.

Comparing LWD offline versus online, a clear trend emerges: as real-world deployment data continuously feeds back, model performance improves on every task.

This trend is more pronounced on long-horizon tasks.

Beyond success rates, the paper also reports cycle time. On long-horizon tasks, LWD shows lower average cycle time with smaller variance. Beyond simply completing tasks, execution becomes smoother with less trial-and-error and hesitation.

Offline data is dominated by long-horizon tasks, with roughly one-third being failure samples. This data would mostly be discarded in traditional behavior cloning, but LWD's reinforcement learning framework can leverage these failure signals, turning them into a key source for continued improvement.

Zooming out

Looking at SOP and LWD together reveals a fairly clear trajectory. SOP solved the paradigm question; LWD puts reinforcement learning to work within that paradigm.

One answered "can it learn"; the other answered "can it get stronger on its own."

In our "2026 AI Kickoff Talk" podcast, Yusen Dai made a point: beyond pre-training and post-training, the "third pocket" that will determine the next phase's ceiling may be Online Learning — a system's ability to continuously absorb feedback and adjust its behavior after deployment.

LWD pushes this assessment a step closer to practical implementation:

Above the Online Learning layer, you need Online RL that leverages both success and failure signals — only then does this truly land on generalist robot policies.

When robots can keep getting stronger from every execution, deployment itself shifts from an endpoint to the starting point of another evolution.

AgiBot's paper is a concrete milestone on this path.

Crossing is looking for independent writers to cover AI product and model evaluations.

If you've written pieces like "Hands-on with PixVerse C1" or "Hands-on with LibTV," please contact zeo0811@gmail.com. Your email should include: ① a brief bio, ② AI evaluation pieces you've written.

We offer competitive rates. Looking forward to observing and documenting the AI era together 🎪