How we build it, in the open.

Watching the robotics industry today feels familiar. Every month brings larger Vision-Language-Action models, more capable world models, increasingly impressive demonstrations. The implicit thesis is one the field has heard before: scale the model, scale the capability.

From inside a warehouse deployment, the picture looks different. Models, by themselves, are no longer the dominant constraint on what robots can do reliably. The constraint has moved upstream — to the experience the models are trained on, and the systems that produce it.

Large Language Models became possible not only because of advances in transformer architectures, but because the internet existed as an effectively unlimited source of high-quality training signal. Physical AI has no such substrate. Every piece of robotic intelligence must ultimately be learned from interaction with the physical world, and the cost of collecting that interaction remains one of the field's most underdiscussed bottlenecks.

In our previous post on the three generations of Physical AI, we argued that the architecture of robotic decision making is splitting into three layers — policy, world model, value function. A similar decomposition is happening on the data side. The robotic experience that trains these models is increasingly drawn from three distinct sources, each serving a different purpose.

Three Sources of Embodied Experience

The strongest robotics foundation models in development today are not built on robot demonstrations alone. They draw from a stack of embodied data sources, each filling a different gap.

Real Robot Data is generated by deployed robotic systems operating in real environments. For warehouse automation, this means trajectories from pick-and-place, bin picking, tote handling, palletizing, conveyor loading, and inventory movement, with the full stack of observations attached: RGB and wrist-camera images, joint states, end-effector poses, gripper states, force and torque signals, action trajectories, and success or failure labels.

This data captures the parts of the physical world that simulators systematically underrepresent — friction at the edge of grasp stability, compliance in deformable objects, the way real sensors degrade and recover, the long tail of unexpected object configurations. As a result, it remains the highest-quality learning signal available for Physical AI. It is also, by some margin, the most expensive. An hour of real robot data carries hardware, operators, infrastructure, maintenance, and supervision costs that no amount of model architecture work can amortize. For many robotics companies, the cost curve of real-world data — not the loss curve of the model — is what actually gates progress.

Handheld Gripper Data is the most recent addition to the stack, and arguably the most interesting. Rather than collecting demonstrations through a robot at all, Stanford's Universal Manipulation Interface — UMI, a handheld-gripper collection method, a research method unrelated to the Laplacian Platform — lets a human operate the same gripper hardware the robot will use, holding it directly while performing tasks. Because the wrist camera viewpoint and the end-effector action space are the gripper's properties rather than the robot's, the embodiment gap between human demonstration and robot execution shrinks to something tractable — much smaller than the gap between, say, observing a person grasp something with their bare hand.

The economic consequence is significant. A single human operator can collect manipulation data at roughly an order of magnitude lower cost than teleoperating a real robot, while producing trajectories that are directly usable for robot policy learning. Handheld gripper data doesn't replace real robot data — the embodiment gap is reduced, not eliminated — but it sits in a sweet spot between scalability and fidelity that neither pure robot data nor pure observational data occupies.

Ego-Centric Data comes from wearable cameras, AR glasses, and head-mounted recording systems — the perspective of a person doing work, captured at scale. Warehouse staff, factory operators, and logistics personnel generate enormous quantities of task-relevant footage in the normal course of their jobs. Crucially, ego-centric data usually lacks the action labels that supervised manipulation learning depends on. What it offers instead is breadth: thousands of hours of how tasks unfold, how objects are handled, what successful execution looks like across thousands of variations.

For perception and language grounding, this breadth is decisive. It teaches a model the vocabulary of the physical world — what objects exist, how environments are organized, what activity looks like — in a way that no robot fleet can match for cost. It is the closest thing Physical AI has to the corpus of the internet, even if its uses are narrower.

A Data Mix for Each Stage of Growth

The right balance among these three sources changes as a robotics company matures. Trying to use the same mix at every stage is one of the more common strategic errors in the field.

Stage 1: Foundation Building. At this stage there are no robot fleets to learn from, or only a handful of platforms in the lab. The goal is to train the first generation of capable models on whatever priors can be assembled cheaply. A reasonable mix here is roughly 60% ego-centric, 30% handheld, 10% real robot. Scale matters more than precision; the priority is broad manipulation knowledge and general physical understanding, and human-generated data is the fastest path to it.

Stage 2: Product-Market Fit. Once pilot deployments begin, the calculus shifts. The model now has to handle a specific operational reality — particular SKUs, particular lighting, particular workspace constraints — that no amount of generic prior fully prepares it for. A mix closer to 30% ego-centric, 40% handheld, 30% real robot reflects this. Real robot data starts pulling more weight, because the failure modes that matter for deployment can only be observed in deployment. Handheld gripper data continues to dominate the manipulation-specific contribution; ego-centric data shifts toward a supporting role.

Stage 3: Fleet Learning. Once robots are operating at scale, the economics invert. Every deployed robot generates successes, failures, recoveries, and edge cases at a cost per episode that — with the right pipeline — approaches zero. A representative mix at this stage is around 10% ego-centric, 20% handheld, 70% real robot. Most importantly, failures become some of the most valuable training signal available, because they reveal exactly where the current model breaks down. A mature Physical AI company should treat every deployed robot as a continuous data generator first, and a revenue-bearing unit second.

Why Most Data Flywheels Never Spin

Almost every robotics company talks about data flywheels. Very few actually build them. The reason is not strategic blindness — it is operational difficulty.

In most deployments, the path from operational experience back into model improvement is broken in multiple places. Logs are written but hard to retrieve. Sensor streams are inconsistent across robots and software versions. Labeling success and failure requires human review. Reproducing a specific failure in a controlled setting is laborious. Each step in the pipeline requires manual intervention, which means the marginal cost of the next training example never approaches the zero that the flywheel narrative requires.

The deeper problem is that data collection is treated as a separate project from robot operation, rather than as a natural byproduct of it. As long as collecting one more training episode requires someone to deliberately collect it, the flywheel will sputter no matter how impressive the headline metrics.

Closing the loop requires investment in unglamorous infrastructure — observability for production robots, automatic labeling pipelines, deterministic replay, dataset versioning, evaluation harnesses that run before deployment rather than after. None of this is exciting to demo. All of it is necessary to make a fleet that learns rather than a fleet that merely operates.

The Real Goal: Lowering the Cost of Experience

The future of Physical AI is less about training larger models than about dramatically reducing the cost of producing the experience those models depend on. A working data flywheel — robots collecting data automatically, that data flowing into evaluation and training automatically, improved models flowing back to robots automatically — is not a marketing story. It is the actual mechanism by which a robotics company turns deployments into compounding improvement.

Tesla is the clearest proof that this inversion works. Its autonomy program is not fed by a handful of instrumented R&D vehicles; it is fed by millions of customer cars already on the road, streaming back real driving experience — every disengagement, every intervention, every edge case — at a marginal cost that rounds to zero. The cars people bought are the data engine. Physical AI needs the same inversion: the robots that customers deploy, not the ones in the lab, must become the primary source of training experience. And that only happens if the cost of collecting an episode keeps falling until collection is a byproduct of operation rather than a project of its own.

This is why the same operational data has to serve more than one purpose. In the three-generation architecture we described previously, fleet data trains not just the policy, but also the world model and the value function. Every successful pick refines the policy. Every failure provides supervision for the world model. Every near-miss recovery sharpens the value function. The same episode contributes to three layers of intelligence at once, but only if the underlying data infrastructure can route it accordingly.

Physical AI Needs Its Own Internet

The internet did not cause language models, but it made them possible. The corresponding infrastructure for Physical AI will not be a network of websites. It will be a network of robots and human-collected experience, instrumented so that every interaction can flow back into training without manual intervention.

The companies that succeed in Physical AI will not necessarily be the ones with the largest models. They will be the ones that can continuously generate, organize, evaluate, and learn from embodied experience at the lowest cost per useful training example. Real robot data provides accuracy. Handheld gripper data provides scalable manipulation. Ego-centric data provides breadth. The combination, run through a pipeline that actually closes, is what an internet for robotics will look like.