Newsletter
The Frontier

Nvidia GR00T 2: robots learn tasks from human video, no teleop

Editorial

Editorial

— 9 min read
Share
Nvidia GR00T 2: robots learn tasks from human video, no teleop

Nvidia's robotics lab stopped mapping camera pixels straight to motor commands and started predicting the next second of video instead. That single change is why a robot can now do a job it has never been taught.

Nvidia Research published GR00T 2, a robot foundation model that learns manipulation tasks from video of people and then performs them on robot bodies that never saw a demonstration of the task — with no teleoperation at all. Nvidia calls the architecture a world-action model: one diffusion transformer denoises future video and the matching robot actions together, then small per-robot heads translate that plan into motor commands for whatever body it is running on — the world-model turn in robotics we looked at in Robots are learning what their actions do, not what to do. Because the model has to predict how the scene should change, it learns physics from footage of humans instead of from hours of an engineer driving a robot arm by hand, according to Nvidia's technical release.

The pretraining mixture explains the design. Nvidia trained GR00T 2 on 402,730 hours of egocentric video of people at work, 10,184 hours of demonstrations recorded with a handheld UMI gripper across 2,897 tasks, and 23,500 hours of robot teleoperation spanning parallel-jaw arms, dexterous hands and the Unitree G1 humanoid. The company has put 100,000 hours of that human footage on Hugging Face as Human-1M-100K, which makes the data story checkable rather than rhetorical.

Nvidia also ran the scaling experiment that the claim depends on. Pretraining four versions of the model on none, 1%, 10% and 100% of the human video, then testing on 72 dexterous tasks, task progress rose at every step: 64% to 77% in video rollouts, and 50% to 70% on the real robot. Human video is doing real work in the weights, not just filling a slide.

The number that matters is under three minutes. On 50 tasks that appear only in other robots' data or in human video — never in the target robot's own training set — GR00T 2 succeeds 40% of the time zero-shot from the instruction alone. One demonstration per task lifts that to 71%. Five demonstrations, under three minutes of robot data, lift it to 95%, with 37 of the 50 tasks succeeding on every trial. Each adaptation is a 30-minute fine-tune on a single RTX 5090.

Set that against the current economics of robot learning. Teaching a new task usually means teleoperating the machine for hours and repeating it across every robot model in the fleet; it is the line item that keeps general-purpose manipulation in pilot programs — the same bottleneck behind Skild's S1 teaching robots a 10-minute task from one video. A 30-minute fine-tune on a consumer GPU, from demonstrations a person can record with a glove and a camera, is a different cost structure — and it is the claim the rest of the release is built to support.

The transfer demos are the vivid part. Shown only human demonstrations of a chemistry-lab procedure — gloves, head camera, wrist camera — a two-handed robot lined a funnel with filter paper and wet it with a wash bottle in 65% of trials. Shown only with a handheld UMI gripper, a YAM arm assembled a toy dump truck end to end in 60% of trials, a four-minute task. With a DexUMI exoskeleton that mirrors a robot hand's joints, the model learned to prepare banana yogurt and scoop granola, and to strike a match and put it out in water.

Dexterity gets the same treatment with a small amount of teleoperation rather than none: with two 22-DoF Sharpa hands, GR00T 2 seats GB300 memory modules in 85% of trials and assembles a toy truck in 65%, tasks Nvidia says previous models could not do. Reinforcement learning on the real robot, driven only by task-level success signals, lifts three contact-rich insertion tasks from 40–76% after supervised post-training to 88–96%.

The agent does the thinking, the model does the hands

GR00T 2 is also built to be driven by a reasoning model. In Nvidia's setup a frontier agent acts as the planner — it splits a long task into steps, checks progress, writes recovery code — and calls GR00T 2 through an API for the contact-rich manipulation. Each assembly trial begins by dumping the parts onto a table, so every run starts from a layout that is not in the post-training data. With the agent, GR00T 2 assembled a toy plane in 98 of 100 trials, a tractor in 97 and a Hot Wheels car in 97: 292 of 300 overall. The same post-trained policy without the agent managed 71 to 84, and supervised fine-tuning alone 23 to 69.

The cheaper steering trick is more interesting than the score. A planner can mark a grasp point on the target object and a release point at its destination directly on GR00T 2's camera view, colour-coded by arm. Success on short-horizon "what and where" instructions rose from 40% to 92% on the YAM arm and from 30% to 70% on Dexter; on long-horizon trash sorting, marks took the YAM arm from completing no full episodes to 17 of 20. No new training data was needed, because the marks are extracted from gripper states that already exist in the demonstrations.

The compact variant is what makes this shippable. GR00T 2 Turbo uses a smaller joint video-action transformer — 2 billion parameters — and a higher-compression video encoder, and it runs onboard a Unitree G1 humanoid on its Jetson Thor computer: 124 milliseconds per prediction, planning 1.6-second chunks of whole-body motion about once a second while a whole-body controller executes at 50 Hz. It scooped and served popcorn in all 48 trials and cleared a table of mixed objects in 93% of 30. Adding simulated episodes raised its out-of-distribution success from 6.7% to 63.3%, against 40% for the previous GR00T 1.7 generation.

There is a credible skeptical case, and Nvidia's own numbers make it. Zero-shot success is 40%, which means most novel tasks fail on the first attempt; the 95% figure comes after fine-tuning on demonstrations collected in exactly the target robot's frame, so it measures fast adaptation, not autonomy. The reasoning planner is the weak link in the loop: The Decoder reports that during trash sorting the robot stood idle roughly a third of the time waiting more than seven seconds on average for the next plan, and that GR00T 2 has no tactile sensing and misses subtle failures like a half-turned screw. Success rates across the headline demos ranged from 10% to 80% depending on the task. Code is not out until December, so nothing here can be independently reproduced yet.

Still, the direction is right. The scarce input in robot learning has been robot time; GR00T 2's bet is that human video is close enough to substitute for most of it, and Nvidia has published the scaling curve that says so. That is a more falsifiable claim than a demo reel, which is why December's code release will matter more than today's videos.

OpenAI opened ChatGPT to business profiles, with a hub where companies claim their listing and an agent that answers customers inside the chat. Businesses in ChatGPT is in beta: any eligible company in a market where ChatGPT operates can claim its profile through Business Hub, verify it, update details and publish product feeds with pricing and availability, per OpenAI's announcement. Verified businesses also get aggregated insights — mentions, profile views, website clicks and the discovery topics where their site gets cited — with no conversation data shared back. Eligible US ecommerce and online service businesses can build a Business Agent that users can open from a model response, which can also be sponsored and used as a landing point for ChatGPT Ads.

The strategic read is not subtle. TechCrunch calls it ChatGPT becoming a little more like Google, and the comparison is fair: this is a business profile and a stripped-down analytics console, wrapped around a 1.2 billion weekly-user assistant. It is also OpenAI's second attempt at commerce after the checkout-and-commission effort it retired in March, and this one asks businesses to do the work of supplying structured data rather than asking OpenAI to run a storefront. Shopify merchants can connect a store from Business Hub or the Shopify admin, which gives OpenAI catalogue coverage without a merchant-by-merchant sales motion. The open question is verification: OpenAI's Benji Shomair told reporters there are "multiple verification methods" but did not describe them, and a claimed profile is only as good as the check behind it.

Gemini 4 Carbon is now generally available, at $4 per million input tokens and $20 per million output tokens once the introductory rate of $2 and $10 ends. Google's model spent two weeks gated to paying developers and business customers through its Fairwind security program; it now ships through the Gemini API, AI Studio, Android Studio and Gemini for Business, per Google's launch post. Google positions Carbon as a complement to Gemini 4 Argon rather than a replacement — Argon for most software engineering and knowledge work, Carbon for tasks that push model intelligence — even though both carry the same token price, and Carbon burns more tokens to get there.

The practical numbers are per-task, not per-token. Artificial Analysis measured Carbon at $7.76 per solved task, cheaper than the older Gemini 3.8 Flash while scoring higher; Fable 5.2 costs about $9 per task at its High tier and still leads the overall index by two points, 62 to 60. Google also claims first place on CursorBench 4.0 at just under 60%, and second on a new multimodal document benchmark built with Artificial Analysis, at under one-sixth the top performer's cost. The security results are the ones no rival has matched: through Wiz's Scan for Good program, Carbon found real zero-days in two large SaaS providers in under two hours, and it lifted detection rates on open-source code by more than 25% over Argon inside Google's Project Fortify. Alongside the model, Google previewed a "/swarm" mode in Antigravity that turns one prompt into a team of agents — a bet the rest of the industry has been publicly skeptical of.

What to watch: whether Nvidia ships GR00T 2's code in December as promised, and whether the planner latency that idles the robot for a third of a run gets fixed before anyone calls this production-ready.

If three minutes of video is enough to teach a robot a job, which task in your workplace goes first? Write to us via our contact page.

Get SiliconNoon in your inbox

One email a day with the AI stories that matter — what happened, the numbers, and why it matters. No spam, unsubscribe in one click.

Read more

Motus2: a robot world model that grades its own actions

Motus2: a robot world model that grades its own actions

Shengshu Technology, the Chinese world-model lab, shipped the rare robotics release where the loop actually closed: one model that acts, imagines, and scores itself — then learns from the grade. Shengshu released Motus2, a "self-evolving" general world model for dexterous manipulation that puts three capabilities — acting, predicting, and evaluating — inside one shared-parameter video-action model, according to the project page and the paper on arXiv.

Google's Genie 4 world model adds sound, speech and longer scenes

Google's Genie 4 world model adds sound, speech and longer scenes

Google DeepMind put Genie 4 into research preview on Tuesday, and it is the first real-time world model that generates the soundtrack along with the scene. The number worth remembering is two minutes: that is how long Genie 4 can hold a consistent world, double what Genie 3 managed. Genie 4 generates audio, interactive dialogue and physical motion at the same time as the pixels, and Google is pitching it as the base layer for simulating anything.

Kling 4.0 goes live with 30-second native video at 10-bit HDR

Kling 4.0 goes live with 30-second native video at 10-bit HDR

Kuaishou's Kling AI has spent two years becoming the video model people actually finish projects with. Its new flagship, Kling 4.0, went fully live on October 15 — and this round is less about prettier frames than about the two things that make a generated shot usable in a real edit: length and control. Kling 4.0 generates up to 30 seconds in a single native pass, outputs 10-bit HDR at 4K, 1080p and 21:9, and takes up to 15 reference assets in one task.

Jev 1.14.0 can read images — if you open its 'Danger Zone'

Jev 1.14.0 can read images — if you open its 'Danger Zone'

TypeSafe's decision model learned to look at pictures this morning, but the company shipped the capability behind a warning label instead of a launch announcement. Jev 1.14.0 adds image input, a new question type called Box, and a 262,000-token context window — all three shipped as experimental features behind a program TypeSafe calls the "Danger Zone." The features are off by default and have to be enabled per request through the API; TypeSafe's own documentation says they do not meet…

Byteification turns Qwen3 and Llama 3 into byte-level models

Byteification turns Qwen3 and Llama 3 into byte-level models

The tokenizer is the last piece of the language-model pipeline that was never learned end to end — and a Nature paper out this week shows it can be swapped out for less than one percent of a pretraining budget. Researchers have retrofitted four open subword language models into true byte-level models using less than 1% of a typical pretraining budget, and the result — published 7 October in Nature — closes the performance gap that has kept byte-level models a niche curiosity for nearly a…

Two fixed opening tokens push a base model past its RL version

Two fixed opening tokens push a base model past its RL version

A new paper from MIT, UC Berkeley, Washington University and the Allen Institute for AI says much of what reinforcement learning teaches a model may come down to how it starts an answer. One good prefix, fixed in advance, is enough to close the gap. Fix the first two tokens and Olmo-3-7B beats its own RL-trained twin. The paper — "Base Models Can Reason By Taking a Cue From Training Data" — tested what happens when researchers pre-fill a model's response opening instead of letting it choose.