Akshay Parkhi's Weblog

NVIDIA’s GR00T Whole-Body Control stack in MuJoCo

20th February 2026

I’ve been running NVIDIA’s GR00T Whole-Body Control stack in MuJoCo. It is the sim-to-real bridge for humanoid robot locomotion. A MuJoCo viewer that shows a simulated robot walking might look like a toy. But the neural network policy inside it is the same binary that runs on a real Unitree G1. Here’s what actually happens.

What sim2mujoco Actually Is

It’s the sim-to-real transfer bridge. The workflow:

1. Train a neural network policy in NVIDIA Isaac Sim (GPU-accelerated, thousands of robots in parallel)
2. Validate in MuJoCo. MuJoCo is lightweight and has accurate physics. It catches policy failures before you touch hardware.
3. Deploy the same ONNX policy onto a real Unitree G1 robot

The policy file is the same binary (.onnx) at every stage. MuJoCo is just the test bench, not the end product. Think of it like watching a self-driving car in a simulator. It looks like a video game, but the model inside it drives real cars.

Real-World Applications

This exact stack (GR00T WBC) is used for:

• Warehouse automation: humanoid robots walk, pick, and place in unstructured environments.
• Manufacturing: robots that can navigate factory floors, climb stairs, and handle objects.
• Hazardous environments: inspection in places that are unsafe for humans (nuclear, disaster zones).
• General-purpose humanoid robotics: NVIDIA’s GR00T project is their bet on foundation models for humanoid control.

Why It Matters Technically

• The policy that runs here is the same binary (.onnx) that runs on real hardware. It is not a separate simulation-only thing.
• Whole-body control with 29 DOFs (legs + arms + torso) is an unsolved hard problem. This is state of the art.
• The PD controller, observation space, and action scaling are all tuned to match real actuator dynamics.

What Runs When You Launch the Simulation

Every 0.005s timestep, the simulation executes this loop:

reads sensors: joint positions, velocities, IMU (gravity, angular vel)
        ↓
builds observation vector (86 dims × 6 history frames = 516)
        ↓
feeds into ONNX policy (neural net)
        ↓
target joint angles (15 values)
        ↓
PD controller → joint torques
        ↓
MuJoCo physics engine steps the simulation
        ↓
repeat

What each piece is:

Component	What It Does
GR00T-WholeBodyControl-Balance.onnx	The trained neural network — same file you’d load on a real G1 robot
g1_gear_wbc.xml	MuJoCo model of the G1 — masses, joint limits, meshes, actuators — standing in for real hardware
PD controller	Converts target joint angles → torques, same math runs on real motor controllers
compute_observation	Simulates what real sensors (IMU, joint encoders) would report
Keyboard input	Simulates the command interface (joystick/autonomy stack on real robot)

MuJoCo replaces the physical robot. Everything else is identical to what runs on real hardware: the policy, the PD controller, and the observation pipeline. If the robot walks here, it has a high chance of walking on the real G1. If it falls here, it would fall in real life too. That’s the whole point: break things in simulation, not on a $50k+ robot.

Which Policy Is Actually Running

The gait script only loads one policy: GR00T-WholeBodyControl-Balance.onnx. That is the balance/standing policy. It does not load GR00T-WholeBodyControl-Walk.onnx.

Compare with the other script, which loads both:

# run_mujoco_gear_wbc.py
self.policy = self.load_onnx_policy(self.config["policy_path"])        # Balance
self.walk_policy = self.load_onnx_policy(self.config["walk_policy_path"])  # Walk

So the gait script runs the Balance policy with gait logic layered on top in Python code. The Walk neural network (GR00T-WholeBodyControl-Walk.onnx) sits unused. These are NVIDIA’s pre-trained policies. They were trained in Isaac Sim with reinforcement learning on the Unitree G1.

Where Does GR00T Fit In

GR00T = Generalist Robot 00 Technology. It’s NVIDIA’s initiative to build foundation models that can control any humanoid robot. The hierarchy:

NVIDIA GR00T (project/platform)
    └── GR00T-WholeBodyControl (this repo)
            └── decoupled_wbc (the control framework)
                    └── sim2mujoco (what we're running)

The full GR00T project has multiple layers:

Layer	What It Does
GR00T Foundation Model	Large multimodal model — understands language and vision, generates robot actions
GR00T Whole-Body Control	Locomotion policies — walk, balance, recover from pushes
GR00T Dexterity	Hand and manipulation policies

The .onnx files in this codebase are low-level controllers that the GR00T project produced. They handle these tasks:

• Do not fall over.
• Walk when told to walk.
• Track height/orientation commands.

The Bigger Picture

In a full GR00T deployment:

Human says: "Go pick up that box"
        ↓
GR00T Foundation Model (vision + language → high-level plan)
        ↓
Commands: walk forward, turn left, reach arm...
        ↓
GR00T WholeBodyControl ← YOU ARE HERE
(balance + locomotion neural net)
        ↓
Joint torques → real robot moves

We’re running the legs of the stack. GR00T is the brain that would sit on top. It would send the locomotion commands that you currently send with the keyboard.