18 posts tagged “physical-ai”
2026
OpenUSD: Advanced Patterns and Common Gotchas.
This post covers deeper OpenUSD concepts: schemas, rendering rules, performance patterns, and the gotchas that surprise people.
[... 1,141 words]OpenUSD Mastery: From Composition to Pipeline — A SO-101 Arm Journey
OpenUSD (Universal Scene Description) is not just a 3D modeling format. It is a universal language that describes complex scenes, their relationships, and their properties. Think of it as JSON for 3D worlds, but one that can do infinitely more.
[... 1,305 words]Learning OpenUSD — From Curious Questions to Real Understanding
Written as I explored OpenUSD before my exam. These are real questions I asked, and the answers that actually made things click for me.
[... 1,135 words]How Everything Connects — NVIDIA’s Cosmos Pipeline from Simulation to Real-World Robots
Training robots and autonomous vehicles is fundamentally dangerous and expensive. You can’t crash 1,000 cars to teach collision avoidance. You can’t let a robot fall off cliffs to learn edge detection. NVIDIA’s solution is an end-to-end pipeline that generates synthetic data. The data is so realistic that AI models trained on it transfer directly to the real world. Here’s how every piece connects.
[... 1,777 words]Complete Guide: Setting Up XRoboToolkit for Robot Teleoperation with Pico 4 Ultra on WSL2
A step-by-step guide to XR-based robot teleoperation with the Pico 4 Ultra headset, XRoboToolkit, and MuJoCo simulation. Everything runs on Windows WSL2.
[... 1,293 words]ROS 2 Humble: Complete Installation Guide with Turtlesim from Zero to First Node
This is a complete walkthrough for installing ROS 2 Humble on Ubuntu 22.04 and running your first robot simulation with Turtlesim. I wrote this after I completed the process myself. The official docs are thorough but scattered across many pages. This puts everything in one place, from locale setup to writing your first Python node.
[... 2,194 words]How a VLA Controls a Robot Arm: GR00T N1.5 System Architecture from Camera to Motor
I built a robot arm system that uses NVIDIA’s GR00T N1.5, a Vision-Language-Action (VLA) model. The system picks up objects from a table with only a camera, natural language instructions, and 50 demonstration episodes. After it worked end-to-end, I wanted to record the full system architecture for anyone who wants to understand how all the pieces connect.
[... 909 words]Collecting Training Data for VLA Robot Fine-Tuning (The Hard Way)
A Vision-Language-Action model takes camera images and a language instruction as input, and outputs robot joint actions. NVIDIA’s GR00T N1.5 is one such model. It is pre-trained on millions of robot demonstrations, and you can fine-tune it for your specific robot and task. The catch: even though GR00T is pre-trained, you still need your own demonstrations. These teach it your robot’s exact joint calibration, camera angles, and task environment. Without this, the model generates actions that are plausible in general but wrong for your specific setup.
[... 1,766 words]Teaching a Humanoid Robot to Wave: Custom Motions with GEAR-SONIC
GEAR-SONIC can track arbitrary motions, not only its built-in locomotion styles. You do this in three stages:
[... 830 words]VLA → WBC → MuJoCo: Two Ways to Wire Up NVIDIA’s GR00T Humanoid Stack
There are two ways to connect NVIDIA’s GR00T stack from vision-language all the way down to physics simulation. One is the official NVIDIA eval pipeline. The other is a custom pipeline that uses the SONIC C++ binary. I configured both. Here’s how they work and where they differ.
[... 696 words]From Vision to Torques: How NVIDIA’s GR00T Stack Controls a Humanoid Robot
NVIDIA’s GR00T stack for humanoid robots has three layers:
[... 981 words]GEAR-SONIC
GEAR-SONIC (Supersizing Motion Tracking for Natural Humanoid Whole-Body Control) is the big upgrade over the Decoupled WBC approach in the GR00T stack. It’s a completely different approach to humanoid control. It controls the whole body as one unit, trained on human motion data rather than hand-crafted reward functions.
[... 335 words]NVIDIA’s GR00T Whole-Body Control stack in MuJoCo
I’ve been running NVIDIA’s GR00T Whole-Body Control stack in MuJoCo. It is the sim-to-real bridge for humanoid robot locomotion. A MuJoCo viewer that shows a simulated robot walking might look like a toy. But the neural network policy inside it is the same binary that runs on a real Unitree G1. Here’s what actually happens.
[... 776 words]GR00T Architecture: A Systems Engineering Breakdown
GR00T is not just a VLM. It is a Perception → Reasoning → Control generator stack.
[... 813 words]GR00T N1.6 Fine-Tuning — Full Internal Deep Dive
GR00T N1.6 is NVIDIA’s Vision-Language-Action (VLA) model for humanoid robot control. I spent time studying the internals. Here’s a comprehensive explanation of exactly how fine-tuning works, from model architecture to gradient flow to the data pipeline.
[... 1,196 words]PPO vs VLM
Modern humanoid robots combine two fundamentally different kinds of intelligence:
[... 638 words]How GR00T Merges Vision, Chat, and Action
The biggest challenge is that vision models speak “Image-ish” (pixels) while chat models speak “Text-ish” (tokens). GR00T uses a specialized component called a Projector to act as a real-time translator.
[... 377 words]GR00T N1.6 Architecture and Parameter Distribution
GR00T uses a massive “backbone” to understand its surroundings. It combines SigLIP 2 (for vision) and Qwen 3 (for language). The eyes stay frozen to keep perception stable. The reasoning layers are partially trainable to help the robot learn specific tasks.
[... 360 words]