EgoHumanoid-V2

🤖 Human-to-humanoid skill transfer for coordinated whole-body loco‑manipulation.

Human demonstrations capture rich whole-body motion across diverse scenes without robot teleoperation. EgoHumanoid-V2 transfers these coordinated skills to a humanoid through coarse-to-fine action alignment: kinematic reference correction improves motion targets, and dynamics-aware refinement improves their execution through a frozen whole-body controller.

Visual alignment replaces human arms with rendered robot arms, while training-time image augmentation improves viewpoint robustness. A π₀.₅ vision-language-action policy learns from the aligned demonstrations and executes through Unitree G1 + BrainCo Revo2 + the SONIC whole-body controller. The paper evaluates zero-shot transfer on four real-world tasks without target-task robot demonstrations.

---

Workflow & Release Checklist

Coming soon: The remaining unchecked items are expected to be released within the next two weeks. Stay tuned!

1. Hardware Setup
   • Robot Hardware
   • Human Hardware
2. Environment Setup
   • Robot Control & Data Collection
   • Human Data Collection
   • Policy Training & Serving
3. Data Collection
   • Human Collection
   • Robot Collection
4. Data Alignment
   • Action Alignment ⭐
   • Vision Alignment
5. Model Training
   • Training pipeline
6. Policy Deployment
   • Deployment pipeline

---

🛠️ Hardware Setup

Human Hardware 🔴 PICO 4 Ultra Tracking: PICO 4 Ultra + trackers for whole-body and hand tracking Software: XRobo​Toolkit Setup guide · Python SDK Connec​tion: same LAN as the PC enable Head, Hand and Send 🔵 5× PICO Trackers Placement: five trackers both wrists, waist and both ankles Calibration: PICO Motion Tracker pair and calibrate in the app (guide) Mode: Full Body in XRobo​Toolkit enable Hand tracking (setup guide) 🟠 GoPro Mount: neck-mounted bracket Example: JD listing Alignment: match height, pitch and framing to the robot camera Connec​tion: Bluetooth pairing and WiFi setup; WiFi / COHN recording and download (API)

Robot Hardware 🟢 Unitree G1 Platform: Unitree G1, 29 body DOF excluding hands (joint layout) Control: SONIC whole-body controller runs on an external Linux PC Connec​tion: wired or WiFi to the PC via the ZMQ bridge 🔵 BrainCo Revo2 Hand: 6 active DOF per hand 11 including passive joints (specs) Software: official SDK hand control and state feedback Mount: custom G1–Revo2 adapters STL: part 1 · part 2 🟠 GoPro Mount: head-mounted bracket STL: part 1 · part 2 Alignment: match height, pitch and framing to the human camera Connec​tion: USB control and video to robot; wired / WiFi forwarding to host (API)

💻 Environment Setup

We use three separate environments: one for robot control, teleoperation and data collection; one for collecting human demonstrations; and one for policy training and serving.

🤖 Robot Control & Data Collection

Host

Ubuntu 22.04 · NVIDIA driver 580+.

1. Install dependencies.

sudo apt-get update
sudo apt-get install -y --no-install-recommends ca-certificates curl build-essential cmake git git-lfs libgl1 libegl1 libegl-mesa0 libglib2.0-0 ffmpeg libyaml-cpp-dev libeigen3-dev libmsgpack-dev libzmq3-dev nlohmann-json3-dev libssl-dev zlib1g-dev
curl -LsSf https://astral.sh/uv/0.11.14/install.sh | sh
source "$HOME/.local/bin/env"
curl -fL https://developer.download.nvidia.com/compute/cuda/repos/ubuntu2204/x86_64/cuda-keyring_1.1-1_all.deb -o /tmp/cuda-keyring.deb
sudo dpkg -i /tmp/cuda-keyring.deb
sudo apt-get update
sudo apt-get install -y --no-install-recommends cuda-minimal-build-12-4

2. Install TensorRT and ONNX Runtime. Download TensorRT-10.13.3.9.Linux.x86_64-gnu.cuda-13.0.tar.gz from NVIDIA to the repository root, then run:

sudo tar -xzf TensorRT-10.13.3.9.Linux.x86_64-gnu.cuda-13.0.tar.gz -C /opt
curl -fL https://github.com/microsoft/onnxruntime/releases/download/v1.16.3/onnxruntime-linux-x64-1.16.3.tgz -o /tmp/onnxruntime.tgz
sudo tar -xzf /tmp/onnxruntime.tgz -C /opt

3. Install environment. Repeat the exports when opening a new terminal.

export TensorRT_ROOT=/opt/TensorRT-10.13.3.9
export onnxruntime_ROOT=/opt/onnxruntime-linux-x64-1.16.3
export CUDAToolkit_ROOT=/usr/local/cuda-12.4
export LD_LIBRARY_PATH="$TensorRT_ROOT/lib:$onnxruntime_ROOT/lib:$CUDAToolkit_ROOT/lib64${LD_LIBRARY_PATH:+:$LD_LIBRARY_PATH}"
bash setup/robot/install_host.sh

4. Activate environment.

source sonic/.venv/bin/activate

Robot

ARM64 · Ubuntu 20.04 · ROS 2 Foxy · Python 3.8.

1. Install environment.

sudo apt-get update
sudo apt-get install -y python3-dev python3-venv build-essential cmake git python3-colcon-common-extensions ros-foxy-rmw-cyclonedds-cpp ros-foxy-rosidl-generator-dds-idl libyaml-cpp-dev libssl-dev libgl1 libglib2.0-0
bash setup/robot/install_robot.sh

2. Activate environment.

source /opt/ros/foxy/setup.bash
source .venv-robot/ros_ws/install/setup.bash
source .venv-robot/bin/activate

👤 Human Data Collection

Ubuntu 22.04 · Python 3.10.

1. Install environment.

sudo apt-get update
sudo apt-get install -y ca-certificates python3.10-dev python3.10-venv build-essential cmake git ffmpeg bluez network-manager
bash setup/human/install.sh

2. Activate environment.

source .venv-human/bin/activate

🧠 Policy Training & Serving

Ubuntu 22.04 · NVIDIA GPU and driver.

1. Install dependencies.

sudo apt-get update
sudo apt-get install -y --no-install-recommends ca-certificates curl git build-essential libgl1 libglib2.0-0
curl -LsSf https://astral.sh/uv/0.11.14/install.sh | sh
source "$HOME/.local/bin/env"

2. Install OpenPI.

bash setup/policy/install.sh

3. Activate environment.

source openpi/.venv/bin/activate

🎥 Data Collection

👤 Human Data Collection

1. Prepare devices.

Connect the host and PICO headset to the same network. Start the XRoboToolkit PC service on the host and the XRoboToolkit app on the headset, then enable body and hand tracking. Turn on the GoPro, enable Bluetooth pairing and select Video mode; the collector connects it to the host's WiFi network in the next step.

2. Start recording.

eh2-collect-human --task-name "{task_name}" --task "{task_instruction}"

Open http://127.0.0.1:8080. Enter the host's WiFi credentials if needed, then click Prepare collection → Start recording → Save episode / Discard episode. Wait for saving before recording again. Episodes are saved in LeRobot format to data/{task_name}/; new recordings append to the dataset.

3. Check and synchronize data.

eh2-sync-human --root "data/{task_name}"

Checks the recordings, synchronizes motion and video at 50 Hz, then checks the output. Results are saved to data/{task_name}_synced/, with synchronization and data quality reports in meta/.

🤖 Robot Data Collection

Coming soon.

⚡ Data Alignment

🦾 Action Alignment

We provide one example trajectory for each of the four tasks in examples. The simulation replays below compare these trajectories before (left) and after (right) alignment.

close_laptop.mp4	open_fridge.mp4
pull_curtain.mp4	close_drawer.mp4

1. Align motion.

Place the input recording at {task_name}/input.parquet. Set {endpoints} to hands for hand alignment or feet for foot alignment.

eh2-align \
  --input "{task_name}/input.parquet" \
  --endpoints {endpoints} \
  --output "results/{task_name}"

Results are saved under results/{task_name}/: data.parquet contains the human SMPL motion and aligned tokens; run.json records per-round metrics and the selected round.

2. Replay motion.

SMPL replay: encode the human motion online to simulate before alignment.

eh2-replay \
  --input "{task_name}/data.parquet" \
  --mode smpl \
  --output "results/{task_name}/before"

Token replay: replay the aligned motion tokens to simulate after alignment.

eh2-replay \
  --input "{task_name}/data.parquet" \
  --mode token \
  --output "results/{task_name}/after"

Each command saves replay.mp4 in its output directory. Results may vary slightly between runs.

🎨 Vision Alignment

Coming soon.

🤖 Model Training

Coming soon.

🚀 Policy Deployment

Coming soon.

📝 Citation

If you use this work, please cite:

@article{chen2026egohumanoidv2,
  title={{EgoHumanoid-V2}: Human-to-Humanoid Transfer of Coordinated Whole-Body Skills for Loco-Manipulation},
  author={Chen, Jin and Jiang, Yiming and Xu, Chongyang and Shi, Modi and Peng, Shijia and Chen, Li and Li, Tianyu and Xu, Mu and Chen, Yilun and Hoi, Steven and Li, Hongyang},
  journal={arXiv preprint arXiv:2609.37181},
  year={2026}
}

---

🙏 Acknowledgments

We sincerely thank the following projects and teams:

Presentation media comes from the EgoHumanoid-V2 project page and paper; assets retain their owners' rights.

VLA models	Whole-body control	PICO tracking	GoPro interface

---

Contact

Questions, ideas, or interested in collaborating? We'd love to hear from you—feel free to connect with Jin Chen on WeChat!