English | 日本語
A step-by-step series of Python samples for learning robot-arm pick-and-place (grasping an object and putting it somewhere else) in NVIDIA Isaac Sim. It starts with moving to known coordinates, adds camera-based detection and multi-object handling, and ends by handing the whole motion to a policy trained with reinforcement learning (PPO).
These samples are for education and simulation. They do not guarantee safety on real hardware.
| No. | Topic | What it does | What you learn |
|---|---|---|---|
| 01 | Simple picking | Moves a cube whose position is known | Inverse kinematics (IK) and a state machine |
| 02 | Camera picking | Finds the cube with an RGB-D camera, then moves it | Colour detection and image-to-3D back-projection |
| 03 | Multi-object picking | Finds several objects and lines them up one by one | Connected-component splitting and a task queue |
| 04 | PPO robot control | Moves the cube as instructed, e.g. "right 20 cm" | Policy learning with reinforcement learning (PPO) |
Each part builds on the previous one, so reading them in order from #01 is recommended.
#04 turns instructions such as "right 20 cm", "left 10 cm" or "back 15 cm" into a goal position, and a single PPO policy (a neural network that maps observations to actions) performs the whole task: approach, grasp, lift, carry, release and retreat. No demonstration data (imitation learning) and no human intervention are used. Only the conversion from the policy's end-effector motion to joint angles is done by a low-level IK.
| Evaluation condition | Successes / trials | Success rate |
|---|---|---|
| All directions, 5–30 cm, maximum start-pose randomisation (latest model) | 224 / 1,024 | 21.88% |
| Fixed near-range position only (earlier model) | 250 / 256 | 97.66% |
- A success is a trial that completes every stage and then holds the result for 2 seconds.
- Evaluation used four random seeds that were never used in training.
- The 90% target has not been reached yet. 97.66% belongs to a different model under a much narrower condition; it is not the latest model's score.
Because PPO improves its policy through trial and error, more training with a suitable curriculum (a schedule that starts easy and gradually gets harder) can be expected to raise the success rate. It does not rise with every update, though — it can dip or plateau — so progress is checked by separate evaluation on unseen seeds.
- NVIDIA Isaac Sim 6.0.1
- CUDA-enabled PyTorch (#04 only)
- Tested on: Windows 11 / RTX 5070 Ti 16 GB / Core Ultra 7 265K / 64 GB RAM
- Prepare the robot USD file. It contains third-party assets, so it is not in Git. Place it yourself by following assets/README.md.
- From the repository root, run a sample with the Python bundled with Isaac Sim:
C:\isaacsim\python.bat ".\01_simple_picking\simple_picking.py"Replace C:\isaacsim with your Isaac Sim install location. The saved #04 models only
run with exactly the same USD used for training (matching SHA-256 hash).
Open_Physical_AI/
├── 01_simple_picking/ # #01 picking at known coordinates
├── 02_camera_picking/ # #02 picking with camera detection
├── 03_multi_picking/ # #03 sequential multi-object picking
├── 04_rl_control/ # #04 control with PPO
│ ├── ppo/ # environment, policy, training, evaluation
│ ├── models/ # published trained models
│ ├── results/ # final evaluation and sample video
│ ├── docs/ # technical notes and article draft (Japanese)
│ ├── tests/ # unit tests that need no simulator
│ ├── ppo_pick_place.py
│ ├── run_ppo_command.py
│ └── train_ppo_until_target.py
├── assets/ # where the robot USD goes (bring your own)
└── docs/publishing.md # pre-publication checklist (Japanese)
The code is under the MIT License. Robot models, trademarks and assets shown in videos are subject to their respective owners' terms.