TL;DR: I ran a frozen `pi05_libero` on arms it was never trained on. UR5e and the I2RT YAM both go 20/20, same as the Panda it was fine-tuned on, as long as the wrist camera sits in the same place relative to the fingers and the gripper is the same. Swap only the gripper on the Panda and it falls to 14/20 on one task and 3/20 on another. The checkpoint ignores proprioception completely, so fixing the state vector does nothing. What fixes it is putting the Panda gripper’s look back onto the Rethink gripper’s physics: 19/20 and 18/20. The policy reads its gripper from the wrist camera. A gripper swap is a visual shift and nothing else.
If we want to run released VLAs, or other labs’ data, on our own arm, what actually has to match? The fine-tuned checkpoints people download were trained on one arm, one gripper, one wrist camera. I wanted to know which of the three the policy leans on.
1. Setup
Frozen `pi05_libero`, LIBERO-Object, the official OpenPI LIBERO stack. Inputs are a third-person image and a wrist image at 224, plus an 8-d state (EE pose and two finger positions). Output is EE deltas and a gripper command at 20 Hz through robosuite’s OSC controller, so joint kinematics never touch the policy.
Every arm runs the same 20 init states. The wrist camera is placed so its transform to the grip point matches the Panda’s, the grip point at reset is matched by IK, and sampling is seeded per reset. Only the embodiment differs. With 20 episodes per arm I can only see big effects, so I count the resets that flip between arms rather than quoting percentages.
2. Arm swaps
| Arm | Gripper | In π0 pretraining? | Success |
|---|---|---|---|
| Panda | Panda | yes, fine-tune target | 20/20 |
| UR5e | Panda | yes | 20/20 |
| I2RT YAM | Panda | no | 20/20 |
| Panda | Rethink | gripper yes (Sawyer, OXE) | 13/20 |
YAM has six joints, about two thirds of the Panda’s reach, and had to sit on a pedestal to reach the table. It looks nothing like a Panda in the third-person view. It takes longer (143 vs 129 steps median) and never fails. UR5e is in the π0 fleet so that one is not a clean test. YAM is.
The Rethink gripper is in the pretraining data and still loses 7 resets, never the other way. A scripted grasp with it works on all 7, so the gripper is fine. What you see in the failures is the gripper opening and closing 12 to 29 times, dropping and regrasping over the basket until time runs out. The Panda does it in one or two.
3. Ruling out proprio
The obvious guess: Rethink opens to 58 mm, Panda to 76 mm, so “open” reads as half closed in the state vector. Except `pi05_libero` never reads the state. `Pi0Config(pi05=True, discrete_state_input=False)` skips the state projection and does not tokenise it either. I checked on the served model: same images, any change to the state (finger positions, EE moved 10 cm, all zeros) changes the action by exactly 0.0. Black out the wrist image and it changes by 2.0. `pi0_libero` does use state, so I ran the same test on both.
| Arm | π0.5 | π0 |
|---|---|---|
| Panda | 20/20 | 18/20 |
| Panda, proprio squashed into the Rethink range | 19/20 (identical actions) | 17/20 |
| Rethink | 14/20 | 10/20 |
| Rethink, proprio stretched to the Panda range | 12/20 | 12/20 |
Fixing the proprio fixes nothing, even for the model that reads it. I had predicted the opposite.
4. Pixels vs physics
Two hybrid grippers: one gripper’s physics (joints, collision geoms, actuators) with the other gripper’s visual meshes.
The match is rough. The Panda-look fingers on the Rethink sit further back and smaller than the real Panda’s, and it is still enough.
| Arm | π0.5 | π0 |
|---|---|---|
| Panda | 20/20 | 18/20 |
| Rethink | 14/20 | 10/20 |
| Rethink physics, Panda look | 19/20 | 14/20 |
| Panda physics, Rethink look | 0/20 | 0/20 |
| Panda physics, Rethink look, camera moved to the Rethink distance | 12/20 | not run |
Rethink physics dressed as a Panda is back to Panda numbers: 19/20, one close per grasp, and six of the resets the Rethink lost now pass. Panda physics dressed as a Rethink is dead. It closes at step 0 and never goes down, as if something were already in the gripper. That 0/20 is exaggerated because the Rethink shell sits 12 mm closer to the camera than in the real swap. Move the camera to match and it is 12/20, roughly what the real swap costs. The look alone accounts for the swap. π0 only comes half way back (10 to 14) because it also reads the state and the aperture really is different.
5. Second task
Alphabet soup, same protocol:
| Arm | Success |
|---|---|
| Panda | 20/20 |
| Rethink | 3/20 |
| Rethink physics, Panda look | 18/20 |
Of the 17 resets the Rethink lost, 16 come back.
6. Masking does not work
Cloak hides the gripper in the wrist view during training. Doing it at inference on the frozen model gives Panda 15/20 and Rethink 9/20. The policy needs to see the fingers. You cannot hide them from it after the fact.
7. What this means for us
For this checkpoint, on these tasks, the arm does not matter: kinematics, link count, colours, whether it was in pretraining. What matters is where the wrist camera sits relative to the fingers, EE-space control, and what the gripper looks like in that camera. Remapping proprio does not help and masking does not help. Making the new gripper look like the old one does.
For YAM: match the wrist camera mount, then either use the same gripper or make YAM’s gripper look like the source one where the camera sees it, with paint, a printed shell, or Mirage-style cross-painting at inference. That is cheap to try on the real robot and is the next step.
Related work: Cloak, Mirage, KITE, CEI, EmbodiSteer. All train or paint their way around the gripper. None checks which channel a frozen released checkpoint reads it through.
Caveat. Two tasks, one checkpoint per family, sim, 20 episodes per arm. The hybrid finger meshes are scaled approximations. Two simulator bugs bit during the rebuild (MuJoCo 3.4 renders the Rethink meshes black in the LIBERO scene; two LIBERO envs in one process corrupt the first one’s renders) and every affected run was thrown out. This does not say the policy survives a real camera or a real gripper. It says which one to protect.
Next: a third model with and without proprio (OpenVLA-OFT), more tasks, and YAM with its own gripper on hardware.