bimanual-widowxai-pour-items-red-cup-pi05

A π₀.₅ (pi0.5) policy LoRA fine-tuned on Trossen Cloud for the pour items from red cup task on the Trossen WidowX AI bimanual stationary platform.

Framework: LeRobot (PyTorch) — the LoRA adapter is already merged into the base. Load this directly; there is no adapter to apply and no base checkpoint to fetch.

Prompt matters: pi0.5 conditions on the task string. Use it verbatim — a paraphrase is a different model input.

Pour the items from the red cup into the empty container

Inputs

• Dataset: TrossenRobotics/bimanual-widowxai-pour-items-red-cup-v3 — LeRobot v3 format, 100 episodes @ 30 fps.
• Base model: π₀.₅ (pi0.5) pretrained checkpoint, LoRA fine-tuned (Trossen Cloud base a8f3c67b-518f-4e42-9ed7-6420fb373ca6).
• Training venue: Trossen Cloud, job 16e0255d-71a5-45f9-8288-ad972f2e1f6d.
• Hardware: Trossen WidowX AI Stationary (bimanual), bi_widowxai_follower_robot.
• Action/state space: 14-dim (7 per arm).
• Cameras: 3 — cam_high, cam_left_wrist, cam_right_wrist (no cam_low), 3 × 480 × 640.

Configuration

• Steps: 30,000
• Fine-tuning: LoRA — r=32, alpha=32, dropout 0.0
• Also trained: state_proj, action_in_proj, action_out_proj, time_mlp_in, time_mlp_out
• Chunk size / action steps: 50 / 50
• Relative actions: False (absolute)
• Precision: bfloat16 (merged)
• Batch size, learning rate and schedule were not recorded at publication time.

How it was trained

LoRA fine-tuned on Trossen Cloud over 100 episodes of bimanual pouring demonstrations, 30,000 steps. The run produced five checkpoints — steps 6000, 12000, 18000, 24000 and 30000 — of which only step 30000 was evaluated on hardware, so it is the best known checkpoint rather than a measured optimum.

Trossen Cloud emits the stock fp32 base and the 221 MB LoRA adapter separately. The published weights were produced by merging them with PEFT's merge_and_unload() on an RTX 5090, casting to bfloat16, and saving a single model.safetensors with the adapter-level config, both processors and the tokenizer copied in alongside.

How to use

from lerobot.policies.pi05.modeling_pi05 import PI05Policy

policy = PI05Policy.from_pretrained("TrossenRoboticsCommunity/bimanual-widowxai-pour-items-red-cup-pi05")
# task string must be exactly:
#   "Pour the items from the red cup into the empty container"

Camera keys and their order must match config.json exactly — the order there is cam_left_wrist, cam_high, cam_right_wrist. A mismatch loads without error and then emits meaningless actions. Camera keys are labels, not mounts: cam_high must be fed by a camera in the position it occupied during collection.

The processor .safetensors carry the normalisation statistics. Without them the policy initialises and emits unnormalised output.

Written under Trossen Cloud's lerobot 0.5.1-era processor naming; if a newer lerobot rejects a name in policy_preprocessor.json, check the 0.5.1 → 0.6.x renames.

Outputs

model.safetensors                                          8.29 GB   merged, bfloat16
config.json
policy_preprocessor.json  + policy_preprocessor_step_2_normalizer_processor.safetensors
policy_postprocessor.json + policy_postprocessor_step_0_unnormalizer_processor.safetensors
tokenizer/                                                           google/paligemma-3b-pt-224

• Best checkpoint: step 30,000
• Checkpoint trained: 2026-07-25
• Merged and published: 2026-09-25

Evaluation

Evaluated on the real robot — the bimanual WidowX AI stationary rig — and reported by the operator as performing the pouring task well. No success-rate count over a fixed number of trials was recorded, so treat this as a qualitative result rather than a measured rate.

Match the scene and camera geometry to the training footage before any rollout.

Versions

• r1 (2026-09-25) — initial publication; merged bf16 checkpoint from Trossen Cloud job 16e0255d, step 30,000.