sft_robodojo_vanilla34k_eefabs_f33fps8_openwam_aligned

Model weights only of the RoboDojo ARX-X5 EEF-only policy SFT on the OpenWAM canvas (H 384 x W 320) with strided video (video_fps 8), absolute EEF targets and the aligned RoPE, warm-started from the vanilla robot pretrain logits/sana_rwm_pretrained_vanilla_e6s34500 (wandb lzknus/sana-rwm/sft_robodojo_eefabs_openwam_384x320_vanilla34k_aligned_f33fps8), at checkpoint epoch_10_step_36180 (epoch 10, optimizer step 36,180 = the full 36,180-step budget, 10 epochs). Final checkpoint of the finished run (Slurm array 19288229, training reached step 36,180 on 2026-09-25 PDT; uploaded 2026-09-26). Its canvas twin on the same donor and recipe is logits/sft_robodojo_vanilla34k_eefabs_f33fps8_sana_pixel_aligned.

Files

file	bytes	sha256
model/pytorch_model_fsdp.bin	17874895202	312c9c102dc945ae2237ecf09d18eeef0579cccd0f55278143b794d9fa0f1f19
metadata.pth	37205	7234c5490c3465caf27e3556d3bbecb8028ba28c7481933a37efe50c30b9201b
config.yaml	12156	5bc8f7c003477e5f33375ca4504fd952f5f119637413a01bf68980738bf8b634
normalization/robodojo_arx_x5_model_fps_25_f33_normalization.json	45608	1fe3b7e72e8fa93eca5efb8a8deeace1daf4434b4b97e9f4445670ac758287ed

• model/pytorch_model_fsdp.bin: accelerate FSDP consolidated state dict, 805 tensors, 4,468,977,840 parameters (804 float32, 1 bfloat16). The eight robot-module tensors (state_embed.proj.*, action_embed.proj.*, action_head.*, plucker_embed.weight) came with the robot-pretrained donor and were fine-tuned here (the donor loaded strictly, missing only pos_embed); pos_embed present.
• metadata.pth: epoch / step / scheduler / RNG bookkeeping read next to the weights.
• config.yaml: the trainer's frozen, fully resolved training config, unchanged (paths are cluster-local). It already uses the current data.extra.robot_sft block and declares model.extra.rope: aligned; it parses with rwm/zekai-merge b86f8a2f9.
• normalization/...json: the robot80 normalization artifact the run trained with (sha256 1fe3b7e72e8fa93eca5efb8a8deeace1daf4434b4b97e9f4445670ac758287ed; f33, absolute targets, frame-aligned corpus). Grippers are q01/q99-normalized too (closedness [0, 1] -> [-1, 1]).
• Not included: model/optimizer.bin, model/scheduler.bin, random_states_*.pkl, the training log (weights only).

Recipe (from config.yaml)

• Model SanaRWMVideoQwenNextSubAttnResV2SelfFlowWorldModelCameraConditionMultiViewPolicy_5B_P1_D36 (32 blocks, softmax attention every 4th, GatedDeltaNet elsewhere), bf16, fp32 attention; data.extra.multiview: openwam.
• Donor model.load_from: sana_rwm_pretrained_vanilla_e6s34500 (Hub logits/sana_rwm_pretrained_vanilla_e6s34500: the 256px vanilla absolute EEF/joint unified robot pretrain at 7.5 fps, epoch 6 step 34500).
• Visual stream: ONE OpenWAM canvas per frame. The three cameras are composited in pixel space into the OpenWAM L-shape RGB canvas of H 384 x W 320 (openwam_canvas.layout openwam_lshape_rgb_v1, encode_mode joint_rgb_canvas, aspect_ratio_type ASPECT_RATIO_OPENWAM_LSHAPE_384_320) BEFORE the LTX-2.3 VAE, so the policy sees one view (V = 1, plain mRoPE) on a 12x10 latent grid. No camera conditioning. Latents were read from the verified stores openwam-rgb-v1-f33-vs4-a347d771ff728190 + tail openwam-rgb-v1-f33-vs4-tail-878f0297a2567b82.
• Windows: 33 source rows at video_fps 8 (frame stride 4): the observation frame plus 8 sampled frames = 9 canvas frames = 2 latent frames, while the actions stay dense: 32 action rows at 25 fps.
• RoPE: aligned (model.extra.rope). Video and actions share one physical clock in base-fps (16) latent-frame units: video latent j at 16 * j * 4 / 25 (0 and 2.56), action row k at 16 * k / (8 * 25) = 0.08 k (0.08 .. 2.56), the state at 0.
• Targets: EEF-only (action_mode_sample_ratio [0.0, 1.0, 0.0], robot_base_eef: both arms' EEF position + Rot6D in the robot base frame plus the grippers), eef_target_mode absolute. Normalization pin 1fe3b7e7....
• Data contract: frame-aligned. Video frame r is paired with source row r-1 (the official RoboDojo one-frame shift), state rows are the measured state of the current frame. All 3,500 episodes (holdout_episodes_per_task_split 0), tail windows (min_rows 2, padding freeze): 1,744,102 full + 108,500 tail = 1,852,602 windows, 3,618 steps per epoch at 512 windows per step (8 nodes x 8 GPUs x bs 8).
• Text contract: G = 1, ONE shared prompt (the composite view's: embodiment, action mode, the canvas layout, instruction); instruction dropout 0.1 per scene (the unconditional row drops the Instruction line).
• Noise schedule (the 2026-09-24 SFT default, rwm/yuyang's): flow shift 5.0 standard for the video, a separate action flow shift 1.0 on the same raw timestep draw, inference 5.0 / 1.0, OpenWAM timestep loss weighting with min_weight 0.1, min_train_timestep 1.
• Optimizer: AdamW peak lr 0.0001 after 2,000 warmup steps, cosine to 1e-06 over 36,180 steps, weight decay 0.0001 on weight matrices only, grad clip 1.0, action_loss_weight 1.0.

Validation (seen-episode monitor: 35 tasks x 1 episode the model trained on; normalized masked action MSE)

Milestones validated by the run's watcher from the training code (rwm/zekai-merge 52f3db1f0, sampler without the old gripper clamp, CFG off, 50 steps):

ckpt step	n	mean	median	max	tasks > 0.2
20,000	35	0.0043	0.0003	0.0570	0
25,000	35	0.0026	0.0001	0.0663	0
30,000	35	0.0027	0.0002	0.0608	0
35,000	35	0.0023	0.0001	0.0626	0
36,180	35	0.0022	0.0001	0.0613	0

Steps 5,000 / 10,000 / 15,000 have no row: the milestone watcher waited in the queue until about step 19,000 and those rolling checkpoints had already been rotated out. The final checkpoint was validated again from rwm/zekai-merge b86f8a2f9 with this config.yaml: 35/35 samples bitwise equal to the watcher's (predictions, targets, masks and MSE).

Loading

--model.load_from=<local dir holding model/ and metadata.pth> for the Sana-RWM trainers and validator on rwm/zekai-merge 875b3f659 or later (the config's data.extra.robot_sft block); the bidirectional deploy takes the same directory with config.yaml and normalization/...f33_normalization.json. The model output is the normalized action; saturate the gripper closedness to [0, 1] after denormalizing, not before.