Stage-1 gate — does the action-motion encoder transfer to Cosmos?
tokenizer cosmos-cv4x8x8 · generated 2026-07-31 07:55:33Z
NO TRANSFER the encoder learns well (D beats zero by 32.3%, C by 33.0%), but the action map adds NOTHING over tactile-only (D is 1.0% worse than C; Wan2.2 had D +3.3% over C). The conditioning signal Stages 2-3 inject is redundant here.
The encoder predicts per-latent-transition tactile flow. Active-pixel EPE is the
endpoint error over pixels whose ground-truth motion exceeds the active threshold;
lower is better. The zero-motion baseline on this val split is
1.2195 (Wan2.2: 1.612).
An ablation is only meaningful if it beats that.
| ablation | inputs | EPE (Cosmos) |
EPE (Wan2.2) | Δ | trained |
|---|
| D (z_ctx, a) -> F | action map + tactile state | 0.8254 | 1.130 | -0.305 | 12 ep |
| B action only | action map only | 1.0553 | 1.362 | -0.307 | 12 ep |
| C tactile only | tactile state only | 0.8170 | 1.168 | -0.351 | 12 ep |
Δ is context, not a result. The two columns are not measured over the same
interval: a Wan2.2 latent transition spans 5 frame pairs at 6 fps (0.83 s),
a Cosmos one spans 4 pairs at 30 fps (0.13 s) — 6.2× less motion
time — on top of a different tokenizer and dataset. The magnitudes happen to
land in a similar range because gel deformation saturates rather than accumulating,
but that is a coincidence of contact mechanics, not a controlled comparison.
Judge the gate on the ordering (D < C < B < zero) and on D's
relative margin over zero, both measured entirely within the Cosmos column.
Seeds. 3 seeds per ablation. The D-vs-C increment is -0.9% ± 0.6% (std). The sign is stable across seeds.
What the two ablations actually see
D's input is a strict superset of C's: the same tactile latent, plus the action
map. D − C is therefore exactly what knowing the sensor's own motion buys
you beyond looking at the gel — the signal Stages 2-3 inject.
The action map (purple) is a near-constant field: 99.5% of its energy is
global translation, so it carries about one 2-D vector. The AllTracker target
(red) is only 33% global translation — two thirds is spatial structure a constant
field cannot express. That is why B (action only) is far behind, and why D adds
nothing C does not already have.
Cosmos vs Wan2.2
The ordering broadly agrees — action-only weakest, tactile dominant. What differs
is the increment the recipe rests on. Note D did not get worse on Cosmos
(29.9 → 32.3): C got better (27.5 → 33.0). The Cosmos tactile
latent already carries more of the motion, leaving no headroom. And the increment
was only +3.3% on Wan2.2 to begin with.
Raw metrics: data.json