Autoregressive Vision-Language-Action (VLA) models offer a scalable path to robot learning, yet existing action tokenizers treat tokenization as a compression problem, producing representations that are semantically misaligned with the autoregressive backbone. We propose CATok, a causal action tokenizer that reframes tokenization as a causally structured generative process.

CATok introduces a conditional annealing mechanism that extracts action tokens by progressively annealing a flow-matching process: each token is conditioned on all preceding tokens and encodes the residual reconstruction signal at a specific noise level, establishing a coarse-to-fine causal token space whose generative semantics are structurally aligned with autoregressive modeling. A token-conditioned flow-matching decoder built on Multimodal Diffusion Transformer (MMDiT) reconstructs continuous action chunks from these discrete tokens with the precision of hybrid diffusion-head architectures. This discrete bottleneck enforces knowledge insulation by design, cleanly separating high-level semantic reasoning from low-level motor execution without requiring explicit attention masking.

Extensive evaluations across three simulation benchmarks demonstrate that CATok consistently surpasses existing tokenization methods in reconstruction fidelity–compression tradeoff and inference efficiency, while improving VLA task success rate, establishing a high-performance, scalable foundation for purely autoregressive VLA systems.

CATok consists of three components: (1) a Dual-Stream Action Encoder that maps raw action chunks into a compact set of latent query tokens via co-attention with an MMDiT-based transformer; (2) a Bottleneck Vector Quantizer that discretizes latent tokens into a finite codebook using factorized codes; and (3) a Flow-Matching Decoder with Conditional Annealing that reconstructs high-fidelity action trajectories while enforcing a coarse-to-fine causal structure.

The key design is the conditional annealing mechanism: at flow-matching timestep t, the first ⌊tK⌋ token embeddings are progressively masked, forcing each token to specialize in a distinct generative stage—from coarse global guidance at high-noise stages to fine-grained residual corrections as the trajectory matures. This induces a causally ordered hierarchy naturally aligned with autoregressive action generation.

We evaluate CATok on three simulation benchmarks—LIBERO, SimplerEnv (Simpler-Bridge), and RoboTwin 2.0—against three representative tokenizer baselines: BIN (uniform per-dimension discretization), FAST (DCT-based compression with BPE tokenization), and OAT (learned fixed-length tokenizer with prefix-based decoding).

Higher task success rate. CATok achieves the highest success rates across all benchmarks (95.9% avg. on LIBERO), with the largest gains on long-horizon tasks (LIBERO-Long: 91.0% vs. 90.1% for FAST). Causal token ordering enables autoregressive policies to model long-range temporal dependencies more effectively.

Faster inference. CATok delivers 1.7× faster VLA inference than FAST and 3.9× faster than BIN, enabling more responsive real-time deployment.

Better fidelity–compression balance. CATok achieves a 3.6× stronger VRR×CR score than FAST at the same token count, demonstrating that causal structure improves reconstruction quality without sacrificing compression.

Higher training efficiency. CATok consistently reaches higher success rates at the same training step count, attributable to its stage-wise token factorization that makes each token prediction more informative for trajectory generation.

Causal Predictive Ordering. Per-token Shannon entropy decreases monotonically in the forward token order (Figure 6(a)), while reversing the order flips the trend—confirming the learned ordering is directional and causal. FAST and BIN show no such structure.

Stage-wise Generative Semantics. t-SNE of VQ embeddings (Figure 6(b)) reveals clear slot-dependent geometry: early tokens form smooth elongated regions, late tokens form compact separated clusters. Prefix reconstructions (Figure 6(c)) show coherent coarse-to-fine trajectory updates, confirming each token contributes a distinct generative increment.

We use two native-decoder interventions—a donor-swap at the velocity level and single-token zeroing at the trajectory level—to characterize the generative roles of individual token positions. Under conditional annealing, token k is bound to flow stage k via suffix masking, so as flow matching refines the trajectory from high- to low-noise stages, each token is trained to contribute at a particular refinement stage. We examine whether the learned tokens acquire distinct, flow-stage-aligned effects and a coarse-to-fine functional progression in action space; here, “coarse-to-fine” denotes the granularity of learned control information along the refinement process, not a prescribed frequency or fixed-basis decomposition.

We use the trained CATok tokenizer checkpoint (K=32, codebook 4096, horizon 20), its MMDiT velocity network v_θ, and its ODE decoder D—the same modules used at inference, not a separately trained prefix decoder. We encode 128 held-out Franka action chunks (7-DoF + gripper) as Q = (q₁, …, q_K). For each token k, we replace its embedding with the same-position embedding from a donor chunk—keeping the intervention within the slot’s empirical distribution—while holding x_t, t, and the noise realization fixed, so the resulting change isolates token k’s effect on the velocity prediction. For each flow time t we measure Δ_k(t) = MSE(v_θ(x_t, t, Q^swap(k)), x₁−x₀) − MSE(v_θ(x_t, t, Q), x₁−x₀), where MSE is ℓ. Because conditional annealing masks token k outside its active range, we normalize its effect within that range.

The influence matrix exhibits two properties:

• Schedule-consistent support. Token k is visible only for t < (k+1)/K, consistent with the suffix-masking schedule κ(t) = tK.
• Stage-specific peaks. Within its active range, each token’s peak influence occurs near its assigned stage, t* ≈ k/K, with corr(k, t*) = 0.985 (0.995 after grouping). Early tokens peak at high-noise (coarse) stages and late tokens at low-noise (fine) stages, so token effects are distinct and aligned with their assigned flow stages rather than interchangeable.

We next characterize how this stage specialization maps onto action dimensions. We zero each token embedding in turn, decode the intervened sequence with the native ODE decoder at 1 step (matching the deployed VLA’s flow_steps=1), and measure the resulting per-dimension trajectory change.

Taken together, the donor-swap and zeroing results show that conditional annealing induces a learned coarse-to-fine refinement process: token effects are distinct and aligned with their assigned flow stages, and their action-dimension footprint progresses from coarse gripper/task-state information to fine joint/trajectory-detail information.

To verify that CATok transfers to real-robot deployment under camera noise, lighting variation, and hardware differences, we design a suite of real-robot evaluation tasks. The VLA built on CATok as its tokenizer consistently outperforms the QwenFAST baseline, with significant gains in spatial generalization, color generalization, and long-horizon task completion.

We collect 362 trajectories of "pick up the yellow cup and place it on the plate." and 220 trajectories of "stack the cups." Using Qwen3.5-VL as the backbone, we train two VLAs that differ only in their tokenizer—QwenFAST and QwenCATok. CATok is pretrained on the collected real-robot data together with InternA1 joint data, so that its action reconstruction accuracy matches that of FAST. We evaluate on three task suites:

Figure 9. Scene layouts for the three real-robot task suites. Left to right: Pick-Spatial, Pick-Color, Stack-Long.

• Pick-Spatial. The task is to pick up the yellow cup and place it on the plate. We fix 10 relative cup-and-plate positions; each model rolls out twice per position (20 rollouts total) and we report the success rate. Tests spatial generalization.
• Pick-Color. The task is to pick up the red cup and place it on the plate. The cup-plate relative position is fixed, and only the yellow cup seen during training is replaced with a red cup; each model rolls out 10 times. The training set contains no trajectories of picking up a red cup. Tests color generalization.
• Stack-Long. A long-horizon task: the robot must first pick up the red cup and stack it onto the yellow cup, then pick up the pink cup and stack it again. The relative positions of the three cups are fixed; each model rolls out 40 times. Tests long-horizon task completion.

We deploy QwenCATok (our causal action tokenizer integrated into a VLA policy) alongside the QwenFAST baseline on a real Franka manipulator across three contact-rich tasks—pick, pick-red, and stack. Each cell below is a single, unedited end-to-end rollout.

These real-world gains corroborate the mechanisms identified in simulation. CATok's causally structured token space—each token encoding a semantically distinct, coarse-to-fine generative stage—aligns the discrete bottleneck with the autoregressive backbone, and this semantic alignment generalizes to real-world perturbations: spatial repositioning (Pick-Spatial, 20 rollouts, 0.45→0.65, +44% relative) and out-of-distribution color (Pick-Color, 10 rollouts, 0.30→0.50, +67% relative). The largest gain appears on Stack-Long (40 rollouts, 0.275→0.475, +73% relative), consistent with the same stage-wise causal factorization that drives training efficiency in simulation—by making each token prediction more informative, causal token ordering lets the policy better capture the long-range temporal dependencies required for compositional, long-horizon tasks. Overall, CATok's structural properties—semantic token alignment and stage-wise causal factorization—translate from simulation into robust and consistent real-world gains.