OxyGen
NeurIPS 2026 · Main Track arXiv 2603.14371

OxyGen

Unified KV Cache Management for VLA Inference under Multi-Task Parallelism

TL;DRLet robots talk while they act.

OxyGen shares what both tasks have already computed, letting language unfold across action updates instead of holding up the next move.

Isolated: language blocks action generation
OxyGen: language resumes across frames in batches
action generationlanguage generation

π0.5 · actions & language

3.7×

faster in both outputs

19→71Hz

action frequency (H=10)

57→213tok/s

language throughput

Speedup across backbones

5.83×

Xiaomi-Robotics-0

6.64×

StarVLA

§ 1 · Demo

Same task. Less waiting.

LIBERO examples
OxyGenInference — · End-to-end 18.7 s
Isolated baselineInference — · End-to-end 25.1 s
Loading video…
Loading replay…Inference 0.00 s
OUTPUT: —
Loading replay…Inference 0.00 s
OUTPUT: —
0.00 / 25.08 s Drag to compare at any moment
Action inferenceLanguage inferenceExecution
OxyGenBaseline

The baseline finishes each language response before moving. OxyGen carries language generation across action updates.

§ 2 · Method

One cache. Across tasks and time.

Share the observation.
Batch language across frames.

How OxyGen manages the KV cache
i. One prefill, shared across experts
frame t observation 3 views, prompt VLM prefill once per frame prefix KV shared, read-only Action expert S denoising steps → H actions New language request shared prefix · private suffix
prefill prefix KV action chunk language tokens
ii. One batch, shared across frames
frame t t+1 t+2 t+3 t+4 t+5 actions batch of 3 request t ✓ t+1 ✓ t+2 ✓ t+3 ✓ t+4 t+5
4 tokens / request / frame✓ completed request
KV sharing helps; batching delivers the larger latency reduction
Isolated
1 request · 30 tokens
524 ms
+ KV sharing
1 request · 30 tokens
469 ms
+ cross-frame batching
6 requests × 5 tokens
176 ms
observation prefill action denoising language decoding
Sharing KV removes redundant prefill; batching amortizes decoding across unfinished requests.

§ 3 · Results

Faster actions and more language.

Measurements on desktop and on-board GPUs, followed by transfer across backbones and model sizes.

Action frequency and language throughput rise together.
3.7×
020406080100 60120180240 Action frequency (Hz) → ↑ Language throughput (tok/s) 30 tokens Baseline Parallel k = 10 k = 5 OxyGen, k = 1
01020304050 255075100 Action frequency (Hz) → ↑ Language throughput (tok/s) 30 tokens Baseline Parallel k = 10 k = 5 OxyGen, k = 1
Batching keeps actions fast as language grows.
3.0×
255075 510152030 Tokens per language request → ↑ Action frequency (Hz) KV only batched
102030 510152030 Tokens per language request → ↑ Action frequency (Hz) KV only batched
Isolated baselineParallel (MPS)+ KV sharing+ batching = OxyGen one frame of decoding only
Speedups transfer across VLA architectures, expert counts and backbone sizes.
Configurationk = 1 / k = 5 · 0–8×
Across architecturesSharing and batching transfer beyond π0.5.
π0.5
3.71× / 2.98×
Xiaomi-Robotics-0
5.83× / 3.23×
StarVLAQwen3-VL-4B + PI_v3
6.64× / 3.60×
More expertsSpeedup holds as experts increase.
+ GR00T3 experts
6.05× / 3.53×
+ GR00T + OFT4 experts
6.30× / 3.69×
Across backbone sizesThe gain persists as the backbone grows.
Qwen3.5-0.8B
6.30× / 3.58×
Qwen3.5-2B
6.31× / 3.62×
Qwen3.5-4B
6.55× / 3.71×
Qwen3.5-9B
7.18× / 3.79×
k = 1 k = 5 isolated baseline · 1×
OxyGen leaves the action policy unchanged.
96.76%LIBERO success, matching openpi (96.85%).
Batching cuts energy per request by 78%.
−78%energy per request · −47% power · +15% memory.

§ 4 · On-board measurement

Lower latency, on board.

2.6× faster inference on the on-board Jetson AGX Thor, with room for language during motion.

Humanoid robot with an on-board NVIDIA Jetson AGX Thor
Faster inference leaves room for language during motion.2.6×
Isolated baseline
dispatch
1030 ms
OxyGen
dispatch
393 ms
02004006008001000
ms
action generationlanguage generationMotion window · 333 ms

§ 5 · Resources

Read it, run it, build on it.

Citation
@article{li2026oxygen,
title={OxyGen: Unified KV Cache Management for VLA Inference under Multi-Task Parallelism},
author={Li, Xiangyu and Tang, Huaizhi and Ding, Xin and Wang, Weijun and Cao, Ting and Liu, Yunxin},
journal={arXiv preprint arXiv:2603.14371},
year={2026}
}