Event - 16 April 2027

Exploiting Per-Step Weight Precision in Robot Action Heads

Lectured by Robin Geens

What

Robots are increasingly driven by vision-language-action (VLA) models such as GR00T, whose action head is a diffusion transformer that turns noise into motor commands over a few flow-matching steps. Each step reloads all of the weights from DRAM to process only about fifty tokens. That makes it memory-bound: latency and energy come down to how many weight bits you move. I discovered that later flow steps clean up the quantization errors of earlier ones, so earlier steps tolerate much more quantization. Going from 2 bits in the first step to 4 bits in the last, with plain rounding and no retraining, keeps task success intact and moves 25% fewer bits. GPUs can't exploit this, because fixed kernel overheads swallow the savings. To exploit this rising sensitivity, I developed an accelerator architecture that keeps its DRAM link saturated at any bit width: it decodes entropy-coded weights into bit-serial sub-arrays and re-divides those sub-arrays between the linear layers and attention at every step. It also derives every bit width from a single stored 8-bit copy, so it doesn't need a seperate copy of the weights per width. The result beats Jetson AGX Thor in both speed and energy, with half its memory bandwidth.

When

16/4/2027 11:00 - 12:00

Where

ESAT Aula R