Vision-language-action (VLA) models let robots carry out tasks from camera images and instructions in plain language. The computation that these models need varies between tasks, and even within a single inference. We design hardware and algorithms that exploit this variability to run VLA models on the robot, in real time and within its energy budget.
Research goal: Robots today run billion-parameter VLA models on edge GPUs such as NVIDIA's Jetson AGX Thor, which draws up to 130W but runs NVIDIA's GR00T N1.6 at about 11 Hz. Our goal is hardware that adapts its computation to the task, to run VLA models at the control rate of up to 50 Hz within a few watts.
Gap in the SotA: Deployed VLA models spend the same computation on every inference, regardless of what the task requires. Some algorithms adapt the computation of each inference at run time, but they run on GPUs. On a GPU, an inference that needs less work is not proportionally faster or more efficient, because fixed overheads remain.
Recent results:
Weight precision that rises over the denoising steps. Because the action head of a VLA's Diffusion Transformer generates almost the same action regardless of its initialisation noise, its later denoising steps attenuate the errors of the earlier ones. The weight precision can therefore rise over the steps, whereas image generators amplify early errors and need the opposite order. With plain per-channel rounding and without retraining, 2, 3, 3 and 4 bits in the four steps preserve the task success of GR00T N1.6, GR00T N1.5 and π0.5. To exploit this per-step precision in the memory-bound action head, we designed a programmable accelerator that decodes entropy-coded weights at the DRAM rate and processes them bit-serially. With 50% of the memory bandwidth of Jetson AGX Thor, the accelerator runs the GR00T N1.6 action head 1.7× faster than Thor with 4-bit NVFP4 weights. At equal task success, it also needs 9.1× less energy per inference.
Robin Geens, Simon Haepers, Jonas Crols, Chao Fang, Marian Verhelst, “Hardware Acceleration of Vision-Language-Action Models with Weight Precision Rising over the Flow Steps,” arXiv preprint arXiv:2611.24234, 2026. arXiv:2611.24234