Adaptive hardware for embodied AI

Robin Geens and Jonas Crols , Marian Verhelst Hardware-efficient AI and ML

Vision-language-action (VLA) models let robots carry out tasks from camera images and instructions in plain language. The computation that these models need varies between tasks, and even within a single inference. We design hardware and algorithms that exploit this variability to run VLA models on the robot, in real time and within its energy budget.

Research goal: Robots today run billion-parameter VLA models on edge GPUs such as NVIDIA's Jetson AGX Thor, which draws up to 130W but runs NVIDIA's GR00T N1.6 at about 11 Hz. Our goal is hardware that adapts its computation to the task, to run VLA models at the control rate of up to 50 Hz within a few watts.

Gap in the SotA: Deployed VLA models spend the same computation on every inference, regardless of what the task requires. Some algorithms adapt the computation of each inference at run time, but they run on GPUs. On a GPU, an inference that needs less work is not proportionally faster or more efficient, because fixed overheads remain.

Recent results:

Weight precision that rises over the denoising steps. Because the action head of a VLA's Diffusion Transformer generates almost the same action regardless of its initialisation noise, its later denoising steps attenuate the errors of the earlier ones. The weight precision can therefore rise over the steps, whereas image generators amplify early errors and need the opposite order. With plain per-channel rounding and without retraining, 2, 3, 3 and 4 bits in the four steps preserve the task success of GR00T N1.6, GR00T N1.5 and π0.5. To exploit this per-step precision in the memory-bound action head, we designed a programmable accelerator that decodes entropy-coded weights at the DRAM rate and processes them bit-serially. With 50% of the memory bandwidth of Jetson AGX Thor, the accelerator runs the GR00T N1.6 action head 1.7× faster than Thor with 4-bit NVFP4 weights. At equal task success, it also needs 9.1× less energy per inference.

Get in touch
Robin Geens
Phd student
Jonas Crols
Phd student
Marian Verhelst
Academic staff

Publications about this research topic

Robin Geens, Simon Haepers, Jonas Crols, Chao Fang, Marian Verhelst, “Hardware Acceleration of Vision-Language-Action Models with Weight Precision Rising over the Flow Steps,” arXiv preprint arXiv:2611.24234, 2026. arXiv:2611.24234

Other research topics in Hardware-efficient AI and ML

Vertically-Integrated Logic Fabrics for Future 3D Computing Platforms
Hardware-efficient AI and ML
Jannes Willemen | Marian Verhelst
A Mix-Signal Ising Accelerator for Real-World Applications
Mixed-signal circuits and data converters, Hardware-efficient AI and ML
Jiacong Sun, Toon Bettens, Sofie De Weer, Axel Theunis, and Andreas Pedersen | Wim Dehaene and Marian Verhelst
Precision-Scalable Microscaling Hardware for Continual Learning at the Edge
Hardware-efficient AI and ML
Stef Cuyckens | Marian Verhelst
XDMA: A Distributed DMA for Flexible and Efficient Data Movement in Heterogeneous Multi-Accelerator SoCs
Hardware-efficient AI and ML
Yunhao Deng and Fanchen Kong | Marian Verhelst
Anda: Unlocking Efficient LLM Inference with a Variable-Length Grouped Activation Data Format
Hardware-efficient AI and ML
Man Shi, Arne Symons, Robin Geens, and Chao Fang | Marian Verhelst
Massive parallelism for combinatorial optimisation problems
Hardware-efficient AI and ML
Toon Bettens and Sofie De Weer | Wim Dehaene and Marian Verhelst
Hardware for low- and variable-precision Large Language Models
Hardware-efficient AI and ML
Robin Geens and Chao Fang | Marian Verhelst
Carbon-aware Design Space Exploration for AI Accelerators
Hardware-efficient AI and ML
Jiacong Sun | Georges Gielen and Marian Verhelst
Decoupled Control Flow and Memory Orchestration in the Vortex GPGPU
Hardware-efficient AI and ML
Giuseppe Sarda | Marian Verhelst
Automated Causal CNN Scheduling Optimizer for Real-Time Edge Accelerators
Hardware-efficient AI and ML
Jun Yin | Marian Verhelst
A Scalable Heterogenous Multi-accelerator Platform for AI and ML
Hardware-efficient AI and ML
Ryan Antonio | Marian Verhelst
Uncertainty-Aware Design Space Exploration for AI Accelerators
Hardware-efficient AI and ML
Jiacong Sun and Fanchen Kong | Georges Gielen and Marian Verhelst
Integer GEMM Accelerator for SNAX
Hardware-efficient AI and ML
Xiaoling Yi | Marian Verhelst
Improving GPGPU micro architecture for future AI workloads
Hardware-efficient AI and ML
Giuseppe Sarda | Marian Verhelst
SRAM based digital in memory compute macro in 16nm
Hardware-efficient AI and ML
Weijie Jiang | Wim Dehaene
Scalable large array nanopore readouts for proteomics and next-generation sequencing
Analog and power management circuits, Hardware-efficient AI and ML, Biomedical circuits and sensor interfaces
Sander Crols | Filip Tavernier and Marian Verhelst
Design space exploration of in-memory computing DNN accelerators
Hardware-efficient AI and ML
Pouya Houshmand and Jiacong Sun | Marian Verhelst
Multi-core architecture exploration for layer-fused deep learning acceleration
Hardware-efficient AI and ML
Arne Symons | Marian Verhelst
HW-algorithm co-design for Bayesian inference of probabilistic machine learning
Ultra-low power digital SoCs and memories, Hardware-efficient AI and ML
Shirui Zhao | Marian Verhelst
Design space exploration for machine learning acceleration
Hardware-efficient AI and ML
Arne Symons | Marian Verhelst
Enabling Fast Exploration of the Depth-first Scheduling Space for DNN Accelerators
Hardware-efficient AI and ML
Arne Symons | Marian Verhelst
Optimized deployment of AI algorithms on rapidly-changing heterogeneous multi-core compute platforms
Ultra-low power digital SoCs and memories, Hardware-efficient AI and ML
Josse Van Delm | Marian Verhelst
High-throughput high-efficiency SRAM for neural networks
Ultra-low power digital SoCs and memories, Hardware-efficient AI and ML
Wim Dehaene and Marian Verhelst
Heterogeneous Multi-core System-on-Chips for Ultra Low Power Machine Learning Application at the Edge
Hardware-efficient AI and ML
Pouya Houshmand, Giuseppe Sarda, and Ryan Antonio | Marian Verhelst

Want to work with us?

Get in touch or discover the way we can collaborate.