Precision-Scalable Microscaling Hardware for Continual Learning at the Edge

Research goals: Microscaling (MX) formats promise a single numerical framework that can cover both low bit-width inference and high dynamic range training, which is exactly what edge continual learning in robotics and next-generation NPUs need. Building on this, our work aims to create precision-scalable MX hardware that (i) enables efficient on-device learning for autonomous robots by supporting all six standardized MX data types in a single MAC array, and (ii) integrates such MX datapaths into a general-purpose NPU platform with streaming and control support for mixed precision training and inference. Together, they target a unified compute fabric where one MAC array can fluidly switch between MXINT8 and multiple MXFP modes, while the surrounding architecture and NPU integration sustain high throughput and energy efficiency across diverse workloads such as robotics policies and generic DNNs.

Gap in the SotA: Existing continual learning processors and MX accelerators fall short in two key ways. At the accelerator level, state-of-the-art systems such as Dacapo only support MXINT-like formats and rely on vector-based shared exponent groups. This organization either forces two copies of the weights in memory (to separately serve forward and backward paths) or requires storing the weights in full precision and quantizing them on the fly. Both options are inefficient from a storage perspective and clash with tight memory budgets on edge robots. At the MAC and NPU level, prior MX MACs are dominated by heavy, exponent-aware reduction trees, where a large fraction of area and energy is spent in accumulation, and NPU streamers are provisioned for static worst-case bandwidth, leading to over-provisioned channels and bank contention when precision is reduced. These limitations prevent current MX solutions from delivering truly precision-scalable, system-efficient MX processing across both training and inference.

Results: Our first work introduces the first precision-scalable MX MAC unit that supports all six MX data types, using 2-bit sub-word multipliers and a unified integer-floating-point datapath, and organizes MX values in 64-element square blocks rather than 32-element vectors to make forward and backward passes symmetric without duplicate storage or on-the-fly requantization. Implemented as an MX processing array and GeMM core in TSMC 16 nm at 400 MHz, this design achieves a substantial reduction in memory footprint and a multi-fold increase in effective training throughput over Dacapo under iso peak throughput, while maintaining comparable energy efficiency on several robotics learning workloads, enabling practical continual learning at the edge. The second work then optimizes the MX MAC’s dominant reduction tree with a hybrid integer-floating-point accumulation scheme that relaxes accuracy where safe, and integrates an 8×8 MAC array into the SNAX NPU platform with bandwidth-aware data streaming. The resulting system reaches 657, 1438 to 1675, and 4065 GOPS/W for MXINT8, MXFP8/6, and MXFP4 at throughputs of 64, 256, and 512 GOPS, respectively, improving MX MAC energy efficiency over the previous state of the art while providing a deployable NPU building block for MX-based continual learning.