Decoupled Control Flow and Memory Orchestration in the Vortex GPGPU

Vortex, a newly proposed open-source GPGPU platform based on the RISC-V ISA, offers a valid alternative for GPGPU research over the broadly-used modeling platforms based on commercial GPU's. Similarly to the push originating from the RISC-V movement for CPUs, Vortex can enable a myriad of fresh research directions for GPUs. However, as a young hardware platform, it lacks the performance competitiveness necessary for wide adoption.

Particularly, Vortex underperforms for regular, memory-intensive kernels like linear algebra routines, which form the basis of many applications, including Machine Learning. For such kernels, we identified the control flow management overhead and memory orchestration as the main causes of performance degradation on this Vortex GPGPU platform.

To overcome these problems, this research proposes:

1.  A hardware control manager to accelerate branching and predication in regular loop execution.
2.  Decoupled memory streaming lanes to further hide memory latency with useful computation.

The evaluation results for different kernels showed 8 times faster execution, 10 times reduction in dynamic instruction count, and performance improvement from 0.35 to 1.63 GFLOP/s/mm2.

These enhancements can be integrated into application-level libraries in the future, to unleash Vortex as a competitive open-source GPGPU platform for the next generation of Machine Learning.