Modeling-based design space exploration (DSE) on hardware accelerators has been proved worthwhile for the deep neural network (DNN) scheduling. Yet, the state-of-the-art (SotA) DSE tools only consider the workload on the basis of individual frames. However, there exists many applications targeting spatio-temporal feature extraction that brings forth causal relationships between DNN input frames, especially the spatial-temporal CNN (ST-CNN). In this work, we propose ACCO, a hardware scheduling optimizer for ST-CNNs that considers both the spatial intra-frame and temporal inter-frame CNN optimizations in the same DSE procedure, leading to new pareto-optimal frontiers in resolving the best scheduling strategy for the given hardware-algorithm target couple. ACCO expands the CNN scheduling search space of loop optimizations and the depth-first approach to consider the impact of causal-frame relationship and layer-fusion depth. Compared to the fixed dilated causal structure, ST-CNNs with ACCO reach an ∼8.4× better Energy-Delay-Product. In frame-based CNN optimization tasks, ACCO also improves ∼20% in its layer-fusion optimal points than the SotA CNN DSE toolchain.
3/5/2024 11:00 - 12:00
ESAT Aula L
