Hybrid Main Memory systems are often considered as a solution to increasing capacity demands of workloads with large working set sizes. Cost benefits over DRAM-only configurations can be achieved by extending the main memory capacity using emerging memory technologies and reducing the footprint of DRAM. In this case, however, sophisticated management methods are necessary to minimize the workload runtime degradation due to higher access latency of non-DRAM devices. Various methods have been proposed to analyze and match memory access patterns to memory device specifications using either fixed-logic or machine learning algorithms. Most proposals start from a naive and often suboptimal initial page allocation and migrate data between memory devices. Migration, however, incurs considerable time costs due to analysis and implementation overhead. In this work, we explore the gap in initial page allocation and propose PARL - an adaptive and technology-agnostic solution to this problem. PARL is a reinforcement learning agent that learns what decisions lead to the optimization of workload runtime based on received rewards. Using system-level attributes instead of analyzing individual memory access patterns significantly reduces the size of the state space. Contrary to the claims in the existing literature, our approach shows that reinforcement learning can be a viable solution in this context. We evaluate our proposal on a number of workloads and memory configurations and compare it against the existing literature. On average, PARL achieves 16%-43% better workload runtime and 21% better DRAM hitrate than fixed-logic methods. PARL also achieves an average of 9% (up to 34%) better DRAM hitrate than a migration-based method using machine learning.
27/9/2024 11:00 - 12:00
ESAT Aula L
