Partial Sum Quantization for Reducing ADC Size in ReRAM-based Neural Network Accelerators

Abstract

While ReRAM (Resistive Random-Access Memory) crossbar arrays have the potential to significantly accelerate DNN (Deep Neural Network) training through fast and lowcost matrix-vector multiplication, peripheral circuits like ADCs (analog-to-digital converters) create a high overhead. These ADCs consume over half of the chip power and a considerable portion of the chip cost. To address this challenge, we propose advanced quantization techniques that can significantly reduce the ADC overhead of ReRAM crossbar arrays. Our methodology interprets ADC as a quantization mechanism, allowing us to scale the range of ADC input optimally along with the weight parameters of a DNN, resulting in multiple-bit reduction in ADC precision. This approach reduces ADC size and power consumption by several times, and it is applicable to any DNN type (binarized or multi-bit) and any ReRAM crossbar array size. Additionally, we propose ways to minimize the overhead of the digital scaler, which is an essential part of our scheme and sometimes required. Our experimental results using ResNet-18 on the ImageNet dataset demonstrate that our method can reduce the size of the ADC by 32 times compared to ISAAC with only a minimal accuracy loss degradation of 0.24 evaluation results in the presence of ReRAM non-ideality (such as stuck-at fault). IEEE