Mitigating The Impact of ReRAM I-V Nonlinearity and IR Drop via Fast Offline Network Training

Abstract

ReRAM crossbar arrays (RCAs) have the potential to provide extremely high efficiency for accelerating deep neural networks (DNNs). However, one crucial challenge for RCA-based DNN accelerators is functional inaccuracy due to nonidealities present in RCA hardware. While nonideality-aware training could be used to mitigate the effect of nonidealities, with currently available methods it would take months to train even a medium size convolutional neural network (CNN). In this paper we propose a nonideality prediction method that enables very fast training of RCA-based neural networks, and show its feasibility through nonideality-aware training of DNNs. Our key ideas include (i) weight-centric nonideality modeling and (ii) data-dependence elimination by tailored input randomization. Our experimental results using a multi-layer perceptron and CNNs demonstrate that our method is very fast (100 15,000× faster training speed) while achieving much better crossbar-level accuracy (2 90× lower RMS error) and post-retraining validated accuracy than previous methods. © 1982-2012 IEEE.