Successive log quantization for cost-efficient neural networks using stochastic computing

Abstract

Despite the multifaceted benefits of stochastic computing (SC) such as low cost, low power, and flexible precision, SC-based deep neural networks (DNNs) still suffer from the long-latency problem, especially for those with high precision requirements. While log quantization can be of help, it has its own accuracy-saturation problem due to uneven precision distribution. In this paper we propose successive log quantization (SLQ), which extends log quantization with significant improvements in precision and accuracy, and apply it to state-of-the-art SC-DNNs. SLQ reuses the existing datapath of log quantization, and thus retains its advantages such as simple multiplier hardware. Our experimental results demonstrate that our SLQ can significantly extend both the accuracy and efficiency of SCDNNs over the state-of-the-art solutions, including linear-quantized and log-quantized SC-DNNs, achieving less than 1∼1.5%p accuracy drop for AlexNet, SqueezeNet, and VGG-S at mere 4∼5-bit weight resolution. © 2019 Copyright held by the owner/author(s).