An Energy-Efficient 4K Super Resolution Accelerator with Variable Length Compression

Abstract

A super resolution (SR) accelerator with variable bit compression is proposed for 4K restoration with > 60 frames- per-second (fps) in mobile video applications. Although demands of 4K video contents grows dramatically, huge feature maps of 4K SR with low bit precision prohibit its real time operation due to the large external memory access (EMA) bandwidth that recent mobile processors cannot support. Despite the effort of previous SR processors to reduce EMA by quantizing both feature maps and weights, they suffered from the trade-off between EMA and network performance drop. To facilitate > 60 fps 4K SR on mobile processor, this work proposes key contributions at both software and hardware level: the tile-based distribution-aware statistical encoding that results in 61.2% compression rate without information loss; and an energy-efficient SR processor that supports variable bit compression that reduces power consumption and and EMA by 34.2% and 89.7%, respectively. Designed with 28 nm CMOS technology, the proposed system can accelerate ×2 scale 4K image restoration at 68.7 fps. It shows 3.69 TOPS of peak performance and 10.1 TOPS/W of energy efficiency, achieving 2.8× higher energy efficiency than the previous SR processor.