Energy-based Optimization for Resource Limited Neural Network Accelerators with Fused-layer Support

Abstract

Deep Neural Networks (DNNs) show high accuracy for several artificial intelligence tasks. However, resource limitations present major challenges for designers of specific DNN accelerators. On-chip memory needs a high amount of chip area, whereas external memory introduces off-chip transfers which consume significantly more energy. Different memory schemes have been proposed in the literature to shift the trade-off point between off-chip transfers and on-chip memory. Furthermore, the fusion of different layers also leverages this trade-off. However, the impact on the total energy consumption was not considered so far. In this paper, we extend the analysis with an energy model to minimize energy consumption while using the lowest possible amount of on-chip memory. For example, we see a memory reduction of 36 % at an unchanged energy consumption by using 14 fused-layers in ResNet-50.