A Survey on Binary and Ternary Neural Networks and Their Realization in Compute-in-Memory for Edge Intelligence

초록

Deep learning has achieved remarkable success across a wide range of applications, such as language modeling, computer vision, recommendation systems, and robotics. However, the growing size of models and their increasing computational demands pose significant challenges, particularly for resource-constrained devices. A promising approach to address these challenges is extreme quantization, exemplified by binary and ternary neural networks. These techniques significantly reduce model size by quantizing weights and activations to 1 bit or 1.58 bits, while simplifying computation, making them well-suited for efficient deployment in resource-limited environments. This paper presents a comprehensive review of extreme quantization techniques, organized into three key areas: (1) a comparative analysis of quantizing only the weights (e.g., binary weight networks, ternary weight networks) versus quantizing both weights and activations (e.g., binary neural networks, ternary neural networks), along with a discussion of the progress and trade-offs of their approaches; (2) an examination of how extreme quantization, initially applied to convolutional neural networks, has been extended to Transformer architectures; and (3) an overview of compute-in-memory architectures optimized for binarization and ternarization, including designs based on advanced bit-cell technologies. © 2025 Institute of Electrical and Electronics Engineers Inc.. All rights reserved.

키워드