Compressing CNN by alternating constraint optimization framework

Peidong Liu; Weirong Liu; Changhong Shi; Zhiqiang Zhang; Zhijun Li; Jie Liu

doi:10.1117/12.2643734

12 October 2022 Compressing CNN by alternating constraint optimization framework

Peidong Liu, Weirong Liu, Changhong Shi, Zhiqiang Zhang, Zhijun Li, Jie Liu

Proceedings Volume 12342, Fourteenth International Conference on Digital Image Processing (ICDIP 2022); 123421U (2022) https://doi.org/10.1117/12.2643734
Event: Fourteenth International Conference on Digital Image Processing (ICDIP 2022), 2022, Wuhan, China

Abstract

Tensor decomposition has been extensively studied for convolutional neural networks (CNN) model compression. However, the direct decomposition of an uncompressed model into low-rank form causes unavoidable approximation error due to the lack of low-rank property of a pre-trained model. In this manuscript, a CNN model compression method using alternating constraint optimization framework (ACOF) is proposed. Firstly, ACOF formulates tensor decomposition-based model compression as a constraint optimization problem with low tensor rank constraints. This optimization problem is then solved systematically in an iterative manner using alternating direction method of multipliers (ADMM). During the alternating process, the uncompressed model gradually exhibits low-rank tensor property, and then the approximation error in low-rank tensor decomposition can be negligible. Finally, a high-performance CNN compression network can be effectively obtained by SGD-based fine-tuning. Extensive experimental results on image classification show that ACOF produces the optimal compressed model with high performance and low computational complexity. Notably, ACOF compresses Resnet56 to 28% without accuracy drop, and the compressed model have 1.14% higher accuracy than learning-compression (LC) method.

Citation Download Citation

Peidong Liu, Weirong Liu, Changhong Shi, Zhiqiang Zhang, Zhijun Li, and Jie Liu "Compressing CNN by alternating constraint optimization framework", Proc. SPIE 12342, Fourteenth International Conference on Digital Image Processing (ICDIP 2022), 123421U (12 October 2022); https://doi.org/10.1117/12.2643734

ACCESS THE FULL ARTICLE

INSTITUTIONAL
Select your institution to access the SPIE Digital Library.

SELECT YOUR INSTITUTION

PERSONAL
Sign in with your SPIE account to access your personal subscriptions or to use specific features such as save to my library, sign up for alerts, save searches, etc.

PERSONAL SIGN IN

No SPIE Account? Create one

PURCHASE THIS CONTENT

SUBSCRIBE TO DIGITAL LIBRARY

50 downloads per 1-year subscription

Members: $195

Non-members: $335 ADD TO CART

25 downloads per 1 - year subscription

Members: $145

Non-members: $250 ADD TO CART

PURCHASE SINGLE ARTICLE

Includes PDF, HTML & Video, when available