Understanding how LLMs complete a classical NLP task by gradient accumulation-based circuit discovery

Aibo Wang; Da Xiao

doi:10.1117/12.3031173

19 July 2024 Understanding how LLMs complete a classical NLP task by gradient accumulation-based circuit discovery

Aibo Wang, Da Xiao

Proceedings Volume 13181, Third International Conference on Electronic Information Engineering, Big Data, and Computer Technology (EIBDCT 2024); 131813L (2024) https://doi.org/10.1117/12.3031173
Event: Third International Conference on Electronic Information Engineering, Big Data, and Computer Technology (EIBDCT 2024), 2024, Beijing, China

Abstract

Currently, large language models (LLMs) are increasingly capable of processing natural language problems. However, there is still limited understanding of the internal mechanisms these models employ for such tasks^[1]. Additionally, existing works on model interpretability face challenges such as low complexity of interpretable language tasks, smaller scale of interpretable models, and lengthy attribution times. In our paper, we use the method of gradient accumulation to attribute the process by which the Vicuna-33b model handles the classical Winograd Schema Challeng (WSC) natural language task, explaining the behavior of large models from their internal components. Gradient accumulation helps us rapidly identify the intrinsic logic in natural language problem-solving within the model, and understand the importance of hidden units and their role during the forward pass. Compared to existing interpretability methods, this approach significantly improves in terms of attribution efficiency and the scale of models it can attribute.

(2024) Published by SPIE. Downloading of the abstract is permitted for personal use only.

Citation Download Citation

Aibo Wang and Da Xiao "Understanding how LLMs complete a classical NLP task by gradient accumulation-based circuit discovery", Proc. SPIE 13181, Third International Conference on Electronic Information Engineering, Big Data, and Computer Technology (EIBDCT 2024), 131813L (19 July 2024); https://doi.org/10.1117/12.3031173

ACCESS THE FULL ARTICLE

INSTITUTIONAL
Select your institution to access the SPIE Digital Library.

SELECT YOUR INSTITUTION

PERSONAL
Sign in with your SPIE account to access your personal subscriptions or to use specific features such as save to my library, sign up for alerts, save searches, etc.

PERSONAL SIGN IN

No SPIE Account? Create one

PURCHASE THIS CONTENT

SUBSCRIBE TO DIGITAL LIBRARY

50 downloads per 1-year subscription

Members: $195

Non-members: $335 ADD TO CART

25 downloads per 1 - year subscription

Members: $145

Non-members: $250 ADD TO CART

PURCHASE SINGLE ARTICLE

Includes PDF, HTML & Video, when available