Multi-label text classification based on graph attention network and self-attention mechanism

Can Lin; Cui Zhu; Wenjun Zhu

doi:10.1117/12.2653459

27 September 2022 Multi-label text classification based on graph attention network and self-attention mechanism

Can Lin, Cui Zhu, Wenjun Zhu

Proceedings Volume 12346, 2nd International Conference on Information Technology and Intelligent Control (CITIC 2022); 1234616 (2022) https://doi.org/10.1117/12.2653459
Event: 2nd International Conference on Information Technology and Intelligent Control (CITIC 2022), 2022, Kunming, China

Abstract

Multi-Label Text Classification (MLTC) is a task to assign documents to one or more class labels. Most of the latest approaches are based on deep learning. However, among the various types of methods, there are still defects in capturing fine-grained information in the text and in modeling the dependencies between labels. In recent years, neural network model-based approaches have made great progress, among which the BERT language model can capture bidirectional contextual information well but cannot capture label-specific semantic information, so we propose a new multi-label text classification framework LAGAT by combining attention mechanism for feature learning. This framework is devoted to mining inter-label dependencies by constructing graphs of label-specific text semantic information and using the graph attention network (GAT) to learn the relative importance between labels by updating the weights on the edges, thus mining the inter-label dependencies. In further experiments, we confirm that the GAT is weaker than the self-attention mechanism in mining text labels by aggregating neighborhood information at moderate label size, and propose an improved model framework LA-Trans, which also utilizes label association relations with text semantic information, focuses on the target region and ignores irrelevant information, and achieves the performance of state of the arts.

Citation Download Citation

Can Lin, Cui Zhu, and Wenjun Zhu "Multi-label text classification based on graph attention network and self-attention mechanism", Proc. SPIE 12346, 2nd International Conference on Information Technology and Intelligent Control (CITIC 2022), 1234616 (27 September 2022); https://doi.org/10.1117/12.2653459

ACCESS THE FULL ARTICLE

INSTITUTIONAL
Select your institution to access the SPIE Digital Library.

SELECT YOUR INSTITUTION

PERSONAL
Sign in with your SPIE account to access your personal subscriptions or to use specific features such as save to my library, sign up for alerts, save searches, etc.

PERSONAL SIGN IN

No SPIE Account? Create one

PURCHASE THIS CONTENT

SUBSCRIBE TO DIGITAL LIBRARY

50 downloads per 1-year subscription

Members: $195

Non-members: $335 ADD TO CART

25 downloads per 1 - year subscription

Members: $145

Non-members: $250 ADD TO CART

PURCHASE SINGLE ARTICLE

Includes PDF, HTML & Video, when available

Members: $17.00

Non-members: $21.00 ADD TO CART

PROCEEDINGS
9 PAGES

DOWNLOAD PAPER SAVE TO MY LIBRARY

GET CITATION

RIGHTS & PERMISSIONS

Get copyright permission Get copyright permission on Copyright Marketplace

KEYWORDS

Neural networks

Data modeling

Classification systems

Mining

Performance modeling

Statistical modeling

Head

Show All Keywords

Keywords/Phrases

Search In:

Publication Years