ConBG: a contrastive learning method for Chinese unsupervised sentence representation based on Bert and GCN

Junjie Niu

doi:10.1117/12.2655946

12 January 2023 ConBG: a contrastive learning method for Chinese unsupervised sentence representation based on Bert and GCN

Junjie Niu

Author Affiliations +

Proceedings Volume 12509, Third International Conference on Intelligent Computing and Human-Computer Interaction (ICHCI 2022); 125090G (2023) https://doi.org/10.1117/12.2655946
Event: Third International Conference on Intelligent Computing and Human-Computer Interaction (ICHCI 2022), 2022, Guangzhou, China

Abstract

Sentence representation is a typical problem in NLP, which is to use a fine vector to encode the sentence, so that the sentence can contain copious semantic. A high-quality sentence representation benefits a wide range of NLP tasks. Although BERT-based pretrained model performs well on many downstream tasks, it’s sentence representation acts poorly on semantic textual similarity (STS) task. In this article, we propose ConBG, a Contrastive Learning Method for Chinese Sentence Representation Based on Bert and GCN, which is to encode the sentence by a model combined with Bert and Graph Convolutional Network which is to incorporate syntactic information. Then we use data augmentation strategies to create samples, and adopt contrastive learning technique to train the model in a unsupervised way. Experiments on Chinese STS datasets demonstrate that ConBG exceeds previous work of over 1% on average.

Citation Download Citation

Junjie Niu "ConBG: a contrastive learning method for Chinese unsupervised sentence representation based on Bert and GCN", Proc. SPIE 12509, Third International Conference on Intelligent Computing and Human-Computer Interaction (ICHCI 2022), 125090G (12 January 2023); https://doi.org/10.1117/12.2655946

ACCESS THE FULL ARTICLE

INSTITUTIONAL
Select your institution to access the SPIE Digital Library.

SELECT YOUR INSTITUTION

PERSONAL
Sign in with your SPIE account to access your personal subscriptions or to use specific features such as save to my library, sign up for alerts, save searches, etc.

PERSONAL SIGN IN

No SPIE Account? Create one

PURCHASE THIS CONTENT

SUBSCRIBE TO DIGITAL LIBRARY

50 downloads per 1-year subscription

Members: $195

Non-members: $335 ADD TO CART

25 downloads per 1 - year subscription

Members: $145

Non-members: $250 ADD TO CART

PURCHASE SINGLE ARTICLE

Includes PDF, HTML & Video, when available