MAML-BERT in addressing low-resource text classification tasks

Yang Hu; Guiyun Zhang

doi:10.1117/12.3038663

7 August 2024 MAML-BERT in addressing low-resource text classification tasks

Yang Hu, Guiyun Zhang

Author Affiliations +

Proceedings Volume 13229, Seventh International Conference on Advanced Electronic Materials, Computers, and Software Engineering (AEMCSE 2024); 132293C (2024) https://doi.org/10.1117/12.3038663
Event: Seventh International Conference on Advanced Electronic Materials, Computers, and Software Engineering (AEMCSE 2024), 2024, Nanchang, China

Abstract

The traditional approach for text classification tasks commonly involves fine-tuning existing general-purpose or base models. However, when training data is extremely scarce, this approach may lead to overfitting or getting stuck in local optima. This paper investigates meta-learning-based methods for low-resource text classification, taking a diabetes dataset as an example, and proposes a meta-learning framework combined with data augmentation techniques. Firstly, we utilize an abstract summarization method to effectively augment the original dataset, alleviating potential issues caused by data imbalance while enhancing the diversity and generalization of the samples. Then, by employing a meta-learning algorithm, we achieve optimization and adjustment of the global initialization parameters. Subsequently, these parameters will guide the fine-tuning of the pre-trained BERT model to adapt to the diabetes text classification task. Experimental results show that our method significantly improves classification performance under low-resource conditions, providing new insights for handling low-resource text classification problems in similar domains.

(2024) Published by SPIE. Downloading of the abstract is permitted for personal use only.

Citation Download Citation

Yang Hu and Guiyun Zhang "MAML-BERT in addressing low-resource text classification tasks", Proc. SPIE 13229, Seventh International Conference on Advanced Electronic Materials, Computers, and Software Engineering (AEMCSE 2024), 132293C (7 August 2024); https://doi.org/10.1117/12.3038663

ACCESS THE FULL ARTICLE

INSTITUTIONAL
Select your institution to access the SPIE Digital Library.

SELECT YOUR INSTITUTION

PERSONAL
Sign in with your SPIE account to access your personal subscriptions or to use specific features such as save to my library, sign up for alerts, save searches, etc.

PERSONAL SIGN IN

No SPIE Account? Create one

;

PURCHASE THIS CONTENT

SUBSCRIBE TO DIGITAL LIBRARY

50 downloads per 1-year subscription

Members: $195

Non-members: $335 ADD TO CART

25 downloads per 1 - year subscription

Members: $145

Non-members: $250 ADD TO CART

PURCHASE SINGLE ARTICLE

Includes PDF, HTML & Video, when available

Members: $17.00

Non-members: $21.00 ADD TO CART

PROCEEDINGS
7 PAGES

DOWNLOAD PAPER SAVE TO MY LIBRARY

GET CITATION

RIGHTS & PERMISSIONS

Get copyright permission Get copyright permission on Copyright Marketplace

KEYWORDS

Data modeling

Education and training

Performance modeling

Machine learning

Transformers

Mathematical optimization

Statistical modeling

Show All Keywords

Keywords/Phrases

Search In:

Publication Years