A data-centric reinforcement learning approach for self-updating
machine learning models

Mandy Sack

doi:10.1117/12.2615873

30 May 2022 A data-centric reinforcement learning approach for self-updating machine learning models

Mandy Sack

Proceedings Volume 12119, Open Architecture/Open Business Model Net-Centric Systems and Defense Transformation 2022; 121190H (2022) https://doi.org/10.1117/12.2615873
Event: SPIE Defense + Commercial Sensing, 2022, Orlando, Florida, United States

Abstract

Continuously Updating Reinforcement Learning (CURL) demonstrates the ability to rapidly maintain deployed ML models when there is a change in use case such as a denied target with minimal performance effects. Traditional Machine Learning (ML) lifecycle requires ML models to be retrained and redeployed in order to maintain performance of deployed models experiencing changes in underlying data such as data drift. Data drift can include a wide variety of changes in data such as the addition of a new class, operating in an entirely new environment, mislabeled data, or subtle changes in targets over time. (CURL) deviates from this traditional lifecycle with dynamic updates using Reinforcement Learning (RL) to identify and capture data changes, and then automatically retrain the model with data changes. CURL learns to identify changes in data through its RL policy that is designed to maximize the reward for identifying changes in data. Specifically, CURL’s RL approach includes an environment with both the model’s performance and current prediction confidence as the observation space for the agent to act on the discrete action space, and reward function of the model’s accuracy subtracted by the labeling cost to learn data changes. Our controlled experiment demonstrated that the same distribution of denied target data (3%) was found by our RL policy, and our retrained model exceeded the initial classifier performance. CURL can be considered a general purpose technology that could be applied to a wide spectrum of fielded ML systems.

Conference Presentation

Citation Download Citation

Mandy Sack "A data-centric reinforcement learning approach for self-updating machine learning models", Proc. SPIE 12119, Open Architecture/Open Business Model Net-Centric Systems and Defense Transformation 2022, 121190H (30 May 2022); https://doi.org/10.1117/12.2615873

ACCESS THE FULL ARTICLE

INSTITUTIONAL
Select your institution to access the SPIE Digital Library.

SELECT YOUR INSTITUTION

PERSONAL
Sign in with your SPIE account to access your personal subscriptions or to use specific features such as save to my library, sign up for alerts, save searches, etc.

PERSONAL SIGN IN

No SPIE Account? Create one

PURCHASE THIS CONTENT

SUBSCRIBE TO DIGITAL LIBRARY

50 downloads per 1-year subscription

Members: $195

Non-members: $335 ADD TO CART

25 downloads per 1 - year subscription

Members: $145

Non-members: $250 ADD TO CART

PURCHASE SINGLE ARTICLE

Includes PDF, HTML & Video, when available