View/ Open
Date
2021-07-16Subject
004 Data processing and computer science Neuronales NetzMaschinelles LernenErneuerbare EnergienPrognoseMetadata
Show full item record
Aufsatz
CLeaR: An adaptive continual learning framework for regression tasks
Abstract
Catastrophic forgetting means that a trained neural network model gradually forgets the previously learned tasks when being retrained on new tasks. Overcoming the forgetting problem is a major problem in machine learning. Numerous continual learning algorithms are very successful in incremental learning of classification tasks, where new samples with their labels appear frequently. However, there is currently no research that addresses the catastrophic forgetting problem in regression tasks as far as we know. This problem has emerged as one of the primary constraints in some applications, such as renewable energy forecasts. This article clarifies problem-related definitions and proposes a new methodological framework that can forecast targets and update itself by means of continual learning. The framework consists of forecasting neural networks and buffers, which store newly collected data from a non-stationary data stream in an application. The changed probability distribution of the data stream, which the framework has identified, will be learned sequentially. The framework is called CLeaR (Continual Learning for Regression Tasks), where components can be flexibly customized for a specific application scenario. We design two sets of experiments to evaluate the CLeaR framework concerning fitting error (training), prediction error (test), and forgetting ratio. The first one is based on an artificial time series to explore how hyperparameters affect the CLeaR framework. The second one is designed with data collected from European wind farms to evaluate the CLeaR framework’s performance in a real-world application. The experimental results demonstrate that the CLeaR framework can continually acquire knowledge in the data stream and improve the prediction accuracy. The article concludes with further research issues arising from requirements to extend the framework.
Citation
In: AI Perspectives Volume:3 (2021-07-16) eissn:2523-3981Sponsorship
Gefördert durch den Publikationsfonds der Universität KasselCitation
@article{doi:10.17170/kobra-202201215585,
author={He, Yujiang and Sick, Bernhard},
title={CLeaR: An adaptive continual learning framework for regression tasks},
journal={AI Perspectives},
year={2021}
}
0500 Oax 0501 Text $btxt$2rdacontent 0502 Computermedien $bc$2rdacarrier 1100 2021$n2021 1500 1/eng 2050 ##0##http://hdl.handle.net/123456789/13604 3000 He, Yujiang 3010 Sick, Bernhard 4000 CLeaR: An adaptive continual learning framework for regression tasks / He, Yujiang 4030 4060 Online-Ressource 4085 ##0##=u http://nbn-resolving.de/http://hdl.handle.net/123456789/13604=x R 4204 \$dAufsatz 4170 5550 {{Neuronales Netz}} 5550 {{Maschinelles Lernen}} 5550 {{Erneuerbare Energien}} 5550 {{Prognose}} 7136 ##0##http://hdl.handle.net/123456789/13604
2022-02-04T10:06:41Z 2022-02-04T10:06:41Z 2021-07-16 doi:10.17170/kobra-202201215585 http://hdl.handle.net/123456789/13604 Gefördert durch den Publikationsfonds der Universität Kassel eng Namensnennung 4.0 International http://creativecommons.org/licenses/by/4.0/ Continual learning Renewable energy forecasts Regression tasks Deep neural networks 004 CLeaR: An adaptive continual learning framework for regression tasks Aufsatz Catastrophic forgetting means that a trained neural network model gradually forgets the previously learned tasks when being retrained on new tasks. Overcoming the forgetting problem is a major problem in machine learning. Numerous continual learning algorithms are very successful in incremental learning of classification tasks, where new samples with their labels appear frequently. However, there is currently no research that addresses the catastrophic forgetting problem in regression tasks as far as we know. This problem has emerged as one of the primary constraints in some applications, such as renewable energy forecasts. This article clarifies problem-related definitions and proposes a new methodological framework that can forecast targets and update itself by means of continual learning. The framework consists of forecasting neural networks and buffers, which store newly collected data from a non-stationary data stream in an application. The changed probability distribution of the data stream, which the framework has identified, will be learned sequentially. The framework is called CLeaR (Continual Learning for Regression Tasks), where components can be flexibly customized for a specific application scenario. We design two sets of experiments to evaluate the CLeaR framework concerning fitting error (training), prediction error (test), and forgetting ratio. The first one is based on an artificial time series to explore how hyperparameters affect the CLeaR framework. The second one is designed with data collected from European wind farms to evaluate the CLeaR framework’s performance in a real-world application. The experimental results demonstrate that the CLeaR framework can continually acquire knowledge in the data stream and improve the prediction accuracy. The article concludes with further research issues arising from requirements to extend the framework. open access He, Yujiang Sick, Bernhard doi:10.1186/s42467-021-00009-8 Neuronales Netz Maschinelles Lernen Erneuerbare Energien Prognose publishedVersion eissn:2523-3981 AI Perspectives Volume:3 false Article number: 2
The following license files are associated with this item: