Machine Learning Программное обеспечение
A Survey on Loss-based Regularization for Model Generalization
Аннотация
Regularization is a central and fundamental research topic in machine learning. Despite the numerous methods developed in current fast growing literature, there is a lack of theoretical understanding and rigorous analysis of regularization. The first part of this paper is an answer to this need: we formulate regularization as a solution of ill-posed problems, examine fundamental principles of regularization, analyze the effectiveness of Tikhonov regularization in linear systems, and propose a consistency principle for the design of regularization methods in both linear and nonlinear systems. Based on these analyses, the second part of this paper is a survey of recent loss-based regularizers. We develop a comprehensive taxonomy and summarize prominent features of these regularizers. Finally, results of benchmark tests confirm the effectiveness of representative loss-based regularizers.