Machine Learning Программное обеспечение

A Survey on Loss-based Regularization for Model Generalization

Huang, Yaxing, Zhang, Chenhe, Xu, Renjun, Zhang, Qinghai
Springer Nature · 2026-10-10 · DOI 10.1007/s10994-026-07139-2

Аннотация

Regularization is a central and fundamental research topic in machine learning. Despite the numerous methods developed in current fast growing literature, there is a lack of theoretical understanding and rigorous analysis of regularization. The first part of this paper is an answer to this need: we formulate regularization as a solution of ill-posed problems, examine fundamental principles of regularization, analyze the effectiveness of Tikhonov regularization in linear systems, and propose a consistency principle for the design of regularization methods in both linear and nonlinear systems. Based on these analyses, the second part of this paper is a survey of recent loss-based regularizers. We develop a comprehensive taxonomy and summarize prominent features of these regularizers. Finally, results of benchmark tests confirm the effectiveness of representative loss-based regularizers.

Читать у издателя →