For nonexpansive fixed-point problems, Halpern’s method with optimal parameters, its so-called H-dual algorithm, and in fact, an infinite family of algorithms containing them, all exhibit the exact minimax optimal convergence rate. In this work, we provide a characterization of the complete, exhaustive family of distinct algorithms using predetermined step-sizes, represented as lower triangular H-matrices, which attain the same optimal convergence rate. The characterization is based on polynomials in the entries of the H-matrix that we call H-invariants , whose values stay constant over all optimal H-matrices, together with H-certificates , the nonnegativity of which precisely specifies the region of optimality within the common level set of H-invariants. The H-invariance theory we present offers a novel view of optimal acceleration in first-order optimization as a mathematical study of carefully selected invariants, certificates, and structures induced by them.
An algorithm based on the interior-point methodology for solving continuous nonlinearly constrained optimization problems is proposed, analyzed, and tested. The distinguishing feature of the algorithm is that it presumes that only noisy...
Introduced by Beck and Teboulle in [1], FISTA (Fast Iterative Shrinkage-Thresholding Algorithm) is a first-order method widely used in convex optimization. Adapted from Nesterov’s accelerated gradient method for convex functions [2],...
The progressive decoupling algorithm solves problems of optimization by iteratively suppressing complications coming from linear linkage relations. Here its formulation is significantly extended and a sharper theory of convergence is...
This paper introduces a class of two-stage stochastic minimax problems where the first-stage objective function is nonconvex-concave while the second-stage objective function is strongly convex-concave. We establish properties of the...
Inspired by Von Neumann’s foundational work on expected utility, this paper extends the data-driven expected utility maximization framework under constant absolute risk aversion (CARA) preferences by incorporating nonlinear payoff...