Strong replica symmetry for high-dimensional disordered log-concave Gibbs measures
Manuel Saenz
In machine learning, convex optimisation problems are ubiquitous. Of them, several have been studied with the tools and techniques of disordered systems. This type of problems include, for example, many instances of empirical loss minimisation and maximum a posteriori inference. Their analysis usually involves establishing some type of replica symmetric formula for the associated free energy and inference or generalisation error. For this, most approaches rely, in one way or another, on the concentration of the relevant order parameters (usually in the form of overlaps). On this talk, we will study disordered systems with concave Hamiltonians. Assuming that the free entropy of the system concentrates, we will establish the joint convergence of the multitioverlaps (generalisations of the overlap). Furthermore, we will show that this strong replica symmetric behaviour implies a simple representation for the distribution of the spins in the large system limit and the asymptotic strong thermal decorrelation of finite sets of spins. These results may prove themselves useful in several machine learning problems, particularly in sparse settings and in inference outside the Bayes-optimal regime.