Statistics seminar

Abstract: Sampling from and estimating the normalizing constant of a distribution known only up to proportionality is a fundamental problem in probabilistic inference. Inspired by the success of diffusion- and score-based generative models, a rich family of \emph{diffusion samplers} has emerged---including DDS, PIS, DIS, MCD, CMCD, LDVI, and LRDSBS---that formulate the inference task as learning controlled stochastic differential equations bridging a tractable prior and the target. As these methods were proposed independently, in different notations and with different motivations, making systematic comparison and a precise understanding of their similarities and differences difficult.

In this talk, I will present a unified framework developed in our recent work for diffusion samplers grounded in the \emph{forward-backward Radon-Nikodym derivative} (FBRND) of the controlled forward and backward path measures. We show three distinct---but equivalent--- FBRND decompositions (FBRND-GBS, FBRND-DDS, and FBRND-path) that weight terminal, running, and stochastic costs differently, and a FBRND-Ref relative to a reference process, recovering the Schr\"{o}dinger bridge perspective. We demonstrate how every existing diffusion sampler is an instance of this framework, clarifying which FBRND decomposition and how the controlled diffusion processes each method models. Building on the framework, we untangle the interplay between the choice of learning objective and the specification of the controlled diffusion process, thus expanding the exploration space of diffusion samplers. Empirical results across standard benchmarks validate our theoretical findings and demonstrate that both the choice of objective and the modeling of diffusion processes can substantially affect both sampling and estimation performance.