Mini-workshop on Statistical Learning and Inference for Complex Dynamical Systems

For online participation please email Adam Andersson.

Schedule:

13.15–13.20 Adam Andersson: Welcome & Introduction

13:20–14:10 Donald Estep: Nonparametric Bayesian Calibration of Computer Models

14:15–14:45 Raul Tempone: From Data Assimilation to Active Sensing: Belief-Space Control with Discrete Observations

14:50–15:20 Christian Andersson Naesseth: Simulation-Free Variational Inference for SDEs

15:20–15:40 Coffee Break

15:40–16:00 Karl Hammar: Low variance importance sampling for discretely observed SDE

16:05–16:25 Mika Persson: When multiple Bayesian estimators estimate each other

16:30–17:00 Larisa Beilina: Adaptive finite element method for malign melanoma detection

17:05–17:35 Anders Szepessy: Convergence for adaptive resampling of random features

Talks:

13:20–14:10 Nonparametric Bayesian Calibration of Computer Models

Speaker: Donald Estep
Affiliation: Department of Statistics and Actuarial Science, Simon Fraser University

Abstract: I will describe a nonparametric Bayesian methodology for calibrating the distribution of parameters in a computer model using field observations. I will describe the theoretical foundation, including; a unique nonparametric Bayesian posterior density corresponding to a chosen prior; a maximum entropy property of the posterior corresponding to the uniform prior; the almost everywhere continuity of the posterior density; the well-posedness of the nonparametric Bayesian posterior, and a comprehensive statistical analysis of an estimator based on importance sampling. Time permitting, I will discuss the condition and stability of nonparametric Bayesian calibration, treatment of asynchronous field observations, and a nonparametric hierarchical Bayesian calibration approach to calibration. Hopefully, on the way, I will answer the question: Can you do Bayesian statistics without using Bayes’ rule? I will illustrate the results using several examples.

14:15–14:45 From Data Assimilation to Active Sensing: Belief-Space Control with Discrete Observations

Speaker: Raul Tempone
Affiliation: King Abdullah University of Science and Technology (KAUST)

Abstract: Many stochastic systems evolve in continuous time but are observed only intermittently, so inference naturally proceeds through a filtering/data-assimilation belief. In decision-making problems, one must also control the system under partial information, and often design the sensing process itself (active sensing). I will present a belief-space formulation for continuous-time stochastic control with discrete-time observations and controllable measurement quality. This setting yields hybrid belief dynamics (prediction between observations and Bayesian jump updates) and two complementary solution viewpoints: dynamic programming (interlaced belief-space HJB with impulse updates) and a belief-space Pontryagin maximum principle (forward–backward system with jump conditions). I will conclude with connections to computational ideas and applications.

14:50 –15:20 Simulation-Free Variational Inference for SDEs

Speaker: Christian Andersson Naesseth
Affiliation: AMLab, Informatics Institute, University of Amsterdam

Abstract: The Stochastic Differential Equation (SDE) is a powerful tool for time series, dynamics and sequence modeling. However, learning SDEs typically relies on adjoint sensitivity methods, which depend on simulation, discretization, and backpropagation through approximate SDE solutions, which limit scalability. I will discuss SDE Matching, a new simulation- and discretization-free variational method for learning SDEs. Inspired by Score- and Flow Matching algorithms for learning generative dynamics, we extend these ideas to the domain of stochastic dynamics for time series and sequence modeling, eliminating the need for costly numerical simulations. Our results demonstrate that SDE Matching achieves performance comparable to adjoint sensitivity methods while drastically reducing computational complexity.

15:40–16:00 Low variance importance sampling for discretely observed SDE

Speaker: Karl Hammar
Affiliations: Department of Electrical Engineering, Chalmers University of Technology & Saab AB

Abstract: Bayesian filtering of nonlinear Stochastic differential equations (SDEs) observed at discrete times presents significant challenges in terms of finding an accurate solution within a limited computational budget. While filtering for linear–Gaussian models admits a computationally cheap exact form - the Kalman filter, this is not the case for nonlinear SDEs. Particle filters provide a solution, but their tractability and scaling are heavily reliant on the availability of efficient proposal kernels that minimize weight degeneracy. So called locally optimal proposals, originating from the kernel-factorization of the smoothed SDE provides an analytically intractable but theoretically attractive importance kernel. On this topic, using Doob’s h-transform, the smoothing measure can be expressed as a controlled SDE with an additional drift term guiding trajectories toward future observations. In this talk, we explore the potential of approximating this control term with neural networks, yielding an approximation to the locally optimal proposal kernel, enabling its practical use. The neural networks are trained using divergencebased objectives and evaluated via effective sample size and likelihood variance, demonstrating their efficiency for inference in nonlinear SDE models.

16.05–16:25 When multiple Bayesian estimators estimate each other

Speaker: Mika Persson
Affiliation: Department of Mathematical Sciences, Chalmers University of Technology and the University of Gothenburg & Saab AB

Abstract: In game theory with partial and noisy information, it is not enough to only estimate the state of the world to take rational decisions, but also to estimate ”what is in the mind” of the other players. But if all players do this, then there will be an infinite hierarchy of ”I believe that you believe that I believe that she believes that he believes” etc. or in mathematical terms, probability measures over probability measures over probability measures indefinitely. In this talk I give a brief high-level inspirational introduction to the mathematical construction of these infinite belief hierarchies or belief spaces. It is intricate construction that requires elements of various parts of mathematics.

16:30–17:00 Adaptive finite element method for malign melanoma detection

Speaker: Larisa Beilina
Affiliation: Department of Mathematical Sciences, Chalmers University of Technology and the University of Gothenburg

Abstract: Inverse and ill-posed problems arise in a wide range of real-world applications, including medical imaging, oil exploration, shape reconstruction, nondestructive material testing, explosive detection, and so on. A typical coefficient inverse problem (CIP) involves determining unknown coefficients in a partial differential equation (PDE), given knowledge of the PDE’s solution either within the computational domain or along part of its boundary.

In this talk we will present how the adaptive finite element method (AFEM) can be effectively employed to solve a coefficient inverse problem for reconstructing the spatial distribution of dielectric permittivity and conductivity functions in Maxwell’s system using backscattering data at the boundary of investigated domain. The reconstruction method relies on an optimization framework that seeks a stationary point of the Lagrangian corresponding to the associated Tikhonov’s functional. A posteriori error estimates will be presented for the Tikhonov’s functional, Lagrangian and the reconstructed parameters. These estimates are then used to formulate different adaptive conjugate gradient algorithms.

Numerical tests will show feasibility of application of an adaptive optimization algorithm for reconstruction of dielectric permittivity and conductivity functions using realistic model of malign melanoma at 6 GHz in 3D.

17:05–17:35 Convergence for adaptive resampling of random features

Speaker: Anders Szepessy
Affiliation: Department of Mathematics, KTH Royal Institute of Technology

Abstract: The machine learning random Fourier feature method for data in high dimension is computationally and theoretically attractive since the optimization is based on a convex standard least squares problem and independent sampling of Fourier frequencies. The challenge is to sample the Fourier frequencies well. I will prove convergence of a data adaptive method based on resampling the frequencies asymptotically optimally as the number of nodes and amount of data tend to infinity. Numerical results based on resampling and adaptive random walk steps together with approximations of the least squares problem by conjugate gradient iterations confirm the analysis.