This WASP AI/Math project aims to shed light on the mathematical structure of unsupervised (deep) learning using techniques and insights from a variety of different fields in mathematics and physics, including quantum mechanics, information theory, differential geometry, group theory and gauge theory.
Specifically, we aim to explore relations between renormalization group flows and the training process of deep Boltzmann machines, which are unsupervised deep neural networks whose construction is heavily influenced by the principles of statistical mechanics. The project also has potential connections with complex geometry and optimal transport theory. A popular presentation of the project is available here
Another direction of the project aims to consider symmetries as an underlying design principle for network architectures. This can be implemented by constructing deep neural networks on a group G that acts transitively on the input data. This is directly relevant for instance in the case of spherical signals where G is a rotation group. Even more generally, it is natural to consider the question of how to train neural networks in the case of “non-Euclidean data”. This is motivated, for example, by applications to omnidirectional vision, biomedicine, and climate observations, just to mention a few situations where data is naturally “non-flat”. Mathematically, this calls for developing a theory of deep learning on manifolds, or even more exotic structures, like graphs or algebraic varieties. A special class consists of homogeneous spaces G/H, where H is a subgroup. The project aims to develop the mathematical framework for convolutional neural networks which are globally equivariant with respect to G and locally with respect to H. The formalism is closely similar to gauge theory in physics and exploring this connection is a potential side project.
Christoffer Petersson, Senior Research Engineer in the Deep Learning Team at Zenuity
, is industrial supervisor in the project.
This project is supported by the Wallenberg AI, Autonomous Systems and Software Program (WASP)
. We currently have one PhD student, Jimmy Aronsson, involved in the project and we are in the process of recruiting another PhD student, scheduled to start in September 2020. Please contact Daniel Persson for further information about this.