Master thesis presentation Gideon Jägenstedt, MPCAS

Abstract: Object tracking in microscopy time-lapse videos is currently mostly done in two steps often using two neural network models, first the images are segmented in order to detect each object within each time frame and extract their centroids using one neural network model. A selected set of properties on the centroids are then used as an input to a second neural network that creates the temporal trajectories by linking the centroids over a sequence of frames.

This work proposes a novel method to combine the object detection step and the linking step which should, in theory, create better linking in time since the combined model has access to not only a set of properties but the complete image of the centroids. Two different architectures of a combined model were tested, one supervised model based on graph neural networks (GNN) and one unsupervised model based on a variational autoencoder (VAE).

The supervised GNN-based model did not succeed in predicting the position of the centroids, but it showed promise in linking the centroids between frames. Therefore, the VAE-based model was developed that uses the same approach for linking. The VAE-based model resulted in a mean absolute error of under 0.002 on its detection placement, a detection miss-rate of 5%, and an F1-score of 65% when linking trajectories on simulated data. While the F1-score for linking trajectories is not great it shows promise and should be able to be improved in future works.

Supervisor: Jesus Pineda Castro
Examiner: Giovanni Volpe
Opponent: Mirja Granfors