Unsupervised learning for dense prediction tasks

Abstrakt:

Dense prediction tasks, such as segmentation and optical flow, are fundamental in many computer vision applications. However, generating large-scale manually annotated training datasets for these tasks is incredibly costly and labour intensive. We tackle this problem by exploring weakly and unsupervised learning strategies derived from multi-frame constraints. First, we introduce the Warp Consistency objective for dense correspondence estimation and optical flow. Given a pair of images of the same scene, the warp consistency objective effectively supervises dense correspondence prediction without relying on photometric losses, while avoiding degenerate solutions. Second, we develop an objective for mask-free learning of segmentation by
leveraging temporal consistency in videos. Even though our model has not seen a single ground-truth mask during training, it achieves highly competitive results for Video Instance Segmentation, opening up the
possibility of harnessing the virtually endless pool of unannotated videos for training high-level vision models.

This talk will be based on the following publications:
Mask-Free Video Instance Segmentation, https://arxiv.org/abs/2303.15904
Warp Consistency for Unsupervised Learning of Dense Correspondences, https://arxiv.org/abs/2104.03308
Probabilistic Warp Consistency for Weakly-Supervised Semantic Correspondences, https://arxiv.org/abs/2203.04279

Om talaren

Martin Danelljan is a group leader and lecturer at ETH Zurich, Switzerland. He received his Ph.D. degree from Linköping University, Sweden in 2018. His Ph.D. thesis was awarded the biennial Best Nordic Thesis Prize at SCIA 2019. He has an h-index of 42 and over 22k citations. His research interests include, tracking, segmentation, deep probabilistic models, image enhancement and generation. His research in the field of visual tracking, in particular, has attracted much attention, achieving first rank in the 2014, 2016, and 2017 editions of the Visual Object Tracking (VOT) Challenge and the OpenCV State-of-the-Art Vision Challenge. He received the best paper award at ICPR 2016, the best student paper at BMVC 2019, and an outstanding reviewer award at ECCV 2020. He is also a co-organizer of the VOT, NTIRE, and AIM workshops. He served as an Area Chair for CVPR 2022 and Senior PC for AAAI 2022.

You can join us in the EDIT room or on Teams.