Examenspresentation för Herman Bergström och Hanna Tärnåsen

Opponent: Elina Petersson

Abstract:
Supervised learning has become a common approach for extracting information fromimages. To effectively train a model, a large amount of labeled data is required.While many image annotation tasks are objective and well-defined, others requirethe annotators to make a subjective assessment. The difficulty and subjective natureof these annotation tasks cause the standard rating-based annotation techniques tosuffer from inconsistencies between annotators, implying that two different annotatorscould assign highly differing labels based on their personal biases. This thesis’overarching goal is to provide an alternate rank-based system for annotating subjectivedata that could be applied to supervised learning, with the hope of increasingconsistency.

The target application for this project is the annotation of the degree of bronchialwall thickening seen in CT scans of the lungs in patients with chronic obstructivepulmonary disease (COPD). Four potential implementations are compared, and consistency,as well as resource demands, are evaluated in several parts. These includeimitating the annotation process with simulation, user evaluation with arbitrarysubjective assessments, and lastly evaluating bronchial wall thickenings with radiologists.After evaluation, it is observed that the implementation showing the most potentialis one based on the TrueSkill algorithm. The findings presented in this thesis indicatea clear increase in inter-annotator agreement for this rank-based system and thestudy demonstrates that the indirect approach of evaluating images creates morereliable labels than the direct rating-based method.

Welcome!

Herman, Hanna and Ida