Overview
Date:
Starts 2 June 2026, 09:00Ends 2 June 2026, 13:00Location:
Virtual Development Laboratory (VDL), Chalmers Tvärgata 4COpponent:
Professor Andrei Lobov, Department of Mechanical and Industrial Engineering, Faculty of Engineering, Norwegian University of Science and Technology (NTNU), NorwayThesis
Read thesis (Opens in new tab)
This thesis examines the role of computer vision in facilitating robotic automation for non-rigid object assembly, with a particular focus on wire harness assembly tasks during the automotive final assembly stage. Despite extensive research in this field, industrial adoption has remained limited. Accordingly, the thesis analyzes the challenges associated with applying computer vision to wire harness assembly automation and explores strategies to enable practical deployment in industrial environments.
Employing both qualitative and quantitative methods, the research progresses from problem identification to artifact design, demonstration, and evaluation in laboratory and industrially relevant environments. The studies identify challenges at the object, scene, data, and operational levels. To address these challenges, three primary artifacts were developed and evaluated. First, a learning-based perception pipeline enables markerless detection of wire harness components. This demonstrates the feasibility of deep learning-based component recognition and highlights limitations when components possess highly similar or occluded visual features. Second, a robot-assisted pipeline for automated multi-view data acquisition and multimodal annotation substantially accelerates the preparation of computer vision datasets. This pipeline also supports the training and evaluation of learning-based perception methods for industrial applications. Third, a vision-based human–robot collaboration framework for wire harness installation significantly reduces localized physical discomfort while maintaining task success. However, this approach increases mental demand and cycle time, with the majority of the additional time attributable to robot execution.
In summary, this thesis provides deployable methods and practical guidance for data-centric development, interaction design, and takt-time-oriented workflow optimization in non-rigid object assembly automation. It also demonstrates that, given current technological constraints, computer vision is most effective as a human-centered enabler of robot-assisted assembly rather than as a direct pathway to fully autonomous robotic assembly of non-rigid objects.
Employing both qualitative and quantitative methods, the research progresses from problem identification to artifact design, demonstration, and evaluation in laboratory and industrially relevant environments. The studies identify challenges at the object, scene, data, and operational levels. To address these challenges, three primary artifacts were developed and evaluated. First, a learning-based perception pipeline enables markerless detection of wire harness components. This demonstrates the feasibility of deep learning-based component recognition and highlights limitations when components possess highly similar or occluded visual features. Second, a robot-assisted pipeline for automated multi-view data acquisition and multimodal annotation substantially accelerates the preparation of computer vision datasets. This pipeline also supports the training and evaluation of learning-based perception methods for industrial applications. Third, a vision-based human–robot collaboration framework for wire harness installation significantly reduces localized physical discomfort while maintaining task success. However, this approach increases mental demand and cycle time, with the majority of the additional time attributable to robot execution.
In summary, this thesis provides deployable methods and practical guidance for data-centric development, interaction design, and takt-time-oriented workflow optimization in non-rigid object assembly automation. It also demonstrates that, given current technological constraints, computer vision is most effective as a human-centered enabler of robot-assisted assembly rather than as a direct pathway to fully autonomous robotic assembly of non-rigid objects.
Hao Wang
- Doctoral Student, Production Systems, Mechanical Engineering