AI and machine learning have accelerated the development of autonomous vehicles. By training the car's systems on a vast amount of data, the car can now recognize a parking space, a cyclist on the road, or a small child on the sidewalk. But how much data is enough to ensure that the vehicle does not make mistakes? How accurate does the data and its labels (parking, cyclist, child) have to be? These, and other questions, are what the FAMER project is set to answer.
"We aim to develop tools which will help all stakeholders in the manufacturing process to collaborate and reach a point where everyone feels confident that both the quantity and quality of the data are sufficient to ensure the system’s safety," says Eric Knauss, professor at the Department of Computer Science and Engineering.
In September he launched the FAMER project (Facilitating Multi-Party Engineering of Requirements), which is set to span three years and is funded by Vinnova. The project is coordinated by the University of Gothenburg and partners include Kognic, RISE, Volvo Cars and Zenseact.
Establishing a Common Language
Developing an autonomous vehicle involves multiple parties: one company creates the machine learning model, another is responsible for the camera capturing images, a third for the annotations linked to the pictures and so on. It is a complex process in which all actors need to understand the requirements for the system, and how the various requirements fit together, and which party is responsible to satisfy which requirements.
"Previous studies have shown that the industry struggles to formulate these requirements as different disciplines have very different languages. In this project, we will work on, among other things, developing a common vocabulary, but also identifying the documentation processes of different parties and how they can be interconnected," explains Eric Knauss. He continues:
"Furthermore, this is an iterative process, new challenges arise during the process, and the requirements need to be revised and reformulated while production is in full swing. This requires mutual understanding and a shared language," says Eric Knauss.
Understanding as a Quality Assurance
Today, we do not have a precise understanding of how much data is needed to train an autonomous vehicle. To guarantee safety, we collect excessive data, resulting in a vehicle that is more expensive than necessary. By clearly articulating requirements and understanding all parties’ processes, conscious choices can be made in data collection and processing. This awareness prevents biased data collection (only training the system on images of people of a certain hair color or gender for example) and it also provides manufacturers with an understanding of what may be missing from the data.
"We expect that through our project results, it becomes easier to determine if the system needs more images of, for example, children in snow and rain, or clearer annotations associated with the images for the car to identify pedestrians in the dark. Through an understanding of the different parts of the process, we can increase our confidence in that the data is sufficient and of high enough quality to guarantee safety", says Eric Knauss.
- Head of Division, Interaction Design and Software Engineering, Computer Science and Engineering