AI Ethics: Superintelligent agents

Welcome to a new series of seminars under the theme: AI ETHICS at Chalmers. The seminars are organised by the AI Ethics Committee, within Chalmers AI Research Centre, CHAIR.

Speaker: Karim Jebari, The Swedish Institute for Futures Studies

Abstract: An intelligent machine surpassing human intelligence across a wide set of skills has been proposed as a possible existential catastrophe (i.e., an event comparable in value to that of human extinction). Among those concerned about existential risk related to Artificial Intelligence (AI), it is common to assume that AI will not only be very intelligent, but also be a general agent (i.e., an agent capable of action in many different contexts). This article explores the characteristics of machine agency, and what it would mean for a machine to become a general agent. In particular, it does so by articulating some important differences between belief and desire in the context of machine agency. One such difference is that while an agent can by itself acquire new beliefs through learning, desires need to be derived from preexisting desires or acquired with the help of an external influence. Such influence could be a human programmer or natural selection. We argue that to become a general agent, a machine needs general desires, but general desires cannot sui generis be derived from non-general desires. Thus, even though general agency in AI could in principle be created, it cannot be arrived at spontaneously by an endogenous process. In conclusion, we argue that a common AI scenario, where general agency suddenly emerges in a non-general agent AI, is not plausible.

About Karim Jebari

Karim Jebari is a researcher at the Institute for Futures Studies. He defended his doctoral thesis in December 2014 at the Royal Institute of Technology (KTH). The thesis is about applied ethics and in particular how we should relate to the risks and opportunities of technological innovation.


​After the seminar you are welcome to stay for a cup of coffee or tea.

Category Seminar
Location: Lecture Hall Palmstedt, university building, Chalmersplatsen 2, Campus Johanneberg
Starts: 10 September, 2019, 10:00
Ends: 10 September, 2019, 11:00

Published: Tue 30 Jul 2019. Modified: Wed 07 Aug 2019