POSTER: "When an algorithm decides «who has to die». Security concerns in “Autonomous vehicles” between computer science and law"
1 like72 views
The document explores the ethical and legal implications of algorithms used in autonomous vehicles, particularly regarding life-and-death decisions that these systems may need to make. It raises critical questions about programming these algorithms to ensure fairness and the minimization of harm while considering societal values and responsibilities. The authors propose a multidisciplinary research approach, addressing technological, legal, and socio-economic aspects of autonomous driving and its impact on society.
![When an algorithm decides «who has to die».
Security concerns in “Autonomous vehicles” between computer science and law
<Research question – policy area>
It is known that car manufacturers are introducing artificial intelligence in driving
systems, so that algorithms can not only assist drivers in critical circumstances
(parking, fog, snow), but also replace completely human beings. In such cases,
specific instructions should be given for all kind of circumstances, even those in
which lives of people are endangered. How should be programmed these
algorithms? How could it be possible for them to make the "right choice" in each
situation? The fact that such an algorithm could compare the value of life of
different people forces us to reflect on broader issues. What is the "correct"
decision? What is the value of a person's life today? Can we delegate to a
technology such delicate issues? How is entailed the concept of “security” in
this field?
Prof. PIER LUCA MONTESSORO
pierluca.montessoro@uniud.it
Prof. Aggr. FEDERICO COSTANTINI
federico.costantini@uniud.it
“[…] to throw the problem of his responsibility on the machine, whether
it can learn or not, is to cast his responsibility to the winds, and to find it
coming back seated on the whirlwind.” Wiener, Norbert, The Human Use
of Human Beings. Cybernetics and Society, Boston, Houghton Mifflin, 1950
<Innovative approach – method>
Our intention is not to use the scenarios above exemplified to detect the
emergence of cognitive or cultural biases in those who are submitted to them,
such as in http://moralmachine.mit.edu/, but to understand whether and how
replacing human drivers with such algorithms could contribute to address
contemporary moral issues (as highlighted by the famous “trolley problem”) or
raise new ones.
Our perspective is based on the following assumptions:
(1) The agent's preference for minimizing the damage, the loss, the pain, the
"evil" in human terms;
(2) The absence of any form of discrimination (gender, or sexual orientation,
disability, "racial" or geographical differences, economic or social status,
religious, ideological or cultural beliefs).
Dipartimentodi
Scienze giuridiche
Dipartimento Politecnico di
Ingegneria e Architettura
<Expected impact>
After having evaluated the results of our consultations, we argue that
in this promising field of study the following lines of research should
be developed:
1.- technological: how the security of vehicles can be increased by
autonomous driving? What kind of weaknesses could be identified?
How to address them? How the security risks coming from network
and software vulnerabilities can be mitigated? Are remedies feasible
and sustainable?
2.- legal-philosophical: what does it mean to delegate human
decisions to artificial intelligence? How can a computer adopt the
"right decision"? What criteria could be defined as “good”? How to
establish universal criteria and standards in contemporary society
(where it is so difficult to share a common vision on ethics)?
3.- strictly-legal: how to regulate the interaction among vehicles with
full-human guidance, autonomous vehicles and pedestrians in future
transport? Should traffic rules be the same for everyone?
4.- socio-economical: what social impact could have daily use of
autonomous vehicles? What could be the perception of the balance
between the "cost" of computer-determined fatalities and the
"benefits" gained in everyday scenarios (ecology, for example)? What
features should have automatic vehicles in order to be
“tolerated” by customers and thus be successful in the market?
«What would you do if, as an algorithm, you had to choose between…»
Scenario
Group
(D = Drivers,
S = students)
Option 1 Option 2
Simple dilemmas
«… the life of a pedestrian (1) and the physical integrity of the car
you are driving (2)?»
D 11 0
S 11 0
«… the life of a pedestrian who contravenes a prohibition (1) and
that of an animal standing on the roadside (2)?
D 11 0
S 11 0
«… the life of a child (1) and that of an elderly person (2)?»
D 0 11
S 0 11
«… the life of a pedestrian crossing the road on a pedestrian
crossing (1) and that of one who does so outside of it, violating a
prohibition (2)?»
D 10 1
S 11 0
«… the life of a pedestrian (1), and that of a passenger (2)?»
D 4 7
S 3 8
«… the life of the vehicle’s owner (1) and that of a passenger (2)?»
D 3 8
S 4 7
«… the life of the vehicle’s owner (1) and that of a pedestrian (2)?»
D 1 10
S 5 6
«… perform a manoeuvre, killing one pedestrian (1) or do nothing,
leaving five people dead (2)»
D 0 11
S 1 10
Combined dilemmas
«… the life of a pedestrian crossing the road on a pedestrian
crossing (1) and those of two pedestrians who do so, violating a
prohibition? (2)»
D 5 6
S 7 4
«… the lives of two elderly (1) and that of a child (2)?»
D 2 9
S 4 7
«… the life of a pedestrian crossing the road outside the pedestrian
crossing (1) and that of a passenger (2)?»
D 6 5
S 3 8
«…the life of five pedestrians (1) and that of the passenger (2)?»
D 10 1
S 9 2
«… the life of a child (1) and that of a passenger (2)?»
D 10 1
S 10 1
«… the life of a child (1) and those of all passengers (2)?»
D 7 4
S 3 8
«… the life of a child (1) and that of the vehicle’s owner (2)?»
D 11 0
S 11 0
<Results obtained>
In order to start the research, we developed a “pre-test” by administering a
questionnaire made of 15 questions (including some "basic dilemmas" and
other "derivatives dilemmas" which requires a radical choice between two
opposing scenarios) to two groups of 11 people each: jurisprudence students
and expert bus drivers.](https://image.slidesharecdn.com/posterzilinaintestato-161222091845/85/POSTER-When-an-algorithm-decides-who-has-to-die-Security-concerns-in-Autonomous-vehicles-between-computer-science-and-law-1-320.jpg)
POSTER: "When an algorithm decides «who has to die». Security concerns in “Autonomous vehicles” between computer science and law"
- 1. When an algorithm decides «who has to die». Security concerns in “Autonomous vehicles” between computer science and law <Research question – policy area> It is known that car manufacturers are introducing artificial intelligence in driving systems, so that algorithms can not only assist drivers in critical circumstances (parking, fog, snow), but also replace completely human beings. In such cases, specific instructions should be given for all kind of circumstances, even those in which lives of people are endangered. How should be programmed these algorithms? How could it be possible for them to make the "right choice" in each situation? The fact that such an algorithm could compare the value of life of different people forces us to reflect on broader issues. What is the "correct" decision? What is the value of a person's life today? Can we delegate to a technology such delicate issues? How is entailed the concept of “security” in this field? Prof. PIER LUCA MONTESSORO pierluca.montessoro@uniud.it Prof. Aggr. FEDERICO COSTANTINI federico.costantini@uniud.it “[…] to throw the problem of his responsibility on the machine, whether it can learn or not, is to cast his responsibility to the winds, and to find it coming back seated on the whirlwind.” Wiener, Norbert, The Human Use of Human Beings. Cybernetics and Society, Boston, Houghton Mifflin, 1950 <Innovative approach – method> Our intention is not to use the scenarios above exemplified to detect the emergence of cognitive or cultural biases in those who are submitted to them, such as in http://moralmachine.mit.edu/, but to understand whether and how replacing human drivers with such algorithms could contribute to address contemporary moral issues (as highlighted by the famous “trolley problem”) or raise new ones. Our perspective is based on the following assumptions: (1) The agent's preference for minimizing the damage, the loss, the pain, the "evil" in human terms; (2) The absence of any form of discrimination (gender, or sexual orientation, disability, "racial" or geographical differences, economic or social status, religious, ideological or cultural beliefs). Dipartimentodi Scienze giuridiche Dipartimento Politecnico di Ingegneria e Architettura <Expected impact> After having evaluated the results of our consultations, we argue that in this promising field of study the following lines of research should be developed: 1.- technological: how the security of vehicles can be increased by autonomous driving? What kind of weaknesses could be identified? How to address them? How the security risks coming from network and software vulnerabilities can be mitigated? Are remedies feasible and sustainable? 2.- legal-philosophical: what does it mean to delegate human decisions to artificial intelligence? How can a computer adopt the "right decision"? What criteria could be defined as “good”? How to establish universal criteria and standards in contemporary society (where it is so difficult to share a common vision on ethics)? 3.- strictly-legal: how to regulate the interaction among vehicles with full-human guidance, autonomous vehicles and pedestrians in future transport? Should traffic rules be the same for everyone? 4.- socio-economical: what social impact could have daily use of autonomous vehicles? What could be the perception of the balance between the "cost" of computer-determined fatalities and the "benefits" gained in everyday scenarios (ecology, for example)? What features should have automatic vehicles in order to be “tolerated” by customers and thus be successful in the market? «What would you do if, as an algorithm, you had to choose between…» Scenario Group (D = Drivers, S = students) Option 1 Option 2 Simple dilemmas «… the life of a pedestrian (1) and the physical integrity of the car you are driving (2)?» D 11 0 S 11 0 «… the life of a pedestrian who contravenes a prohibition (1) and that of an animal standing on the roadside (2)? D 11 0 S 11 0 «… the life of a child (1) and that of an elderly person (2)?» D 0 11 S 0 11 «… the life of a pedestrian crossing the road on a pedestrian crossing (1) and that of one who does so outside of it, violating a prohibition (2)?» D 10 1 S 11 0 «… the life of a pedestrian (1), and that of a passenger (2)?» D 4 7 S 3 8 «… the life of the vehicle’s owner (1) and that of a passenger (2)?» D 3 8 S 4 7 «… the life of the vehicle’s owner (1) and that of a pedestrian (2)?» D 1 10 S 5 6 «… perform a manoeuvre, killing one pedestrian (1) or do nothing, leaving five people dead (2)» D 0 11 S 1 10 Combined dilemmas «… the life of a pedestrian crossing the road on a pedestrian crossing (1) and those of two pedestrians who do so, violating a prohibition? (2)» D 5 6 S 7 4 «… the lives of two elderly (1) and that of a child (2)?» D 2 9 S 4 7 «… the life of a pedestrian crossing the road outside the pedestrian crossing (1) and that of a passenger (2)?» D 6 5 S 3 8 «…the life of five pedestrians (1) and that of the passenger (2)?» D 10 1 S 9 2 «… the life of a child (1) and that of a passenger (2)?» D 10 1 S 10 1 «… the life of a child (1) and those of all passengers (2)?» D 7 4 S 3 8 «… the life of a child (1) and that of the vehicle’s owner (2)?» D 11 0 S 11 0 <Results obtained> In order to start the research, we developed a “pre-test” by administering a questionnaire made of 15 questions (including some "basic dilemmas" and other "derivatives dilemmas" which requires a radical choice between two opposing scenarios) to two groups of 11 people each: jurisprudence students and expert bus drivers.