Artificial Intelligence and Meaningful Human Control: Agent-Neutral-Reason-Responsive Mechanisms and the Possibility of Tracing Responsible Agent(s)

Some ethical issues related to artificial intelligence relate to the nature of autonomy and the problem of the responsibility gap; autonomous AI systems have behaviors that would be called “decisions” if humans made them, and humans would be held responsible if those “decisions” caused harm. However, when an “action” is performed by autonomous technology, although there is still a need for responsibility, nothing may be held responsible for it. Therefore, it is suggested that autonomous technologies remain under human responsibility if they meet two (design) requirements: 1) the ability to “track” human reasons and 2) by following the chain of human reasons, a human agent can be “traced.” These conditions will create many ambiguities for autonomous AI systems; for example, the moral reasons that need to be tracked are often agent-neutral, but tracing a human agent is only possible through agent-dependent reasons. In this article, by analyzing the interpretations of the two conditions, an interpretation that has not been proposed before will be defended as the least problematic interpretation. To this end, it is argued that the responsibility gap is solved when the history of events produced by autonomous systems involves a process by which the agent takes responsibility for an agent-neutral normative reason-responsive mechanism.
 
Introduction
According to Santoni de Sio and van den Hoven (referring to Fischer and Ravizza’s compatibilist approach to human responsibility), autonomous artifacts remain meaningfully under human control if they meet two specific conditions: 1) The artifact can “track” human reasons, and 2) a human agent can be “traced” by following a chain of human reasons. Fulfilling these two conditions still faces problems and ambiguities that continue to be the subject of philosophical and empirical debate.
Two Interpretations of Tracing
In describing the second condition (originally for human responsibility), Fischer and Ravizza point out that responsibility is essentially historical. Every act has a certain causal history, and the attribution of responsibility for actions to an agent depends on the nature of the causal chain that shaped that history. The historical nature of responsibility can be interpreted (at least) in two different ways.
In one interpretation, responsibility is attributed to the person who first intentionally (i.e., with reason) performed an effective act. Therefore, an event (such as killing people on the road) includes a purposeful act (such as drinking or forcefully feeding alcohol) in the causal history of those events, and therefore responsibility for the event is attributed to the agent of that historical purposeful act. A drunk driver, although driving unintentionally, is responsible for accidents and deaths on the road because he/she drank even though he knew he wanted to drive. If he/she were forcibly fed alcohol, the agent who fed the alcohol to the driver would be responsible for the events. So, responsibility for a bad event caused by an autonomous car historically goes back to a human who made an informed decision.
In another interpretation, taking responsibility is not simply an act. Responsibility is attributed to an agent when the history of events includes a process of taking responsibility by an agent, in which a set of beliefs is formed for users or stakeholders; for example, in the process of taking responsibility, the belief is formed that an autonomous AI system belongs to a human agent. The set of beliefs is part of (not the causal chain, but) the mental or conceptual space of human beings who may not play any direct role in the causal chain.
The two interpretations will be effective in the argument presented about tracing conditions.
Taking responsibility for an agent-Neutral-Reason-responsive AI
Reasons may be motivational or normative. Motivational reasons are the reasons why an agent acted, and normative reasons are the reasons that justify an action morally or rationally. We are interested in having an AI system track normative reasons. However, distinguishing normative from motivational reasons requires a normative theory and is not directly possible for the system.
Another issue is that Fisher and Raviza’s conditions of responsibility were originally defined for humans (not AI systems). Since they are defined for humans, they have assumptions that are not necessarily present in the use of artificial intelligence (AI). For example, it is assumed that when humans are responsible for their actions, not only the decision mechanism but also the reasons to which the decision mechanism responds are the agent’s own.
This is crucial because, as I have argued when we attribute responsibility to AI systems (following the approach of Santoni De Sio and van den Hoven), the assumption that reasons belong to certain human agents is not necessarily true.
Such an assumption has also been challenged by others in recent years, including in the argument presented by Veluwenkamp (2022). He argues that being reason-responsive is incompatible with moral theories that consider agent-neutral normative reasons. To this end, he distinguishes between agent-dependent reasons and agent-neutral reasons. Then he argues that tracking is important in establishing who is responsible, and if all reasons are agent-neutral, tracking cannot play its role. He concludes that if normative reasons are agent-neutral, then not all reasons traced will be normative.
Veluwenkamp’s argument, by applying agent-neutral and agent-dependent division in Santoni de Sio and Van Den Hoven’s MHC conditions, implicitly emphasizes that it is possible for a mechanism to track reasons that do not belong to the agent(s) that are supposed to be responsible. This conflicts with the assumption that reasons always belong to the agent.
Although it has been claimed that the tracing condition requires the reasons being tracked to be partially agent-dependent, the claim does not guarantee the opposite. Tracking reasons does not necessarily require tracing an agent. Therefore, the claim allows the mechanism to track only agent-neutral reasons.
So according to the proposed approach, tracking can be realized as a normative reason-responsive mechanism, and it can be completely agent-neutral. In other words, a normative reason-responsive mechanism can be realized without any connection with an agent, and the agent-neutral reason-responsive mechanism, although it does not guarantee to trace an agent, makes the possibility of being only responsive to normative agent-neutral reasons.
I have argued that the possibility of two different interpretations of the historical nature
of responsibility (and indeed the tracing condition) raises the hope that, at least in one interpretation, there is the possibility that an agent-neutral reason-responsive mechanism also has the tracing condition.
It is argued that the relationship between an agent and reasons, (or an agent and a reason-responsive mechanism) can be differently interpreted. In other words, the relationship between a reason-responsive mechanism and an agent can be constructed during the process of “taking responsibility.” So, there should be a social process during which beliefs are constructed in a way that an agent-neutral reason-responsive mechanism becomes an agent’s own.
Conclusion
Although in Fischer and Ravizza’s approach, both the decision mechanism (based on second condition) and the reasons (as a presupposition) are an agent’s own, in the case of autonomous weapons, it can be intuitively assumed that only one of these two (that is, either the decision mechanism, or normative reasons that it tracks) during a process will be an agent’s own, and based on that, the responsibility for the results of using an autonomous AI weapon can be attributed to the agent.
Regarding this, we can expect that the two conditions of Santoni de Sio and van den Haven will be fulfilled, without the reasons used going beyond the normative reasons (which are the reasons justifying actions based on a moral theory).