Ethical Concerns Regarding AI and Autonomous Warfare

Most ethical concerns regarding the application of artificial intelligence to autonomous warfare can be grouped in the following concerns: responsibility and accountability for deployment, malfunction and risk, the effect that autonomous weapons have on the dynamics of warmaking and national defence, and concerns about the future of humanity. The most significant concerns hold that autonomous weapons will undermine systems of responsibility and accountability for casualties and damage, that these weapons can be prone to unpredictable and erroneous behaviour. This has led to calls for autonomous weapons to be programmed with independent moral reasoning capabilities to inform how they process the rules of engagement. This article will consider arguments for and against this idea.

Responsibility and Accountability

Because weapons powered by artificial intelligence can operate autonomously, many fear that our ability to hold humans accountable for deploying them will be undermined If a weapon is free to choose its own target, then who is responsible in case of error and civilians are targeted by mistake? The developer who designed the weapon? The soldier who initiated the weapon? The general who gave the order? AI can be unpredictable, making it difficult to know what actions an autonomous weapon will take to accomplish its goals.¹

Critics argue that autonomous weapons will create a diffusion of responsibility, making it easier to shift blame, creating room for plausible deniability, and making it difficult to hold people accountable for using them. However, defenders note that within military command structures, commanders are responsible for actions taken by forces and individual soldiers are responsible for the use of their weapon.² Even in this case of an autonomous weapon, the duty to explain an attack and responsibility for using a weapon remain with human operators who must activate any weapon, whether it is semi-autonomous or fully autonomous.

Creating Moral Weapons

Responding to concerns raised, and mindful that such weapons will have increased operational autonomy over time, some have called for the development of autonomous weapons programmed with the ability to make moral decisions on their own. Within the AI sector, certain companies have already attempted to incorporate ethical intuitions into their model. Anthropic and other researchers have attempted to build “constitutional AI” which incorporates political and ethical principles for the purposes of generating more moral outcomes, and to facilitate AI’s “help to supervise other AIs.”¹

One of the arguments for imbuing morality into autonomous weapons is connected to the moral reasons that may exist for developing autonomous weapons in the first place. The argument would first suggest that we have moral reasons to want to develop autonomous weapons, such as minimizing threats and harm to human soldiers, and if this is the case, then we ought to develop such weapons as ethically possible. Autonomous weapons will also not engage in war crimes out of retribution.³ If they are equipped with the power of moral reasoning, it would also allow greater battlefield information to be processed at once, allowing for greater precision and the reduction of potential casualties.⁴

Building in ethical reasoning into autonomous weapons may also foster trust and ease their adoption. Autonomous systems can reassess outcomes based on new information, potentially making them better able to respond to uncertainties on the battlefield. According to proponents such as Ronald C. Arkin, autonomous weapons do not have to behave perfectly ethically, they only have to perform more ethically than human soldiers are capable of performing.⁵

The Case Against Moral Weapons

While counter-intuitive, building moral reasoning into autonomous weapons may be the wrong thing to do. Ironically, adding further layers of AI reasoning with ethical parameters can exacerbate the same problems that occur with autonomous weapons more generally: contributing to greater diffusion of responsibility and creating more room for error and unpredictable behaviour. If we can accept, for example, that an autonomous weapon could attack a group of civilians by mistake, then we also must accept that an AI can make mistakes even if it is programmed to behave in ethically predictable ways.

This result, in turn, risks undermining the responsibility of the operators because an additional causal decision point makes responsibility harder to assign. If we claim, for example, that responsibility ultimately does fall on the general who gave the order to fire and on the soldier for firing the weapon, then having the weapon make autonomous moral decisions after the fact is either redundant at best or undermines this clear line of responsibility at worst. An autonomous weapon will already be programmed to distinguish between permissible and impermissible targets; otherwise, it could not autonomously select its targets in the first place. Those criteria are set by humans, who remain responsible for deciding when and where the weapon may be deployed. The weapon is already following human-defined rules about which targets it may engage, so a further layer of autonomous moral reasoning adds little and risks being redundant.

If the weapon behaves in an unpredictable way, we can identify this as a malfunction of the system. Military personnel are “responsible for deploying AWS only in sanctioned environments for which they were designed and tested. And governments and international bodies are responsible for specifying the requisite level of reliability when it comes to proportionality, distinctness, and surrender.”⁶ Having a machine making additional moral choices after the fact only opens the door to undermining operator responsibility, particularly since we are no longer talking about a malfunctioning weapon designed to tell the difference between friend or foe, but a weapon that is further designed to consider the ethical merits of potentially doing harm to either.

The problem is compounded by the fact that there is no obvious consensus about what moral reasoning should mean in this context. Ethicists would likely argue that moral reasoning requires critical reasoning along multiple dimensions of thought.⁷ Still, there may be temptation to simplify the ethics in the application. While there have been attempts, as noted, to imbue moral principles into AI, we should be skeptical of attempts by weapons developers to codify ethics and treat that as a sufficient basis for moral reasoning. Additionally, any kind of international governance of autonomous weapons systems requiring the use of autonomous moral decision-making would require consensus on what kind of moral framework or moral reasoning should be employed.⁸

Conclusion

The case for morally reasoning autonomous weapons rests on the hope that greater autonomy can make warfare more precise, predictable, and humane. Yet this may mistake the problem. The central challenge posed by autonomous weapons is not simply whether machines can make better moral decisions than humans, but whether delegating such decisions to machines can preserve clear lines of human responsibility. Adding moral reasoning to an autonomous weapon risks making the weapon’s behaviour more complex, its failures harder to anticipate, and responsibility harder to assign.

← Back to Research