Ahead of the latest round of international talks on autonomous weapons systems, scheduled to take place at the UN in Geneva from 11-14 April, Article 36 has circulated a series of papers providing input to these discussions. While it seems clear that human control over individual attacks will be central to any productive outcome from the current discussions on autonomous weapons, states have approached the predominant framing of meaningful human control with varying degrees of openness. Whichever word is used to describe the nature of the human control being exerted, it should be possible for states to set out what is the necessary control over the use of weapons systems and how this is ensured.

With the recognition that human beings are the agents to which international humanitarian law is addressed, a central question will be how the introduction of autonomy in the critical functions of weapons would challenge the structure of the law as it stands. Against this background, assertions by the UK and others that no new law is required on autonomous weapons – or that the concerns are best dealt with through national level reviews of weapons – merit close scrutiny. Indeed, to be considered meaningful contributions to the international debate, such assertions would need to engage in some detail with the considerations of human control set out in the papers described here.

With this landscape in mind, the four new briefing papers look at: key elements to consider regarding maintaining meaningful human control over individual attacks; the limitations of and questions raised by proposing national legal reviews of weapons as a response to the concerns raised by lethal autonomous weapons systems; and an analysis of UK policy on lethal autonomous weapons systems. In partnership with Dr Heather Roff of Arizona State University, we are also publishing a paper on meaningful human control, artificial intelligence and autonomous weapons. A summary of these four papers is provided below.

Meaningful Human Control, Artificial Intelligence and Autonomous Weapons

In the developing international discussion, the concept of ‘meaningful human control’ has emerged as one point of coalescence. Primarily, it has been used to describe a threshold of human control that is considered necessary; however, the particulars of the concept have been left open so as to foster conversation and agreement. This paper offers a framework for considering meaningful human control, identifying key elements. The paper looks how human control needs to be embedded through mechanisms operating before, during and after use of technologies in conflict. It also addresses how meaningful human control must be applied over attacks at the most basic tactical level of warfighting, rather than simply being applied to the operational or strategic levels.

Key elements of meaningful human control

Drawing on material developed for the briefing above, this paper has been prepared as background for the expert presentation to the CCW by Richard Moyes scheduled for 12 April. The paper proposes that key elements for human control over technology are: predictable, reliable and transparent technology; accurate information for the user on the outcome sought, operation and function of technology, and the context of use; timely human action and a potential for timely intervention; and accountability to a certain standard. It notes that, as a result of the development of autonomy in the critical functions of weapons, the concept of “an attack” risks being construed more and more broadly. Against this background, the paper argues that without recognizing a requirement for human control to be in some way substantial or meaningful, the existing legal framework leaves scope for human legal judgment to be diluted to the point of being meaningless.

Article 36 reviews and addressing Lethal Autonomous Weapons Systems

A number of states have suggested that action around national legal reviews of weapons, under the framework of article 36 of Additional Protocol I to the Geneva Convention, could constitute a basis for addressing the serious concerns that states, international organisations and civil society have raised in relation to autonomous weapons. Attention to improving and widening the implementation of weapons reviews, and states’ sharing of their procedures, is welcome and necessary. This paper provides an analysis suggesting that national reviews are, however, insufficient to deal with LAWS. It argues that multilateral agreement is essential in this area in order to provide clear boundaries for all states on technologies and practices that would fundamentally alter the use of force.

The United Kingdom and lethal autonomous weapons systems

The United Kingdom has asserted that its weapons systems will always be under human control, indicating some commitment to the basic principle that underpins the concept of meaningful human control. Building on this, the UK has subsequently said that it will not develop lethal autonomous weapons systems. However, UK policy has not yet provided an explanation of what would constitute human control over weapons systems whilst at the same time suggesting a narrow and futuristic concept of LAWS that appears permissive towards the development of weapons systems that might have the capacity to operate without the necessary levels of human control. The UK is also arguing that new international law is not necessary in order to prevent the development of lethal autonomous weapons systems. No substantive rationale has been offered for this assertion, although politically it is consistent with previous UK positions at the early stages of other international weapons regulation processes. This paper provides an analysis of UK statements so far on the topic of lethal autonomous weapons systems and provides a set of recommendations to the UK government.

Download these papers

Roff Moyes MHC AI and AWS thumbnailMeaningful Human Control, Artificial Intelligence and Autonomous Weapons

Briefing paper
April 2016

2016 MHC paper thumbnailKey elements of meaningful human control

Briefing paper
April 2016

 

LAWS and A36 thumbnailArticle 36 reviews and addressing Lethal Autonomous Weapons Systems

Briefing paper
April 2016

UK and LAWS thumbnailThe United Kingdom and lethal autonomous weapons systems

Briefing paper
April 2016

 

 

Posted in: Autonomous weapons,