Presentation at the Campaign to Stop Killer Robots side event at the CCW, Laura Boillot

13 November 2014, Geneva

Next steps on addressing autonomous weapons systems

In my presentation today I am going to look at three issues:

  • the principle of meaningful human control,
  • why this principle provides a useful framework to approach autonomous weapons systems,
  • and some next steps on this issue to move us forward.

In current practice, there is an expectation that human control is exercised over when, where and how weapons are used, and over their effects.

Whilst the principle of human control is implicit in existing international law governing the use of force, currently there is no explicit requirement for it.

As such, the concern is that with the development of increasingly autonomous weapons systems, future systems may come to erode what we expect and consider is necessary in terms of human control – and that weapons systems could be developed that operate without meaningful human control.

Whilst no state would likely support the development of an autonomous weapon that operates without any human control, it is also apparent that having a person “in”, “on”, or “touching” the loop does not mean that adequate human control is being exercised.

We are also seeing a trend towards increasingly autonomous functions in weapons systems such as over the detection, selection and engagement of targets, where human control may be detached, reduced, or removed.

In recent discussions, including here at the CCW, states have used terms such as meaningful / adequate / sufficient human control or human judgement, and it would be useful to better understand how states understand these terms, and how control is exercised over existing weapons systems, as this can inform our orientation to future weapons systems.

Furthermore, at the CCW expert meeting on ‘Lethal Autonomous Weapons Systems’ in May, the Chair’s report noted that “many interventions stressed that the notion of meaningful human control could be useful to address the question of autonomy”.

Moving forward, this principle should be considered as a useful framework to explore this issue further.

Ensuring meaningful human control over individual attacks can be understood to include the process of predicting the outcome of an attack. This would mean having information on the context, including the military objective and on protected persons and objects. It would also mean having information on the attack process including target detection, selection and engagement, and on the weapons effects.

But, ensuring the principle of meaningful human control is not only about technical issues such as the predictability of the outcome of an attack or the behaviour of a machine in a particular environment.

It should also be used to weigh up this information, and inform the decision-making process of whether it is acceptable to launch an attack or not. This includes moral and ethical considerations and responsibility, and legal considerations.

In our view, a useful way of moving this issue forward is:

  • For states to acknowledge that the requirement for meaningful human control over every individual attack is both implicit in existing law, and also a necessary requirement. This would provide common ground for collectively moving forward.
  • For states to acknowledge that developing autonomous weapons systems that operate without meaningful human control over individual attacks is of particular concern.

States could then consider how to formulate the policy and legal framework needed to ensure every attack remains under meaningful human control in the future.

This process should include sharing national experience of current weapons systems that detect and engage targets. These weapons challenge MHC and we hope states can come to the next meeting with concrete examples of how they manage these systems to ensure their use is permissible, acceptable and appropriate. In turn, this should inform us of the sorts of concrete controls needed.

Whilst it has been positive to see how quickly the issue of LAWs been picked up by states and put on the agenda of the CCW, it is also important to see clear progress over the next 12 months.

More discussions on this issue are needed. We hope and expect that the mandate for future work will be agreed tomorrow.

This should mean more substantive engagement on all aspects of the issue.

These discussions should be inclusive – including civil society, women experts, and involving a broad range of states.

And crucially, discussions should lead towards developing clear new rules that set a strong international standard.

Posted in: Autonomous weapons, Conference statements,
Tagged: , ,