Prohibiting autonomous weapons systems

Campaigners against 'killer robots' Noel Sharkey (International Committee on Robot Arms Control), Jody Williams (Nobel Women’s Initiative) and Steve Goose (Human Rights Watch)—plus an unwelcome guest. Flickr / Campaign to Stop Killer Robots. Some rights reserved.

With
states continuing to research and develop relevant weapons systems, international
action is needed to prevent the emergence of fully autonomous weapons—systems which,
once activated and their mission defined, could select people or objects to attack
without any further human intervention.

Last
week, 89 countries participated in a second informal meeting of experts on ‘lethal
autonomous weapons systems’, within the framework of the United Nations’
Convention on Certain Conventional Weapons (CCW). International organisations,
NGOs and academics, including roboticists and ethicists, contributed. This
issue has now been on the international agenda for two years, in part due to
the Campaign to Stop Killer Robots, which is calling for a treaty to ban
fully autonomous weapons.

Despite
most states rejecting weapon systems that operate without human control, a
decisive and co-ordinated response has not been forthcoming. Discussions must now
advance towards an international prohibition, before the speed of technological
development overtakes diplomatic processes.

Core issue

Whether
used in an armed conflict or law-enforcement context, fully autonomous weapons
would fundamentally challenge the relationship between human beings and the
application of violent force. The core issue remains whether the act of using
potentially lethal force should ever be delegated to hardware and software
systems, which cannot hold responsibility.

This
fundamentally dehumanising possibility was firmly rejected as unacceptable by a
large number of states and civil-society organisations last week in Geneva.
Experts, including the UN special rapporteur on extrajudicial, summary or
arbitrary executions, highlighted the affront to human dignity it would
represent.

The
discussions, chaired by Germany, were mandated to examine autonomous
weapons systems in the context of the purpose of the CCW, which is to ban or
restrict the use in armed conflict of specific types of weapons that “trouble
the conscience of humanity”. The treaty specifically recognises the need to
“continue the codification and progressive development of the rules of
international law”—as existing rules will not always be adequate in a changing
world.

Since the
latest discussions were informal, the only official outcome will be a summary report
to the next meeting of the 120 states-party to the CCW, in November. States
will make a decision then on how to proceed—including considering the option of
negotiating a new protocol on autonomous weapons systems. This makes the next
few months crucial for galvanising commitment to action.

Little information

Autonomous
weapons systems were discussed at the Human Rights Council in 2013 and at
previous meetings in the framework of the CCW in 2014. Some states have presented
policy positions to these forums but little information is available on what weapons
technology is being developed, and few states have elaborated comprehensive
policies on autonomous weapons systems. States with high-tech militaries—such
as China, Israel, Russia, South Korea, the UK and the US—are investing in
autonomy for various functions of weapons-systems technologies. But no state
has explicitly stated that it wishes to obtain ‘lethal autonomous weapons
systems’ as discussed at the CCW meeting, or that it is actively pursuing them.

A clear, co-ordinated leadership group for a ban has not yet emerged.

Many
appear still to be determining their position—or considering what they should
reveal at this stage—including how they should align with others with whom
their interests may coincide but who are not their normal allies. Attendance at
the discussions was high, due to considerable interest. There was also high interest
from media and national parliaments.

A number
of state and civil-society delegates however noted a serious
under-representation of delegations and statements from countries of the global
south, which were also less likely to have brought experts from their capitals
given the lack of a sponsorship programme associated with the CCW. Perspectives
from potential developer countries risked being over-represented.

Prohibition

Only
Israel and the US indicated last week that they felt the door should be left
open on these weapons’ development. A number of states, including France, Japan
and the UK, wished to make clear they had no intention of developing them—though
France and the UK do not support their prohibition. No states have put in place
a national moratorium, as recommended by the UN special rapporteur to the Human
Rights Council.

Those who
support a ban include Bolivia, Cuba, Ecuador, Egypt, Ghana, the Holy See, Palestine, and Pakistan. Others said prohibition or new legal regulations might have
merit, without making firm commitments. A clear, co-ordinated leadership group
for a ban has not yet emerged.

Given
that there appears to be a consensus within the CCW context that the
development of ‘lethal autonomous weapons systems’ would be undesirable, the
main point of contention is what the international response should be. A range
of critical ethical, technical, legal, strategic and regulatory concerns were
discussed over the five days of meetings in Geneva, with different options
proposed for next steps as a result.

Measures
to increase transparency and encourage national legal reviews of weapons were proposed
by some states, led by the US, to deal with the concerns raised by autonomous
weapons systems. A better understanding of technologies under development would
certainly assist international discussions by making certain issues more
concrete. Reviewing the legality of the technologies of violence states could
deploy is also an important obligation (originating in additional protocol I to
the Geneva Conventions and a customary obligation on states not party). But these
are procedures which states should be undertaking in any case, insufficient as
a specific response to autonomous weapons systems.

National-level
weapons reviews cannot give a global answer to the ethical questions raised by such
systems. Autonomous weapons would represent an unprecedented development, which
could not have been foreseen when current laws were made. Concerted
consideration by the international community is therefore required. Weapons
reviews are conducted by individual states and their results are not made open
to scrutiny. They also only examine strictly legal questions in relation to the
conduct of armed conflict, yet autonomous weapons systems should not be subject
only to legal reasoning, given the fundamental moral issues raised.

Meaningful human
control

Members
of civil society and some others have advanced the principle, to structure this
debate, of requiring meaningful human control over every individual attack. Certain
weapons systems would be prohibited as a result. In Geneva this concept was aired,
and debated, more than ever before, in both states’ statements and expert-panel
discussions. Numerous states asserted that work on autonomous weapons should
include a focus on this issue, and on reaching consensus on where the acceptable
limits of meaningful human control lie.

This
offers the most promising avenue towards a collective international response on
autonomous weapons systems. More in-depth discussion is needed, proceeding from
how states ensure sufficient control over existing weapons systems. This will
help to operationalise the principle, and can help lay the foundations for a
legal instrument prohibiting systems that do not comply.

A small
number of states doubted the utility of discussing meaningful human control, pointing
to the lack of an accepted definition, or referred to the issue using slightly
different formulations. This represents a stalling tactic. Rather than posing a
problem, the ambiguity gives an opportunity for states to build a common
concept to deal with autonomous weapons systems—as, for example, the concept of
‘unacceptable harm’ helped frame discussion on the prohibition of cluster
munitions.

A formal
process of work to prohibit fully autonomous weapons should be set in train. This
should home in on elaborating and agreeing the key elements of ‘meaningful
human control’. The 2016 review conference of the CCW will no doubt be a
significant marker. Sri Lanka, which will be CCW chair at the next meeting of
states-party, has already pointed to the precedent of the pre-emptive
prohibition of blinding laser weapons by a CCW protocol. Given the relevance of
autonomous weapons systems to law enforcement as well as armed conflict, they
should also be considered in the Human Rights Council, as a number of states
have pointed out.

Some
states argued in Geneva that both the technology and the debate on autonomous
weapons systems were in their early stages. The urgency of swift consideration
was highlighted by robotocists, who pointed out that the debate not only concerns
advanced artificial intelligence decades away but also systems using sensor
technology which could be built now or in the near future. If the consensus
approach associated with the CCW impedes take swift action, the process to
prohibit fully autonomous weapons will need to be taken forward elsewhere.

Like us on Facebook to follow the latest openSecurity articles, and tell the editors
what we should publish next.