We Need a Legal and Ethical Framework for Lethal Autonomous Weapons

As part of a recent Defence Review, our Prime Minister has said that the UK will invest another £1.5 billion in military research and development designed to master the new technologies of warfare and establish a new Defence Centre for AI. The head of the British Army, recently said that he foresees the army of the future as an integration of “boots and bots”.

The Government however have not yet explained how legal and ethical frameworks and support for personnel engaged in operations will also change as a consequence of the use of new technologies, particularly autonomous weapons, which could be deployed by our armed forces or our allies.

The final report of the US National Security Commission on Artificial Intelligence, published this March however considered the use of autonomous weapons systems and risks associated with AI-enabled warfare and concluded that “The U.S. commitment to IHL” - international humanitarian law - “is long-standing, and AI-enabled and autonomous weapon systems will not change this commitment.”

The UN Secretary General, António Guterres goes further and argues: “Autonomous machines with the power and discretion to select targets and take lives without human involvement are politically unacceptable, morally repugnant and should be prohibited by international law”.  Yet we still have no international limitation agreement.

In company with a former Secretary of State for Defence and a former Chief of Defence Staff I recently argued in Parliament for a review of how legal and ethical frameworks need to be updated in response to novel defence technologies. This is my speech in which I pointed out the slow progress being made by the UK Government in addressing these issues.

In a written response subsequent to the debate, the Minister stated that whilst there is a NATO definition of “automated system” and “autonomous system”, the UK Ministry of Defence has no operative definition of Lethal Autonomous Weapon Systems or "LAWS".  Given that the most problematic aspect – autonomy - HAS been defined that is an extraordinary state of affairs.

A few years ago, I chaired the House of Lords Select Committee on AI which considered the economic, ethical and social implication of advances in artificial intelligence. In our Report published in April 2018 entitled ‘AI in the UK: Ready, willing and able’ we addressed the issue of military use of AI and stated that 'perhaps the most emotive and high stakes area of AI development today is its use for military purposes’ recommending that this area merited a ‘full inquiry on its own.’ (para 334)

As the Noble Lord Browne of Ladyton has made plain, regrettably, it seems not to have yet attracted such an inquiry or even any serious examination.  I am therefore extremely grateful to the Noble Lord for creating the opportunity to follow up on some of the issues we raised in connection with the deployment of AI and some of the challenges we outlined.

It’s also a privilege to be a co-signatory with the Noble and Gallant Lord HAWTON Houghton too who has thought so carefully about issues involving the human interface with military technology.

The broad context of course, as the Noble Lord Browne has said, are the unknowns and uncertainties in policy, legal and regulatory terms that new technology in military use can generate. 

His concerns about complications and the personal liabilities to which it exposes deployed forces are widely shared by those who understand the capabilities of new technology.  All the more so in a multinational context where other countries may be using technology which either we would not deploy or the use of which could create potential vulnerabilities for our troops.

Looking back to our Report, one of the things that concerned the Committee more than anything else was the grey area surrounding the definition of lethal autonomous weapon systems or LAWS. 

As the Noble Lord Browne has said as the Committee: explored the issue, we discovered that the UK’s then definition which included the phrase “An autonomous system is capable of understanding higher-level intent and direction" was clearly out of step with the definitions used by most other governments and imposed a much higher threshold on what might be considered autonomous.

This allowed the government to say “the UK does not possess fully autonomous weapon systems and has no intention of developing them. Such systems are not yet in existence and are not likely to be for many years, if at all”

Our committee concluded that, ”In practice, this lack of semantic clarity could lead the UK towards an ill-considered drift into increasingly autonomous weaponry”.

This was particularly in the light of the fact that at the UN’s Convention on Certain Conventional Weapons Group of Governmental Experts (GGE) in 2017 the UK had opposed the proposed international ban on the development and use of autonomous weapons.

We therefore recommended that the UK’s definition of autonomous weapons should be realigned to be the same, or similar, as that used by the rest of the world.

The Government in their response to the Committee’s Report in June 2018 however replied: The Ministry of Defence “has no plans to change the definition of an autonomous system”.

It did say however: “The UK will continue to actively participate in future GGE meetings, trying to reach agreement at the earliest possible stage.”

Later, thanks to the Liaison Committee we were able - on two occasions last year - to follow up on progress in this area.

On the first occasion, in reply to the Liaison Committee’s letter of last January which asked “What discussions have the Government had with international partners about the definition of an autonomous weapons system, and what representations have they received about the issues presented with their current definition?” 

the government replied:

“There is no international agreement on the definition or characteristics of autonomous weapons systems. HMG has received some representations on this subject from Parliamentarians ……” and has discussed it during meetings of the UN Convention on Certain Conventional Weapons (CCW) Group of Governmental Experts (GGE) on Lethal Autonomous Weapons Systems (LAWS), an international forum which brings together expertise from states, industry, academia and civil society. 

The GGE is yet to achieve consensus on an internationally accepted definition and there is therefore no common standard against which to align. As such, the UK does not intend to change its definition.”

So no change there my lords until later in the year……. December 2020 when the Prime Minister announced the creation of the Autonomy Development Centre to “accelerate the research, development, testing, integration and deployment of world-leading artificial intelligence and autonomous systems”

In the follow up Report “AI in the UK: No Room for Complacency” published in the same month, we concluded: “We believe that the work of the Autonomy Development Centre will be inhibited by the failure to align the UK’s definition of autonomous weapons with international partners: doing so must be a first priority for the Centre once established.”

The response to this last month was a complete about turn by the Government. They said: 

“We agree that the UK must be able to participate in international debates on autonomous weapons, taking an active role as moral and ethical leader on the global stage, and we further agree the importance of ensuring that official definitions do not undermine our arguments or diverge from our allies. 

“In recent years the MOD has subscribed to a number of definitions of autonomous systems, principally to distinguish them from unmanned or automated systems, and not specifically as the foundation for an ethical framework. On this aspect, we are aligned with our key allies. 

Most recently, the UK accepted NATO’s latest definitions of “autonomous” and “autonomy”, which are now in working use within the Alliance. The Committee should note that these definitions refer to broad categories of autonomous systems, and not specifically to LAWS. To assist the Committee, we have provided a table setting out UK and some international definitions of key terms.”

The NATO definition sets a much less high bar as to what is considered autonomous:  “A system that decides and acts to accomplish desired goals within defined parameters, based on acquired knowledge and an evolving situational awareness, following an optimal but potentially unpredictable course of action.”

The Government went on to say: “The MOD is preparing to publish a new Defence AI Strategy and will continue to review definitions as part of ongoing policy development in this area.”

Now, I apologize for taking my Noble Lords at length through this exchange of recommendation and response but if nothing else it does demonstrate the terrier-like quality of Lords Select Committees in getting responses from government.

This latest response is extremely welcome. But in the context of the Lord Brown’s amendment and the issues we have raised we need to ask a number of questions now: What are the consequences of the MOD’s fresh thinking?

What is the Defence AI Strategy designed to achieve. Does it include the kind of enquiry our Select Committee was asking for?

Now that we subscribe to the common NATO definition of LAWS will the Strategy in fact deal specifically with the liability and international and domestic legal and ethical framework issues which are central to this amendment?

If not my Lords, then a review of the type envisaged by this amendment is essential. 

The final report of the US National Security Commission on Artificial Intelligence referred to by the Noble Lord Browne has for example taken a comprehensive approach to the issues involved. The Noble Lord has quoted three very important conclusions and asked whether the government agrees in respect of our own autonomous weapons.

There are three further crucial recommendations made by the Commission:

“The United States must work closely with its allies to develop standards of practice regarding how states should responsibly develop, test, and employ AI-enabled and autonomous weapon systems.”

And “The United States should actively pursue the development of technologies and strategies that could enable effective and secure verification of future arms control agreements involving uses of AI technologies.”

And of particular importance in this context “countries must take actions which focus on reducing risks associated with AI enabled and autonomous weapon systems and encourage safety and compliance with IHL (international Humanitarian Law) when discussing their development, deployment, and use”.

Will the Defence AI Strategy or indeed the Integrated Review undertake as wide an enquiry?  Would it come to the same or similar conclusions?

My Lords, the MOD it seems has moved some way to getting to grips with the implications of autonomous weapons in the last three years.  If it has not yet considered the issues set out in the amendment, it clearly should, as soon as possible, update the legal frameworks for warfare in the light of new technology, or our service personnel will be at considerable legal risk.  I hope it will move further in response to today’s short debate.

---

by Tim Clement-Jones

COVID-19, Artificial Intelligence and Data Governance: A Conversation with Lord Tim Clement-Jones

 

BIICL June 2020

 

https://youtu.be/sABSaAkkyrI

 

This was the first in a series of webinars on 'Artificial Intelligence: Opportunities, Risks, and the Future of Regulation'.

In light of the COVID-19 outbreak, governments are developing tracing applications and using a multitude of data to mitigate the spread of the virus. But the processing, storing, use of personal data and the public health effectiveness of these applications require public trust and a clear and specific regulatory context.

The technical focus in the debate on the design of the applications - centralised v. decentralised, national v. global, and so on - obfuscates ethical, social, and legal scrutiny, in particular against the emerging context of public-private partnerships. Discussants focused on these issues, considering the application of AI and data governance issues against the context of a pandemic, national responses, and the need for international, cross border collaboration.

Lord Clement-Jones CBE led a conversation with leading figures in this field, including:

Professor Lilian Edwards, Newcastle Law School, the inspiration behind the draft Coronavirus (Safeguards) Bill 2020: Proposed protections for digital interventions and in relation to immunity certificates;

Carly Kind, Director of The Ada Lovelace Institute, which published the rapid evidence review paper Exit through the App Store? Should the UK Government use technology to transition from the COVID-19 global public health crisis; 

Professor Peter Fussey, Research Director of Advancing human rights in the age of AI and the digital society at Essex University's Human Rights Centre;

Mark Findlay, Director of the Centre for Artificial Intelligence and Data Governance at Singapore Management University, which has recently published a position paper on Ethics, AI, Mass Data and Pandemic Challenges: Responsible Data Use and Infrastructure Application for Surveillance and Pre-emptive Tracing Post-crisis.

The event was convened by Dr Irene Pietropaoli, Research Fellow in Business & Human Rights, British Institute of International and Comparative Law.

 

---

by Tim Clement-Jones

Regulating artificial intelligence: Where are we now? Where are we heading?

By Annabel Ashby, Imran Syed & Tim Clement-Jones on March 3, 2021

https://www.technologyslegaledge.com/author/tclementjones/

Hard or soft law?

That the regulation of Artificial intelligence is a hot topic is hardly surprising. AI is being adopted at speed, news reports frequently appear about high-profile AI decision-making, and the sheer volume of guidance and regulatory proposals for interested parties to digest can seem challenging.

Where are we now? What can we expect in terms of future regulation? And what might compliance with “ethical” AI entail?

High-level ethical AI principles were made by the OECD, EU and G20 in 2019. As explained below, great strides were made in 2020 as key bodies worked to capture these principles in proposed new regulation and operational processes. 2021 will undoubtedly keep up this momentum as these initiatives continue their journey into further guidance and some hard law.

In the meantime, with regulation playing catch-up with reality (so often the case where technological innovation is concerned), industry has sought to provide reassurance by developing voluntary codes. While this is helpful and laudable, regulators are taking the view that more consistent, risk-based regulation is preferable to voluntary best practice.

We outline the most significant initiatives below, but first it is worth understanding what regulation might look like for an organisation using AI.

Regulating AI

Of course the devil will be in the detail, but analysis of the most influential papers from around the globe reveals common themes that are the likely pre-cursers of regulation. It reveals that conceptually, the regulation of AI is fairly straightforward and has three key components:

• setting out the standards to be attained;
• creating record keeping obligations; and
• possible certification following audit of those records, which will all be framed by a risk-based approach.

Standards

Quality starts with the governance process around an organisation’s decision to use AI in the first place (does it, perhaps, involve an ethics committee? If so, what does the committee consider?) before considering the quality of the AI itself and how it is deployed and operated by an organisation.

Key areas that will drive standards in AI include the quality of the training data used to teach the algorithm (flawed data can “bake in” inequality or discrimination), the degree of human oversight, and the accuracy, security and technical robustness of the IT. There is also usually an expectation that certain information be given to those affected by the decision-making, such as consumers or job applicants. This includes explainability of those decisions and an ability to challenge them – a process made more complex when decisions are made in the so-called “black box” of a neural network. An argument against specific AI regulation is that some of these quality standards are already enshrined in hard law, most obviously in equality laws and, where relevant, data protection. However, the more recent emphasis on ethical standards means that some aspects of AI that have historically been considered soft nice-to-haves may well develop into harder must-haves for organisations using AI. For example, the Framework for Ethical AI adopted by the European Parliament last Autumn includes mandatory social responsibility and environmental sustainability obligations.

Records

To demonstrate that processes and standards have been met, record-keeping will be essential. At least some of these records will be open to third-party audit as well as being used for self due diligence. Organisations need a certain maturity in their AI governance and operational processes to achieve this, although for many it will be a question of identifying gaps and/or enhancing existing processes rather than starting from scratch. Audit could include information about or access to training data sets; evidence that certain decisions were made at board level; staff training logs; operational records, and so on. Records will also form the foundation of the all-important accountability aspects of AI. That said, AI brings particular challenges to record-keeping and audit. This includes an argument for going beyond singular audits and static record-keeping and into a more continuous mode of monitoring, given that the decisions of many AI solutions will change over time, as they seek to improve accuracy. This is of course an appeal of moving to AI, but creates potentially greater opportunity for bias or errors to be introduced and scale quickly.

Certification

A satisfactory audit could inform AI certification, helping to drive quality and build up customer and public confidence in AI decision-making necessary for successful use of AI. Again, although the evolving nature of AI which “learns” complicates matters, certification will need to be measured against standards and monitoring capabilities that speak to these aspects of AI risk.

Risk-based approach

Recognising that AI’s uses range from the relatively insignificant to critical and/or socially sensitive decision-making, best practice and regulatory proposals invariably take a flexible approach and focus requirements on “high-risk” use of AI. This concept is key; proportionate, workable, regulation must take into account the context in which the AI is to be deployed and its potential impact rather than merely focusing on the technology itself.

Key initiatives and Proposals

Turning to some of the more significant developments in AI regulation, there are some specifics worth focussing in on:

OECD

The OECD outlined its classification of AI systems in November with a view to giving policy-makers a simple lens through which to view the deployment of any particular AI system. Its classification uses four dimensions: context (i.e. sector, stakeholder, purpose etc); data and input; AI model (i.e. neural or linear? Supervised or unsupervised?); and tasks and output (i.e. what does the AI do?). Read more here.

Europe

Several significant proposals were published by key institutions in 2020.

In the Spring, the European Commission’s White Paper on AI proposed regulation of AI by a principles-based legal framework targeting high-risk AI systems. It believes that regulation can underpin an AI “Eco-system of Excellence” with resulting public buy-in thanks to an “Eco-system of Trust.” For more detail see our 2020 client alert. Industry response to this proposal was somewhat lukewarm, but the Commission seems keen to progress with regulation nevertheless.

In the Autumn the European Parliament adopted its Framework for Ethical AI, to be applicable to “AI, robotics and related technologies developed, deployed and or used within the EU” (regardless of the location of the software, algorithm or data itself). Like the Commission’s White Paper, this proposal also targets high-risk AI (although what high-risk means in practice is not aligned between the two proposals). As well as the social and environmental aspects we touched upon earlier, notable in this proposed Ethical Framework is the emphasis on human oversight required to achieve certification. Concurrently the European Parliament looked at IP ownership for AI generated creations and published its proposed Regulation on liability for the Operation of AI systems which recommends, among other things, an update of the current product liability regime.

Looking through the lens of human rights, the Council of Europe considered the feasibility of a legal framework for AI and how that might best be achieved. Published in December, its report  identified gaps to be plugged in the existing legal protection (a conclusion which had also been reached by the European Parliamentary Research Services, which found that existing laws, though helpful, fell short of the standards required for its proposed AI Ethics framework). Work is now ongoing to draft binding and non-binding instruments to take this study forward.

United Kingdom   

The AI Council’s AI Roadmap sets out recommendations for the strategic direction of AI to the UK government. That January 2021 report covers a range of areas; from promoting UK talent to trust and governance. For more detail read the executive summary.

Only a month before, in December 2020, the House of Lords had published AI in the UK: No room for complacency, a report with a strong emphasis on the need for public trust in AI and the associated issue of ethical frameworks. Noting that industry is currently self-regulating, the report recommended sector regulation that would extend to practical advice as well as principles and training. This seems to be a sound conclusion given that the Council of Europe’s work included the review of over 100 ethical AI documents which, it found, started from common principles but interpreted these very differently when it came to operational practice.

The government’s response to that report has just been published. It recognises the need for public trust in AI “including embedding ethical principles against a consensus normative framework.” The report promotes a number of initiatives, including the work of the AI Council and Ada Lovelace Institute, who have together been developing a legal framework for data governance upon which they are about to report.

The influential Centre for Data Ethics and Innovation published its AI barometer and its Review into Bias in Algorithmic Decision-Making. Both reports make interesting reading; the barometer report looking at risk and regulation across a number of sectors. In the context of regulation, it is notable that CDEI does not recommend a specialist AI regulator for the UK but seems to favour a sectoral approach if and when regulation is required.

Regulators

Regulators are interested in lawful use, of course, but are also are concerned with the bigger picture. Might AI decision-making disadvantage certain consumers? Could AI inadvertently create sector vulnerability thanks to overreliance by the major players on any particular algorithm and/or data pool (the competition authorities will be interested in this aspect too). The UK’s Competition and Markets Authority published research into potential AI harms in January and is calling for evidence as to the most effective way to regulate AI. Visit the CMA website here.

The Financial Conduct Authority will be publishing a report into AI transparency in financial services imminently. Unsurprisingly, the UK’s data protection regulator has published guidance to help organisations audit AI in the context of data protection compliance, and the public sector benefits from detailed guidance from the Turing Institute.

Regulators themselves are now becoming more a focus. The December House of Lords report also recommended regulator training in AI ethics and risk assessment. As part of its February response, the government states that the Competition and Markets Authority, Information Commissioner’s Office and Ofcom have together formed a Digital Regulation Cooperation Forum (DRCF) to cooperate on issues of mutual importance and that a wider forum of regulators and other organisations will consider training needs.

2021 and beyond

In Europe we can expect regulatory developments to develop at pace in 2021, despite concerns from Denmark and others that AI may become over regulated. As we increasingly develop the tools for classification and risk assessment, the question, therefore, is less about whether to regulate but more about which applications, contexts and sectors are candidates for early regulation.Print:EmailTweetLikeLinkedIn

---

by Tim Clement-Jones