Feature

How ‘AI killer robots’ are threatening global security

Think thank SIPRI and GlobalData analysts unpack how the use of AI by the militaries raises an urgent need for regulation. Kris Cooper reports.

AI use in weapons systems has profound humanitarian consequences. Credit: metamorworks/Shutterstock

The threat from artificial intelligence (AI) autonomous weapons and the need for international cooperation to mitigate the potentiality of “AI killer robots” was re-emphasised at the recent Humanity at the Crossroads: Autonomous Weapons Systems and the Challenge of Regulation conference in Vienna, Austria. 

Allowing AI control over weapons systems could mean targets being identified, struck and killed without human intervention. This raises serious legal and ethical questions. 

Highlighting the gravity of the situation, Austria’s foreign minister Alexander Schallenberg said: “This is the Oppenheimer Moment of our generation.” 

Current use of AI killer robots

Indeed, to what extent the genie is already out of the bottle is a question in itself. Drones and AI are already widely used by militaries around the world. 

GlobalData defence analyst Wilson Jones tells Army Technology: “The use of drones in modern conflict by Russia and Ukraine, by the US in targeted strike campaigns in Afghanistan and Pakistan and, as recently revealed last month, as part of Israel’s Lavender programme, shows that AI’s ability to process information is already being used by world militaries to increase striking power.” 

Investigations from The Bureau of Investigative Journalism into drone warfare of US brought to light the repeated airstrikes by the US military killing civilians in Pakistan, Afghanistan, Somalia and Yemen. More recently, the IDF’s Lavender AI system has been used to identify tens of thousands of targets, with civilians killed as a result of the strikes. 

Sources quoted in a report by +972 said that, at the start of the IDF’s assault on Gaza, it permitted the deaths of 15 or 20 civilians as collateral for strikes aimed at low-ranking militants, with up to 100 allowed for higher-ranking officials. The system has been said to have a 90% accuracy rate in identifying individuals affiliated with Hamas, meaning that 10% are not. Moreover, deliberate targeting of militants in homes has reportedly occurred, resulting in entire families being killed at once due to the AI’s identification and decisions. 

A threat to global security

The use of AI in this way emphasises the need for regulation of the technology in weapons systems. 

Dr Alexander Blanchard, senior researcher for the Governance of Artificial Intelligence programme at the Stockholm International Peace Research Institute (SIPRI), an independent think tank focussing on global security, explains to Army Technology: “The use of AI in weapon systems, especially when used for targeting, raises fundamental questions about us – humans – and our relationship to warfare, and, more particularly our presumptions of how we may exercise violence in armed conflicts.” 

“AI changes the way militaries select targets and apply force to them. These changes raise in turn a series of legal, ethical and operational questions. The biggest concern is humanitarian.” 

“There are big fears amongst many that depending on how autonomous systems are designed and used they could expose civilians and other persons protected under international law to risk of greater harm. This is because AI systems, particularly when used in cluttered environments may behave unpredictably, and may fail to accurately recognize a target and attack a civilian, or fail to recognize combatants who are hors de combat.” 

Elaborating on this, Jones notes that the issue of how culpability is determined could be called into question. 

“Under existing laws of war there is the concept of command responsibility,” he says. “This means that an officer, general, or other leader is legally responsible for the actions of troops under their command. If troops commit war crimes, the officer bears a responsibility even if they did not give orders, the burden of proof falls on them proving they did everything possible to prevent war crimes.” 

“With AI systems, this complicates everything. Is an IT technician culpable? A system designer? It’s unclear. If it’s unclear, then that creates a moral hazard if actors think their actions are not covered by existing statutes.” 

Historical arms control conventions

Several major international agreements limit and regulate certain uses of weapons. There are bans on the use of chemical weapons, nuclear non-proliferation treaties and the Convention on Certain Conventional Weapons, which bans or restricts the use of specific types of weapons that are considered to cause unnecessary or unjustifiable suffering to combatants or to affect civilians indiscriminately. 

“Nuclear arms control required decades of international cooperation and treaties after that to be enforceable,” explains Jones. “Even then, we continued to have atmospheric tests until the 1990s.” 

“A major reason anti-proliferation worked was because of US-USSR cooperation in a bipolar world order. That doesn't exist anymore, and the technology to make AI is already accessible by many more nations than atomic power ever was.” 

“A binding treaty would have to sit everyone involved down at a table to agree to not use a tool that increases their military power. That isn't likely to work because AI can improve military effectiveness at minimal financial and material costs.” 

Current geopolitical stances

While the need for the responsible use of AI by militaries has been recognised by states at the UN, there is still some way to go. 

Laura Petrone, principal analyst at GlobalData, tells Army Technology: “In the absence of a clear regulatory framework, these declarations remain largely aspirational. It doesn't come as a surprise that some states want to retain their own sovereignty when it comes to deciding on matters of domestic defence and national security, especially in the context of the current geopolitical tensions.” 

Petrone adds that, while the EU AI Act does lay out some requirements for AI systems, it does not address AI systems for military purposes. 

“I think that despite this exclusion, the AI Act is an important attempt to establish a long overdue framework for AI applications, which could lead to a certain level of alignment of relevant standards in the future,” she comments. “This alignment will be critical to AI in the military domain as well.” 

Go to article: Home | Get ready for ESG 2.0Go to article: ContentsGo to article: BriefingGo to article: Foreword: A growing sense of urgency in ESG Go to article: ESG 1.0 is over – get ready for ESG 2.0 Go to article: ESG becomes mandatory: how to prepareGo to article: Theme timeline: the past, present and future of ESG Go to article: GlobalData’s Christopher Papadopoullos on net-zero strategy for businesses Go to article: 38% of companies lack an ESG strategy – GlobalData survey Go to article: Why sustainability reporting standards must be simple, consistent and transparentGo to article: HealthcareGo to article: Net-zero strategies for the pharma sector Go to article: Leading pharma companies and initiatives in the race for net zeroGo to article: Case study: Medtronic MiniMed extended infusion set a step forward for sustainability Go to article: Latest news: ESG in healthcareGo to article: Sustainability challenges in rare disease trials remain, but change is happeningGo to article: ESCMID 2024: The infectious consequences of climate change Go to article: Can pharma overcome generative AI’s bias problem?Go to article: China-US tensions are spilling into the biotech sector Go to article: World Air Quality Report: what are the health impacts and where is it worst? Go to article: Deal activity related to ESG in the pharma industry since 2021Go to article: Bring your own device: How patients own tech is being used in clinical trialsGo to article: Another recall: does Philips have a quality control problem? Go to article: FemTech: the world’s largest ‘niche’ market Go to article: Are therapists safe from AI disruption? Go to article: Japanese patients among least empowered in the developed worldGo to article: Why has New Zealand U-turned on the world’s first smoking ban?Go to article: Deal activity related to ESG in the medical device industry since 2021  Go to article: Sponsorship opportunitiesGo to article: PowerGo to article: Net-zero strategies for the power sector Go to article: Leading power companies in the race for net zeroGo to article: Latest news: ESG in power Go to article: ‘Sheer scale of the challenge’ the biggest barrier to net zero in APAC Go to article: How can AI help us adapt to an evolving climate? Go to article: How “energy-hungry AI” is straining grids Go to article: Innovation in green hydrogen driving surge in investment Go to article: Can natural hydrogen play a role in net zero? Go to article: Wind turbines are ageing – what happens next? Go to article: What are sustainability-linked loans and why are they struggling? Go to article: Deal activity related to ESG in the power industry since 2021Go to article: Oil & GasGo to article: Net-zero strategies for the oil & gas sector Go to article: Latest news: ESG in oil & gas Go to article: New report accuses fossil fuel companies of greenwashing, but profits are up Go to article: MethaneSat will remind the oil and gas sector “to get back to work”Go to article: How will MethaneSat help to reduce methane emissions? Go to article: How realistic is a global fossil fuels tax to aid the green transition? Go to article: “Britain has become more unequal”: Greenpeace on green transition tax Go to article: Deal activity related to ESG in the oil and gas industry since 2021  Go to article: MiningGo to article: Net-zero strategies for the mining sector Go to article: Leading mining companies in the race for net zeroGo to article: Latest news: ESG in miningGo to article: GlobalData’s Thomas Pothalingam on the critical battery industryGo to article: Pentagon looks to AI to predict prices of critical minerals for weaponsGo to article: McKinsey’s Roland Rechtsteiner on the energy transition and commodities marketsGo to article: Deal activity related to ESG in the mining industry since 2021Go to article: GlobalData Thematic IntelligenceGo to article: DefenceGo to article: Net-zero strategies for the defence sectorGo to article: Leading defence companies in the race for net zero Go to article: Latest news: ESG in defenceGo to article: Toxic artillery: Paradigm Shift Technologies on chrome plating Go to article: How is Russia weaponising ‘ecocide’ in Ukraine? Go to article: Environmental damage rampant across Ukraine and Gaza – CEOBS Go to article: What would orbiting nuclear weapons mean for the space economy? Go to article: Ethical constraints the biggest challenge for AI in defence Go to article: OpenAI faces protests for military applications, but the market is growing Go to article: How ‘AI killer robots’ are threatening global securityGo to article: Deal activity related to ESG in the aerospace & defence industry since 2021  Go to article: AutoGo to article: Net-zero strategies for the automotive sectorGo to article: Leading auto companies in the race for net zeroGo to article: ESG News in the Automotive sectorGo to article: GKN Automotive: Sustainability “is constantly evolving”Go to article: “Zero Emission Vehicle” – the biggest automotive fib?Go to article: How China’s EV battery tech is cementing it as the auto industry leaderGo to article: Carmakers face allegations of Uyghur forced labour in ChinaGo to article: Deal activity related to ESG in the automotive industry since 2021Go to article: ApparelGo to article: Net-zero strategies for the apparel and retail sector Go to article: Sustainability solutions for apparel companies Go to article: Latest News: ESG in apparelGo to article: Are fashion brands on track with ESG goals? Go to article: WRAP SVP: EU’s ESG leadership sets tone for US fashion brands Go to article: Fashion’s polluting title debunked but tackling toxic impact remains vital Go to article: The greenwashing crackdown and how fashion brands can ensure compliance Go to article: Gen Z’s fraught relationship with ESG-friendly missions Go to article: Social media ‘coolness’ trumps sustainability for retail consumers Go to article: Are mushrooms the future of leather? Go to article: Deal activity related to ESG in the apparel industry since 2021Go to article: ConsumerGo to article: Net-zero strategies for the consumer goods sectorGo to article: Case study: Kraft Heinz green hydrogen plantGo to article: Latest news: ESG in foodGo to article: Is Unilever’s bold ESG plan enough to hit net zero? Go to article: The problems with ‘carbon negative’ claimsGo to article: Why claims of ‘sustainable’ beef should be taken with a pinch of saltGo to article: Cocoa prices continue to rise. When might the tide turn? Go to article: EU pesticide bans: what are the impacts and are they necessary? Go to article: Why are UK infant formula prices rising – and what will the CMA do about it?Go to article: Deal activity related to ESG in the food industry since 2021Go to article: Latest news: ESG in drinksGo to article: Preparing for the “second wave” Go to article: Can genetic breeding ease brewers’ climate concerns?Go to article: Deal activity related to ESG in the beverages industry since 2021Go to article: PackagingGo to article: Consumer goods companies focus on sustainable packaging Go to article: Latest news: ESG in packaging Go to article: The current state of ESG in the packaging industry Go to article: Judging a book by its cover Go to article: Key players shaping sustainable packaging trends Go to article: Interview Enhancing ESG through innovative food packaging solutions Go to article: Eco-packaging solutions for perishable goods in transit Go to article: Why manufacturers need to take the circular economy seriously Go to article: Microplastics in placentas: how serious is the problem? Go to article: Global trend of plain tobacco packaging gains momentum Go to article: Deal activity related to ESG in the packaging industry since 2021  Go to article: BankingGo to article: Net-zero strategies for the financial services sector Go to article: Leading financial services companies in the race for net zero Go to article: Latest news: ESG in banking Go to article: Sustainable investing in a tech-driven future Go to article: “ESG is good business” – UNEP FI’s Eric Usher Go to article: Financial services firms must focus on Scope 3 to achieve net zero Go to article: Why the payment industry needs to “go bespoke” to deliver true financial inclusiGo to article: How will the EU’s AI act impact financial services? Go to article: What is digital cash and why are there privacy concerns around it? Go to article: Social media continues to disrupt the banking industry Go to article: Deal activity related to ESG in the retail banking industry since 2021Go to article: Next issue