Can AI systems make moral decisions in war?

Artificial intelligence is reshaping how wars are fought. From autonomous drones to AI-powered command-and-control, machines are paving the way for fewer decision-makers in the war room and boots on the ground.

That transformation is already reshaping conflicts by introducing new technologies in Kyiv, Gaza and Tehran. AI-powered interceptor drones in Ukraine. Israel’s AI-enabled target generation used in Gaza. And, most recently, Palantir’s Maven Smart System (MSS) used to identify and strike thousands of targets during the US-Israel war on Iran.

On February 28, the first day of the US-Israel war on Iran, Maven helped to strike more than 1,000 targets, among them an Iranian primary school in Minab where more than 150 people were killed, most of them children. Four months later, the Pentagon’s investigation of the attack remains stalled, with senators demanding answers and Bloomberg reporting that the error traced back to outdated satellite intelligence.

As AI becomes firmly embedded in militaries, the question isn’t just whether these systems can be trusted to make accurate decisions, but whether they can grapple with the moral consequences of the decisions they make.

As part of a series on AI and warfare, Al Jazeera examines if it can successfully replace human decision-makers.

Decision advantage: The rise of AI on the battlefield

Global military expenditure rose 2.9 percent last year to $2.87 trillion. Within that figure, spending on AI for military applications is accelerating fast, projected to grow from $11.7bn to nearly $19.3bn by 2030, according to industry reports.

In April, the administration of US President Donald Trump proposed $1.5 trillion in defence spending for the fiscal year beginning in October to build a “dream military” and accelerate the shift from a traditional military-industrial complex to a military-tech one.

The US Pentagon has set its sights on becoming an “AI-first fighting force,” compressing the OODA loop – a decision-making model developed by US Air Force Colonel John Boyd in the late 20th century, built on the premise that whoever moves faster through decisions comes out on top.

Fed continuously with new information, the OODA loop – observe, orient, decide, act – lets commanders decide actions at the latest possible moment, trading certainty for speed and adaptability.

Driving the push for speed is Project Maven, Palantir’s flagship AI intelligence platform, paired with Anthropic’s Claude, built to offer a real-time picture of the battlefield.

The OODA loop’s logic was already at work in the drone wars of the 2000s and 2010s, when algorithmic pattern recognition flagged individuals whose behaviour matched a suspicious profile, feeding computationally generated kill lists.

It wasn’t called AI then, but the logic was the same as today’s: find patterns in data faster than a human analyst, and convert them into action.

How does the Maven Smart System work?

To understand what Maven does, it’s worth considering what it has changed. In previous conflicts, such as the 2000s US wars in Iraq and Afghanistan, analysts tracked adversarial networks in Excel spreadsheets and PowerPoint slides, logging names and sketching out connections, while kill chains relied on printed dossiers analysed by senior officials. These processes took time.

During Operation Epic Fury, as the current US war on Iran is called, Maven has been used to process thousands of strikes in minutes. Timelines that once took hours or days have been compressed to mere seconds, according to Admiral Brad Cooper, head of US Central Command.

Project Maven was launched in 2017 as the Algorithmic Warfare Cross-Functional Team (AWCFT), to use AI and machine learning to automatically sort through the massive backlog of drone and surveillance footage that the military was collecting but couldn’t fully analyse — from identifying and prioritising the target to selecting the appropriate weapon and finally assessing the battle damage.

Google built computer vision models to analyse drone footage, until it withdrew in 2018 following staff protests in which more than 4,000 employees signed a petition stating that “Google should not be in the business of war.”

Today, more than 20,000 US military personnel use Maven across 35 military software tools and three security classification domains, according to Vice Admiral Frank Whitworth, director of the country’s National Geospatial-Intelligence Agency.

The system pulls data from more than 150 resources such as:

- Satellites

- Commercial radar constellations returning imagery through cloud cover and darkness

- Drone videos

- Signals intelligence (SIGINT)

- Radio emissions

- Social media geolocation

- Field reports from deployed units

Once identified, a target moves through a digital pipeline modelled on the military kill chain sequence of find, fix, track, target, engage, assess (F2T2EA), carrying coordinates, intelligence sources, risk assessments and legal sign-offs at every stage.

An AI recommender proposes weapons and munitions based on proximity and suitability. Anthropic’s Claude, integrated through Palantir’s platform, lets operators query the system in natural language.

Anthropic was the first “frontier” AI company to deploy its models on the classified networks that Maven runs on. However, tensions erupted in March when the Pentagon blacklisted Anthropic and designated it a “supply-chain risk” after it refused to loosen Claude’s restrictions on autonomous weapons and surveillance, prompting Anthropic to sue the government.

Removal of decision-makers

According to the US Department of Defense, no AI weapons will be fully autonomous, and they will always require human authorisation.

Admiral Cooper said in March that “Humans will always make final decisions on what to shoot and what not to shoot and when to shoot, but advanced AI tools can turn processes that used to take hours and sometimes even days into seconds.”

However, some academics are less reassured for this very reason. What may seem like frictionless workflows, compressed timelines and “pre-packaged” decisions also carry the risk of eroding the conditions under which meaningful human judgement and moral decision-making can occur.

“There is an implicit tension in the mandate for moral deliberation – ethical deliberation, legal deliberation – which takes time, and which requires a different way of thinking about action,” Elke Schwarz, professor of political theory at Queen Mary University London and author of the book Death Machines: The Ethics of Violent Technologies, told Al Jazeera.

“You’re saying: we’re going to sacrifice a more rigorous deliberative process in the interest of speed and scale.”

Can military AI ethically make human decisions?

The deeper question is whether AI can, even in principle, replicate moral reasoning.

Schwarz is categorical that it cannot, and not for lack of sophisticated large language models (LLMs), which are used to run queries and help make sense of data in simple natural language.

Just war theory was developed because of what’s at stake in decisions to use lethal force. The framework exists to force restraint, born from hard-won understanding of war’s true costs. Schwarz says it was never meant to be a checkbox exercise.

“Ethics is a social practice,” Schwarz says. “It rests on the fact that we take each other’s vulnerability very seriously, and that we trust one another not to violate that unless circumstances dictate.

“A system is a computational system. It has no concept of the meaning of human life. It has no concept of mourning, of suffering.”

Schwarz dismisses talk of agentic AI – systems capable of pursuing tasks semi-autonomously with limited human oversight – “displaying” fear or emotion as a marketing gimmick, instead asserting that “grappling with principles” and doubt, and “weighing of how many lives are at stake” cannot simply be programmed into a system.

A need for restraint

Analysts say there is a real risk in military AI, where outputs that sound plausible are wrong, built on data that is incomplete, outdated or corrupted – the kind of failure that is reported to have driven Maven’s erroneous strike on the Minab primary school in Iran. Maven’s actual error rate has not been disclosed, but estimates hint at a system that is far less reliable than purported by the Pentagon.

Schwarz says large language models simply shouldn’t be used for targeting, since they will inevitably hallucinate and contribute to civilian harm.

The Intercept, an independent investigative outlet, reported on Monday that the Pentagon slashed its Civilian Protection Center of Excellence from 40 staff to just nine while leaning on AI tools, including one built on the Maven Smart System, to speed up civilian harm assessments.

Research from the Max Planck Institute for Human Development noted in a study that delegating a task to AI creates a moral distance that makes people more willing to engage in behaviour they wouldn’t otherwise themselves, since the AI can lend a veneer of legitimacy, with existing safeguards doing little to stop it.

Told three years ago that LLMs would be used in military targeting by 2026, Schwarz said: “I would have said that is lunacy. And I will still say that today.”

“To pretend that an AI system can be a moral decision-maker constitutes, for me, an abdication of this uniquely human task – to weigh, understand the weight, feel the weight of a morally difficult decision for which one might bear the burden of responsibility.”

Can AI systems get unethical militaries to act ethically?

Conversely, the question arises of whether AI could practically constrain less ethical militaries by imposing restraint from above.

In 2024, the Israeli-Palestinian publication +972 Magazine and Hebrew-language media outlet Local Call reported that the Israeli army was isolating and identifying thousands of Palestinians as potential bombing targets using an AI-assisted targeting system called Lavender.

It had an estimated error rate of about 10 percent. Human rights experts and legal scholars warned the practice risked violating international humanitarian law, with human oversight in some cases reduced to a matter of seconds before a strike was authorised.

Schwarz is equally sceptical of AI being able to constrain militaries. “If a military has a system that scales up and speeds up the violence they want to enact, they’re not going to abdicate to a system that says ‘no, you shouldn’t do that,'” she said. “There’s already a problem under way that needs to be addressed beforehand.

“An AI system will not act as a moral decision-maker over an unethical human.”

Reporting originally appeared via Al Jazeera. Read the full source for additional context.