Rights At The Mercy of AI: How AI-Enabled Security Is Putting Human Rights at Risk

Rights At The Mercy of AI: How AI-Enabled Security Is Putting Human Rights at Risk

•

Reading Time: 7 minutes

Artificial intelligence is now part of the modern cybersecurity nucleus and security operations. Used well, it protects networks. Used carelessly or deliberately against people, it turns into a ruthless tool to undermine the right to life, privacy, free expression, equality and remedy.

On the first day of the US-Iran war, two missiles struck an elementary school in Minab. According to Bloomberg’s reporting, more than 150 people died; at least 123 of them were children. Bloomberg’s sources say that US databases still listed the compound as a military site years after satellite imagery showed a school, and that Palantir’s Maven targeting system recommended it among the day’s targets. Whether the fault lies with the software, the data or the institution is disputed. The dead children are not. No justice can bring them back on their school desks.

Why security AI is different

Three important features explain security AI’s exponential potential for harm. First is its scale: models can monitor millions of messages or camera feeds continuously, removing the cost that once limited the surveillance. Second is the inference: machine learning does not just record behaviour; it predicts it, assigning risk scores and labelling people as likely associates of a target. Third is the opacity: these systems are proprietary or classified, so the people judged by them rarely learn they were assessed, let alone how to challenge it.

International law has a clear test that any interference with rights must be lawful, necessary and proportionate, but law hardly matters when it is the power itself that decides to assert its will as a world order. Much AI-enabled security fails the first part for want of a clear legal basis, and the last because proportionality cannot be judged when no independent body can inspect the system. Cybersecurity is also an unusually persuasive excuse. Ransomware and attacks on hospitals are real, so demands for deeper monitoring are easy to defend and hard to scrutinise.

The right to life: when an error is a body count

War shows the stakes most starkly. The Minab strike, in reporting based on sources, followed a chain of preventable failures, including an outdated database, a warning note from an analyst that was never linked to it, civilian-harm mitigation staff who had been cut, and a system that ranked the site as a target. According to the Bloomberg report ‘Inside the US “Kill Chain” That Destroyed an Iranian School’, an unreleased Pentagon review reportedly cited overreliance on Maven, bad intelligence and outdated imagery as having contributed to the missile strike that killed 123 children in Minab. A UN fact-finding mission found reasonable grounds to believe the strike was an indiscriminate attack and a war crime. Palantir says it is not responsible for the underlying data, and the Pentagon says its investigation continues. A war crime is there, but establishing the criminal responsibility would be a challenge; there is no penalty that can be given to AI, after all.

The nuance matters. The legal scholar Rebecca Crootof argues in Lawfare that the failure was a flawed target-selection process fed by bad data, not AI as such. But AI speeds up and scales such errors, and keeping a human in the loop is no cure because of the human tendency to trust and over-rely on recommendations, outputs or decisions generated by automated systems or artificial intelligence, as the 1988 USS Vincennes shoot-down arguably showed, when the Navy guided-missile cruiser shot down Iran Air Flight 655, a civilian Airbus A300, over the Persian Gulf, killing all 290 people on board (including 66 children). Commentators also speculated about the role of Anthropic’s Claude Gov, because US Central Command used Palantir’s Maven Smart System, which integrates Anthropic’s Claude models to process intelligence and generate target lists at high speeds during the initial strikes on Iran. Crootof rejects blaming the technology.

A second case came close to a different kind of disaster. CNN reported in September 2026, citing four anonymous sources, that an analyst at US Special Operations Command Pacific used a chatbot to blend open-source and classified intelligence, then had it formatted as a standard report. The report claimed a Chinese-flagged ship in the Middle East carried nuclear weapons components. One source called it entirely false. Armed personnel were reportedly preparing to board, with aircraft in the air, when the report was traced to its AI origin and the operation aborted. Officials did not respond to CNN, so the account is unconfirmed. No nuclear weapon was involved; the danger was a US-China confrontation sparked by fabricated intelligence that could lead to a disastrous chain reaction globally.

In Gaza, +972 Magazine and the Hebrew-language media outlet Local Call reported, from anonymous Israeli military sources, that a system called Lavender scored residents from 1 to 100 for likelihood of militancy, with a tolerated false-positive rate of about 10% and human checks lasting around 20 seconds. According to Al Jazeera, that database is responsible for drawing up kill lists of as many as 37,000 targets.

The common thread is rarely exotic machine ‘hallucination’. It is stale data, unchecked outputs and compressed human review, which AI accelerates and civilians pay for.

Pre-AI remote killing: drone strikes

Although the unmanned aerial vehicle (UAV) used for drone strikes during the War on Terror was being operated by humans remotely, the underlying questions of human rights were the same.

Behavioural ‘signatures’: Instead of targeting a named, specific individual (known as a ‘personality strike’), the US military and the CIA utilised signature strikes to target groups of people exhibiting pre-defined ‘signatures’ or suspicious patterns of behaviour associated with militant or terrorist activity such as gathering in specific combat-prone areas, carrying weapons or travelling in convoys. The decision-makers failed to acknowledge that in countries like Afghanistan, carrying weapons and travelling in convoys is the default social character.

‘Military-age males’: Human rights investigations revealed that under certain operational guidelines, particularly during the Obama and Bush administrations, the criteria for these strikes frequently broadened to include broad demographic categories, most notoriously ‘military-age males’ (MAMs). This meant men and boys within a specific age range (generally defined as those aged 16 and over) found in designated militant-influence zones were sometimes presumed to be combatants unless proven innocent after the fact.

Human rights organisations and legal scholars have heavily criticised signature strikes because targeting individuals whose identities are completely unknown dramatically increases the risk of misidentification, resulting in the deaths of civilians, farmers, first responders and local villagers. Despite being controlled by humans remotely, the generalised targets caused devastation for civilian lives. Now, replacing humans with AI as decision-makers has the potential to be lethal beyond reckoning.

Expression and association: watched, then silenced

Generative AI is lowering the cost of reaching critics abroad. A September 2026 Anthropic threat-intelligence report, discussed at Just Security, documented several cases of AI-enabled digital transnational repression. The UN Special Rapporteur on freedom of assembly has described China as an extreme case of surveillance ubiquity, and recorded Tibetan defenders’ reports of state-linked network and SIM-cloning attacks. Human Rights in China has noted a rise in AI-powered harassment, including deepfakes aimed at female activists.

In Myanmar, civil society groups have told UN mechanisms that AI-powered cameras and spyware are used to hunt journalists and activists. Elsewhere, deep-packet-inspection systems in states such as Russia and Iran show how defensive filtering can identify dissenters and censor speech. The deepest harm may be invisible: lawyers, reporters and organisers who assume they are being watched stop meeting, stop writing and stop trusting their devices.

Equality and liberty: the wrongly flagged

CIVICUS’s 2026 report warns that live facial recognition operating without meaningful human oversight, introduced on security grounds, normalises intrusive surveillance and pre-emptive policing. The cost of a wrong match is concrete: Robert Williams was wrongfully arrested in Detroit in 2020 after a facial-recognition mismatch.

The harm also reaches workplaces and private messages. Behavioural analytics sold as insider-threat detection can sweep in union organising, whistleblowing or health conditions. Proposals for client-side scanning, in which AI classifiers run on a device before a message is encrypted, recast mass inspection as a safety feature. The European Court of Human Rights signalled the limits of that logic in Podchasov v Russia (2024), holding that obliging providers to weaken encryption violates the right to private life.

Remedy: why the law is behind

The EU AI Act is the most ambitious attempt at regulation, and its bans on social scoring, workplace emotion recognition and most real-time biometric identification matter. But the real-time ban has exceptions for serious crime, missing persons and imminent threats, and the Act excludes systems used exclusively for national security, defence or military purposes. General application began in August 2026, yet the Commission proposed in late 2025 to delay parts of the high-risk regime, and that delay is now law: Regulation (EU) 2026/1744, in force since 27 July 2026, moves most high-risk obligations to 2 December 2027 for stand-alone systems and 2 August 2028 for those embedded in regulated products.

Elsewhere the picture is thinner. Export controls rely on inconsistent enforcement. Corporate due diligence under the UN Guiding Principles on Business and Human Rights is largely voluntary. And most victims never learn they were targeted, let alone obtain evidence held in classified or proprietary systems. A right without a remedy is only a hope.

What comes next

Agentic surveillance. Autonomous agents can run reconnaissance, craft tailored phishing lures and sift stolen data with little supervision, putting once-elite capabilities in the hands of smaller states and private firms.

Synthetic harassment. Deepfakes, cloned voices and automated impersonation can discredit or entrap critics for almost nothing.

Converging data. Recorded Future warns that spyware, AI monitoring and biometric databases are merging into fuller surveillance ecosystems. Once a phone, a face and a location history are linked, anonymity in public is gone.

False intelligence at crisis speed. A fabricated report in standard format acquires unearned authority. Among nuclear-armed states, compressed decision times make every such error costlier.

What must change

  1. Assess rights before deployment. Require independent human rights impact assessments, with periodic review, before any AI security or surveillance system is procured. Make summaries public for state use, with a presumption against deployment where risks cannot be mitigated.
  2. Close the security loopholes. Narrow national-security and law-enforcement exemptions. Require prior authorisation by a judge or independent authority, limited in time and scope, and bar bulk inference about political opinion, association or other protected characteristics.
  3. Treat commercial spyware as a controlled weapon. Adopt binding export licensing with human rights criteria and real penalties, and turn the UK-French Pall Mall Process into enforceable commitments. Explain publicly any decision to lift sanctions.
  4. Tell people they were watched. Notify individuals once covert surveillance ends, publish registers of AI surveillance procurement and give oversight bodies technical staff able to audit models, data and error rates.
  5. Protect encryption and workers. Reject mandated client-side scanning, consistent with European human rights case law, and ban emotion recognition and covert behavioural profiling of employees.
  6. Hold vendors and AI developers to account. Impose mandatory human rights due diligence aligned with the UN Guiding Principles and liability for knowingly supplying abusers. Developers should monitor and disrupt misuse, publish threat reports and protect whistleblowers.
  7. Keep a human accountable in war. Label AI-generated intelligence by provenance, require independent verification before lethal or interdiction decisions, restore civilian-harm mitigation capacity, publish investigation findings and commit that AI will never decide nuclear use.
  8. Fund defenders and secure remedies. Support the forensic labs that have uncovered most documented spyware abuses, grant NGOs standing in court and shift the burden of proof where evidence sits with the state or vendor.
  9. Build global norms. Ask the UN Human Rights Council to consider a dedicated mandate on AI-enabled surveillance, and coordinate export, sanctions and standards so vendors cannot shop between jurisdictions.

None of this is an argument against AI in security. It is an argument that speed is not a substitute for judgement, and that the people on the receiving end, the schoolchildren, the journalist and the wrongly flagged, are entitled to be counted before the system is switched on. The question every deployment should answer first is not whether the technology works, but whether it is lawful, necessary and proportionate, and who answers when it fails.

Aisha Ghazi is a human rights lawyer, author, and documentary filmmaker dedicated to uncovering untold histories. Her work focuses on war and conflict, socio-psychological grounds dictating violence, refugee crisis, geopolitics and history. She is the director of iResist, a human rights platform that advocates human rights through documentary, and a Solicitor of the Supreme Court of England and Wales.