Autonomous Weapons

In March 2020, a skirmish in Libya reportedly crossed a historic threshold. According to a United Nations report, a Kargu-2 droneKargu-2 drone autonomously tracked and attacked retreating soldiers without a human command. While the details remain disputed, the implication is profound: we have entered an era where algorithms, not humans, can decide who lives and who dies on the battlefield.

Here is the key insight: Autonomous weapons are not a distant sci-fi scenario; they are a present-day reality driven by the same technologies powering self-driving cars and facial recognition. For governance professionals, this issue highlights the extreme edge of AI risk—where the cost of a "black box" error is not a denied loan or a biased hiring decision, but a lethal strike.

You might wonder how much freedom these machines actually have. Military strategists classify autonomy into three distinct levels based on the human's role in the decision cycle.

First is Human-in-the-Loop. Here, the machine targets, but a human must authorize the strike. This is how most current systems, like Predator drones, operate. The machine provides the data; the human provides the moral agency.

Second is Human-on-the-Loop. The system selects and engages targets on its own, but a human supervisor watches the operation and can hit a "kill switch" to abort. This is common in defensive systems like the Iron Dome, where the speed of incoming missiles exceeds human reaction time.

Finally, there is Human-out-of-the-Loop. The system is activated and sent into a zone to hunt freely. Once launched, there is no human intervention. This category creates the most significant ethical and legal concerns, as it completely removes human judgment from the lethal act.

The core ethical problem with autonomous weapons is accountability. International Humanitarian Law relies on the premise that a human combatant is responsible for their actions. If a human soldier commits a war crime, they can be court-martialed.

Think of it this way: If an autonomous weapon strikes a wedding party instead of a military convoy due to a classification error, who is to blame? The programmer who wrote the code months ago? The commander who deployed the system? The manufacturer? Without a clear answer, we face an "accountability gap" where war crimes could occur without anyone being legally punishable. This mirrors the liability issues we see with autonomous vehicle accidentsautonomous vehicle accidents, but with vastly higher stakes.

Beyond ethics, there is the risk of instability. Major powers are investing billions in these technologies, driven by a fear of falling behind. As Vladimir Putin famously statedVladimir Putin famously stated, "Whoever becomes the leader in this sphere will become the ruler of the world."

This pressure creates a classic arms race. The danger is not just that these weapons exist, but that they interact at machine speed. Two opposing autonomous systems could escalate a border skirmish into a full-scale war in milliseconds, faster than any diplomat could pick up a phone. This phenomenon, known as a "flash war," is the military equivalent of the 2010 Stock Market Flash Crash2010 Stock Market Flash Crash, where algorithmic interaction wiped out trillion of dollars in minutes.

You might assume there are treaties preventing this. There are not. Efforts at the UN Convention on Certain Conventional Weapons (CCW) have largely stalled. While the "Campaign to Stop Killer Robots" advocates for a preemptive ban, major military powers prefer non-binding "codes of conduct."

For the private sector, this creates a dual-use dilemma. The technology you build for commercial purposes—like drone navigation or image recognition—can be repurposed for weaponry. This reality led to the Project Maven controversyProject Maven controversy, where Google employees protested the use of their AI technology by the Pentagon.

Here is what matters: Even if your organization does not build weapons, you must be aware of how your technology travels. Governance in this era requires strict export controls and clear ethical policies on dual-use applications to ensure your innovations are not repurposed for harm.