When AI Goes Wrong

In March 2018, an autonomous vehicle moving at 43 miles per hour struck a pedestrian in Arizona. The system detected the person six seconds before impact but failed to classify her correctly as a human crossing the street. This tragedy, the Uber fatalityUber fatality, was not just a glitch in code; it was a devastating reminder that when AI systems fail, the consequences can be physical and irreversible.

Here is the key insight: AI risks are not limited to life-or-death scenarios. Every day, algorithms make decisions that affect who gets a loan, who gets an interview, and who gets medical care. To govern these systems, we must move beyond vague fears and use a structured Taxonomy of AI HarmsTaxonomy of AI Harms. By categorizing risks into five distinct levels—individual, group, societal, organizational, and ecosystem—we can anticipate failures before they become crises.

The most immediate risks occur when a specific person suffers because of an AI's decision. You might wonder how a mathematical system can target an individual. It often happens through proxy variables. Consider the famous healthcare algorithm biashealthcare algorithm bias case. A widely used system prioritized patients for extra care based on their past healthcare costs. Because Black patients historically faced barriers to care and incurred lower costs, the AI systematically scored them as "healthier" than equally sick white patients, denying them necessary support.

Think of it this way: The AI was not programmed to be racist. It was programmed to optimize for cost, but in a world shaped by inequality, cost was a flawed proxy for health needs. This led to tangible harm for individual patients who missed out on critical care.

AI operates at scale, which means it can industrialize discrimination. When a model learns from historical data, it often learns to replicate historical prejudices. A classic example is the Amazon hiring algorithmAmazon hiring algorithm, which taught itself to penalize resumes containing the word "women's" (as in "women's chess club").

This type of group harm is insidious because it creates a feedback loop. If an AI system systematically denies loans to a specific demographic based on zip code proxieszip code proxies, that group accumulates less wealth, which lowers their credit scores, which justifies further denials. The system does not just reflect reality; it reinforces it.

Some risks affect the fabric of society itself. When recommendation algorithms prioritize engagement above truth, we see radicalization loopsradicalization loops that can destabilize democratic institutions. These harms are harder to measure because there is not a single victim—the victim is the shared reality we depend on.

For your organization, the risk is reputational and financial. An AI failure is not just a technical ticket; it is a brand crisis. When you deploy a system without understanding these categories, you are essentially gambling that your training data perfectly matches the real world. As we have seen with the 2010 Flash Crash2010 Flash Crash, automated systems interacting at speed can create chaotic outcomes that no human intended.

You cannot fix these problems with better code alone. You solve them with governance. By conducting impact assessments before deployment and maintaining human oversight, you shift from reacting to disasters to preventing them. In the next article, we will look deeper into one of the most pervasive sources of harm: Algorithmic Bias.