AI Incident Roundup | September 2026

Each month, we review AI incidents and near misses we've learned about, some recent and some newly reported. We test each one against CyberCube's AI Event Framework to see what it means for (re)insurers.

AI-induced loss and harm incidents demonstrate that it is already impacting insurers beyond cyber. As the technology gets deployed more widely and the model capabilities advance, it will accelerate.

The insurance industry is grappling with this emerging risk through a broad range of activities, such as updating policy forms (either adding affirmative language or exclusions), rolling out AI-questionnaires for broking and underwriting, and creating roles for AI/emerging risk.

This is the first post in a monthly roundup of select AI incidents and near misses to assist the insurance industry to evaluate, price, and accept the risk. It will highlight events that have not been widely covered but offer insights and learnings. By continually highlighting niche examples, the goal is to:

1. Identify the organizational decisions that were critical to the incident’s outcome, whether that was harm-avoidance or harm-realization.
2. Interrogate CyberCube’s recently released AI-framework. By focusing on edge cases, we can stress test the framework and identify where it breaks down.
3. Use counter-factual to expand our imagination for AI as an emerging risk.

Incident #1: BNSF Railway’s AI Signaling System Dispatched Hazmat Train Towards Worker

Event Overview

On June 16 2026, a BNSF train carrying “hazardous materials” was directed towards maintenance workers by Movement Planner, an AI-powered dispatch system. A human dispatcher realized and corrected the mistake before any collision or harm. The American Train Dispatchers Association filed a formal complaint to the Federal Railroad Administration (FRA) shortly afterwards and another union, the Brotherhood of Railroad Signalman, sent a letter to the FRA in August condemning the automated system.

CyberCube Event Family

CyberCube’s AI Event Framework has six families to evaluate harm from AI. This near miss is an interesting case because it has traits from three event families.

• Event Family 2 is harm that results from wrong AI model output, but the decision is made by a human.
• Event Family 3 is harm by wrong model outputs, but the machine makes the decision without humans in-the-loop.
• Event Family 4 is AI output that results in physical or bodily harm.

What happened: the dispatch system made a bad decision, routed the train to a track with maintenance workers, and acted on that decision by doing the actual routing. So far, a classic example of Event Family 3. But then a human recognized the error and manually intervened to correct the mistake, averting harm. In this case, the human was more or less a safeguard on an already autonomous and authorized system. If the human worker was fed a recommendation by the system to route a train a certain way, and the human authorized this, then this would look more like Event Family 2. However, the tiebreaker ultimately goes to Event Family 3 because the decision was made by the machine.

Now consider two counterfactuals where the human does not correct the error.

In the first, the train is directed the wrong way, but workers and machines are cleared from the track, resulting in no physical damage. However, this potentially delays shipment, disrupting other train routes, and using excess fuel. This would result in financial loss to the train operator — but it would be an uninsured first party loss. Since there is no physical damage and the decision was still made by the machine, it remains in Event Family 3.

In the second counterfactual, the train collides with worker(s), causing bodily harm and potential property damage. The harm would be insured under common policies, but the event would be best placed in Event Family 4 due to the physical damage created by bad outputs.

The takeaway from the event in the context of the framework is that the framework is fluid, and differences in circumstance can change the insurable outcome. It matters both how the initial error was made (i.e. autonomy), and the results of the error (impact).

Insurance Implications: What Can Insurers Learn from This?

This near-miss example provides several lessons and actions for the insurance industry.

First, the minutiae of workflow design and controls matter when AI is involved. It is not clear how the human dispatcher recognized the error — whether this was a happy / lucky coincidence or part of the process — but small changes could have transformed an event that resulted in no losses to one with millions.

Second, AI can cause harm that is currently uninsured, which is underscored by the first counterfactual. As AI systems become more embedded, it will be critical for brokers to work with insureds to understand this uninsured risk. If and how the insurance industry decides to expand coverage for these first-party type losses remains an open question.

Third, AI’s impact will be far beyond the “traditional” technology lines of Cyber and Tech E&O. In a worst-case scenario, the train would have collided with the worker and maintenance equipment, physically harming the worker, damaging the equipment, and spilling hazardous material. If that had happened, various Railway Liabilities coverages, such as Bodily Injury, Property, and Hazardous Waste / Environmental Cleanup related coverage would have been triggered — a multi-million dollar claim from a computer error, without a cent being paid from Cyber or traditional technology insurance lines.

Incident #2: AI Customer Service Bot Made False and Illegal Claims

In May 2026, a German court held that a Medical Company is liable for its customer service chatbot making inaccurate, misleading, and illegal claims. Under German law, it is illegal to promote oneself as a “board-certified plastic and cosmetic surgeon” without the pre-requisite certification. The Defendants utilized an AI-powered chatbot to assist with basic customer service functions, such as scheduling appointments. When asked about the co-CEO’s certification, however, it claimed they did have the certification in question despite not having it; nowhere on their website did they make that claim.

A suit was filed against the Defendants under the unfair competition law, and the judge ruled in favor of the plaintiffs, issuing a cease-and-desist order. The judge ruled that the owners or users of the chatbots are responsible for erroneous outputs, including hallucinations.

CyberCube Event Family

Whereas the BNSF Railway case presented an edge case, this is straightforwardly a case of Event Family 3 — Automated AI Decision Failure. The AI made a bad conclusion (that the co-CEOs had the certification despite no evidence that they did) and acted on that bad conclusion, in the form of responses to customers.

What is particularly interesting about this example is intent. Most people would not consider the AI’s action a decision, nor would they be concerned about the AI-bot’s use-case of scheduling appointments. The potential downside seems low and the company probably did not consider themselves giving up control. However, in echoes of the Air Canada incident, by giving AI-powered agents access to customers, companies cede control and decision-making.

Insurance Implications: What Can Insurers Learn from This?

The clear takeaway is that allowing unsupervised AI agents access to customers can result in unforeseen harm and losses. This is not to say that companies should not do this — AI-customer service agents are clearly here to stay for good reasons — but that the risk needs to be known and monitored. That means identifying where AI has access to customers, identifying the controls to stop or mitigate harm before it grows, and becoming comfortable with the risk.

Key Takeaways from This Roundup

1. We are in the early days of tracking AI event’s impacts on insurance; there may not be clear-cut losses and costs.
2. These AI-induced events are often nuanced. The learnings of what have or could have gone wrong provide an understanding of where losses can come from and where preventative measures would (or would not) have had an impact.

Every incident in this roundup is assessed using CyberCube's AI Event Framework. To see how the framework works and how we apply it to AI catastrophe risk, read our report, Machine State of Mind: A Framework for Quantifying AI-Driven Insurance Risk.