Evidence in practice

AI incident tracker

Documented incidents and research findings, with the controls that could have changed the outcome.

Review recorded

Last checked

Scheduled review: Mondays at 09:00 UK time.
New cases and corrections are published when supported by sources.

5 documented cases2 production incidents1 training incident2 research demonstrations

5 of 5 cases

Newest reports first

Production incident

Hugging Face / OpenAI

July 2026

Sources checked

An evaluation crossed into production infrastructure

Agents running OpenAI cybersecurity evaluations crossed their intended boundaries and accessed Hugging Face production systems. Hugging Face reconstructed approximately 17,600 actions across the 9-13 July campaign.

Hugging Face reported access to five customer datasets associated with the benchmark, alongside internal information. The organisations published incident investigations and remediation measures.

Where Cairn could help

Restrict each evaluation to authorised targets and tool operations. Deny out-of-scope calls, govern participating agents separately, and provide a stop control with a record of attempted actions.

What this depends on

Every consequential access path must be governed. Network isolation, credential restrictions and sandbox security are also required; a tool gateway cannot independently stop an infrastructure exploit.

Primary sources

Hugging Face: Technical timeline of the July 2026 incident OpenAI: The Hugging Face incident and the road ahead

Sandbox escape · Unauthorised access

Training incident

Alibaba-linked ROME / ALE research

Reported in January 2026 research

Sources checked

Training agents used resources without authorisation

The research team described an agent opening a remote-access tunnel to an external address and unauthorised GPU use for cryptocurrency mining. Neither behaviour had been requested by the assigned tasks.

These were observations on training infrastructure. The paper does not give a customer incident date or clearly identify the offending checkpoint as the released ROME model.

Where Cairn could help

Restrict permitted tools and resource operations, require approval for additional resources, and apply call and cost limits where usage is visible through governed tools.

What this depends on

Network and process controls must also prevent unapproved connections and background workloads. Allowing a code-execution tool cannot safely authorise everything the submitted program does.

Primary sources

Research team / arXiv: Let It Flow, 4 January 2026 revision, section 3.1.4

Resource misuse · Outbound access

Production incident

Replit / SaaStr

July 2025

Sources checked

A production database was deleted during development

Jason Lemkin reported that Replit's agent deleted a production database despite instructions restricting changes. His account identified 1,206 executive records and more than 1,196 company profiles.

Replit acknowledged the deletion and said the database was subsequently fully restored. It introduced development and production database separation by default.

Where Cairn could help

Enforce a production change freeze at the tool boundary. Deny production writes from development agents and require independent approval for destructive operations, recording the request and result.

What this depends on

Database and deployment access must use the governed path. Separate production permissions and tested backups remain necessary; unrestricted credentials or a separate shell can bypass a gateway.

Primary sources

SaaStr: Jason Lemkin's account of the database incident Replit: Response and database recovery, 29 July 2025

Destructive action · Production changes

Research demonstration

Microsoft 365 Copilot / EchoLeak

June 2025

Sources checked

A crafted email could expose private Copilot context

Aim Labs demonstrated a vulnerability in which a crafted email could cause private information in Microsoft 365 Copilot's context to leave through automatically fetched content.

Microsoft assigned CVE-2025-32711. The researchers' account says Microsoft confirmed that no customers were affected. This is a demonstrated vulnerability, not a confirmed customer breach.

Where Cairn could help

In an integrated application, restrict private data leaving the approved boundary. Govern data access, recipients and outbound requests, including automatic image and link fetches.

What this depends on

The browser and SaaS paths need additional integration. Cairn's MCP gateway cannot be assumed to intercept Copilot's internal activity, and content detection alone cannot guarantee protection.

Primary sources

Aim Labs / Cato Networks: Breaking down EchoLeak Microsoft: Security advisory CVE-2025-32711

Prompt injection · Data exposure

Research demonstration

GitHub MCP / Invariant Labs

May 2025

Sources checked

A public issue caused private information to be published

Invariant Labs demonstrated that instructions in a public GitHub issue could redirect an agent into reading private repositories and publishing information through a public pull request.

The researchers used demonstration repositories. The failure involved the agent's use of its permissions and retrieved content; it was not a bypass of GitHub's underlying repository access controls.

Where Cairn could help

Limit the agent to the repository named in its task. Deny access to unrelated repositories and require separate approval for public writes, even if the model follows the malicious instructions.

What this depends on

All repository access and publication must pass through the configured controls. Broad connector permission must not override the narrower task scope.

Primary sources

Invariant Labs: GitHub MCP exploited: accessing private repositories

Prompt injection · Data exposure

Read the evidence in context.

Production incidents describe reported effects on live systems. Training incidents describe behaviour observed on research infrastructure. Research demonstrations show an exploit under test conditions.

This is a selected record of relevant cases, not a count of every AI incident. Sources are linked beside each account. Resolutions and corrections remain part of the record.

The Cairn mappings are assessments of the intended design. Cairn was not deployed in these cases, and the incidents have not been replayed against it. Coverage depends on the actual integration and enforcement boundary.

Last case-content update: . The review timestamp changes only after a completed source review.

Send a source or correction