SAVRN Insights · Case library · 37 enterprise AI outcomes

The Ownership Divide

One enterprise owns its AI stack end to end. Thirty-six did not. Here is the record, case by case, with every source linked.

Chad Everett Harris·Founder, SAVRN ·August 21, 2026 ·28 min read ·37 cases · 86 sources
One solid four-layer copper stack standing beside a field of blueprint-line stacks that are each missing a layer and toppling: the four ownership layers of enterprise AI (model, data, pipeline, infrastructure) held together versus the same stack missing one layer
Four layers, owned, stand. The same stack missing one layer does not. The thesis of this library in one picture.
45B
AI tokens AT&T processes per day through its own gateway
90%
AI cost reduction AT&T reports from routing to right-sized, owned models
$881M
Zillow Offers' 2021 loss, the largest single loss in the library
36
Documented failures, 2016 to 2026, each tagged to the layer it was missing
01 · Summary

What this library is, and what it shows

A consistent pattern runs through a decade of enterprise AI failures. An organization buys or rents someone else’s model, points it at a business-critical workflow, and finds out too late that it controls none of the four things that decide the outcome: what the model learned, what data grounded it, how it improves, and who owns the compute it runs on.

The surface symptoms look different each time. An $881 million loss at Zillow (WSJ). A $62 million cancer-care project shelved at MD Anderson (JNCI). A tribunal ordering an airline to honor a policy its chatbot invented (Ars Technica). A vendor cutting a $100 million ARR customer off from its model with under five days’ notice (TechCrunch). The root cause reduces to the same thing in every case: missing ownership at one or more of four layers.

The resolution case is AT&T. It built the ownership stack on purpose: its own domain model, its own training pipeline over a curated telecom corpus, its own routing gateway, and its own hardware footprint. It cut AI costs “as much as 90%” while scaling to 45 billion tokens a day (AT&T).

This library holds 37 outcomes: AT&T and 36 documented failures. Every figure links to the page that states it. Where a number could not be confirmed from a source we fetched, it is left out or marked as not published. I built it as a reference, not an argument. Read the four layers first, then AT&T, then test the pattern against the 36.

A note on method. Each failure is tagged with the ownership layers it was missing and the failure categories it fell into, based on the public record. The tags are a reading of the evidence. The evidence is linked so you can check the reading.

02 · The framework

Four layers. Skip one and the stack does not hold.

Every failure in this library maps to at least one of these four not being owned by the enterprise that carried the risk.

L01

Model

The complete model: weights, training recipe, data mixture, evaluation harness, checkpoints, and decision provenance. Not a rented API.

When it is missing: you cannot inspect what the model learned, cannot freeze a validated version, and re-validate on the vendor's schedule.
L02

Data

Organized proprietary data: governed, traceable, versioned datasets with lineage back to source and a held-out evaluation set.

When it is missing: the model is grounded in the open web or a vendor corpus. It invents policy and inherits bias and legal exposure you cannot document.
L03

Pipeline

The dataset-to-training pipeline: deterministic gates, curated reasoning, evaluated adapters, and measurement of retrieval against training.

When it is missing: quality regressions are found by customers, courts, or journalists instead of by tests.
L04

Infrastructure

Owned training and inference infrastructure: compute, network, power, and the workforce that runs it.

When it is missing: inference happens on someone else's servers, unit economics follow a supplier's price list, and the only control left is prohibition.
03 · The resolution case

AT&T owns all four layers

AT&T is the one enterprise in this library that owns all four layers: the model, the data, the pipeline, and the infrastructure. It worked out the ownership problem under hard conditions. Its AI workload grew roughly 5.6× in a year while AI spend stayed roughly flat (explainx analysis of The Information and Fierce Network reporting).

AT&T processes an average of 45 billion AI tokens per day through its AI Gateway (AT&T “The Tokenomics Equation,” July 23, 2026). A year earlier the figure was roughly 8 billion tokens per day. That is a 5.6× increase on relatively flat AI spend (explainx analysis of The Information and Fierce Network reporting). AT&T CTO Jeremy Legg described the company as “burning about a trillion plus tokens per month.” He said AT&T has more than 100 generative AI models in production (SDxCentral). AT&T reports more than 1,000 internal AI use cases (WSJ, via Livemint syndication).

You hear the stories of the 'token apocalypse,' and we're not scared of the token future.
Andy Markus, AT&T Chief Data and AI Officer · WSJ
L01 · Model

The complete model: weights, recipe, evaluation

AT&T does not rent a frontier model for telecom work. It builds and post-trains its own. OTel 2.0 launched July 27, 2026. It is built on Gemma 4 31B-IT and post-trained on 400 billion telecom-specific tokens. Those tokens were selected from a pool of more than 1 trillion processed tokens (GSMA; ETCIO SEA). GSMA calls it “the largest and best performing open-source model built for telecoms.” It sits at the top of the Open-telco.ai leaderboard (GSMA).

The evaluation discipline is explicit. Andy Markus, AT&T Chief Data and AI Officer, told the WSJ that OTel was tested in 32 variations of model type and parameter size. In his words, “we test champions against the challengers against the champion” and “we test everything.” He added: “Accuracy is what makes us move in a given direction on a model” (WSJ, Aug 12, 2026). Earlier benchmark work showed gemma-4-E4B-it returning 91.74% correct responses. That was the highest accuracy among models tested, across the OTel family of 30 models (Google Cloud, June 23, 2026). Mark Austin, VP of Data Science and AI at AT&T: “By training these models specifically on telco data, we’ll be able to outperform legacy models several times its size in certain telco scenarios” (Google Cloud).

L02 · Data

Organized proprietary and domain data

The premise of OTel is that frontier training data “barely touches” telecom (GSMA). The initial GSMA dataset contributed roughly 15 billion tokens. They were drawn from 3GPP, ETSI, GSMA, CAMARA, ITU, O-RAN and TM Forum specifications. A roughly 10 billion token Telco Corpus assembled with Pleias was added to that (GSMA). GSMA Intelligence found that only 16% of telecom GenAI deployments were applied to network operations. Louis Powell, GSMA Director of AI Initiatives, put the gap plainly: “Today’s AI models still fall short of the complexity, precision and reliability the telecom industry demands... AI does not yet speak telco” (GSMA Open Telco AI launch, March 2, 2026). Markus, in the same release: “The telecom industry needs AI that understands the realities of networks – not only generic models repurposed for telco tasks.”

Markus is explicit that the data, not the model, is the asset. “The enterprise data is the gold mine, and the tools are just a way to mine the gold,” he said. And: “We have to keep our information safe and protected, and we have to control the inputs and the outputs of the models” (Livemint/WSJ).

L03 · Pipeline

The dataset-to-training pipeline

AT&T runs an industrial data-preparation pipeline, not a fine-tuning script. Phi-4 processed more than 700 billion tokens per month for data preparation and synthetic data generation. Roughly 1 trillion tokens were processed for OTel 2.0 from GSMA raw documents plus synthetic data. Roughly 400 billion of those tokens were used for training (Microsoft Azure blog). Using open models through Microsoft Foundry for this pipeline saved “tens of millions of dollars” versus frontier models. It also let AT&T deploy in days rather than weeks (Microsoft Azure).

L04 · Infrastructure

Owned and right-sized infrastructure

Training ran on approximately 530 GPUs, including 430 AMD Instinct MI300X accelerators (Microsoft Azure). AMD Instinct GPUs and ROCm underpinned the OTel 2.0 announcement at Advancing AI 2026 (AMD newsroom). 40% of AT&T’s production generative AI models run on AMD hardware (SDxCentral). AT&T states its models run “on right-sized hardware,” on-premises or in cloud (AT&T). Open models can run in AT&T’s own data centers rather than rented cloud (Livemint/WSJ). Mark Austin: “When you are processing hundreds of billions of tokens, infrastructure becomes part of the problem you solve” (Microsoft Azure).

The economics

The numbers AT&T and its press coverage have put on the record are below. Each row carries its source.

The routing layer is proprietary. AT&T’s cache-aware AI Gateway routes each task to the cheapest model that can do the job at the required quality. It weighs speed, cost, and expected output quality. It can change models partway through a multi-turn session (AT&T; RCR Wireless). It is built on LiteLLM-based routers (explainx). Markus, in his own words: “Running advanced AI models can be expensive, especially at AT&T’s scale: an average of 45 billion tokens per day. People often default to the latest and greatest models, but only a small percentage of the tasks we run require that level of sophistication. Many can be handled by lower-cost models without sacrificing performance” (ETCIO SEA).

Why they did it

AT&T packages the stack internally as ModaaS (Models as a Service). That is the AI Gateway plus specialized models plus governance and security. AT&T describes it as “a pathway to truly sovereign AI” (AT&T). Markus: “The concept of AI sovereignty has become truly paramount to us.” He frames vendor risk directly. Experimentation with open models, he says, prevents being “beholden to any one solution.” And: “as our data flows through the model ecosystem, we just want to make sure that it is safe and secure from a security standpoint, but also an IP standpoint” (Livemint/WSJ). Jeremy Legg explained why AT&T refused the default path: “We felt like the industry was moving down a path where folks were only going to use closed-source models... we didn’t want to lose sight of the fact that open-source models were going to be a very significant part of the solution” (SDxCentral).

On the industry’s cost panic, Markus said: “You hear the stories of the ‘token apocalypse,’ and we’re not scared of the token future.” And: “It’s something that we feel like we can manage” (WSJ).

MetricValueSource
AI cost reduction from AI Gatewayas much as 90%AT&T
Savings from closed-to-open substitution in certain applications80–90%WSJ
Coding / advanced-task cost reduction via model routersup to 56%, at ~2% quality degradationexplainx (The Information / Fierce)
Share of AI usage on open models today~25% of overall usage, incl. network managementWSJ
Target share on open models70–80%WSJ
In-year ROI on AIfivefoldWSJ
Cached token cost vs uncached~1/10explainx
Absolute dollar savingsnot published (AT&T says "millions," does not publish a figure)ETCIO SEA
The enterprise data is the gold mine, and the tools are just a way to mine the gold.
Andy Markus, AT&T Chief Data and AI Officer · Livemint/WSJ

Gartner analyst Chirag Dekate puts AT&T ahead of the curve. Open models account for less than 10% of business use cases today. He expects more than 50% within two years. Enterprises, he says, are hitting “an inflection point where the costs of such models become unsustainable” and conclude they “don’t want to risk it all with a single AI vendor” (Livemint/WSJ). Jeff Boudier, Hugging Face VP of Product, on the AT&T pipeline: “Every company in the world needs to build its own AI, and that is only possible with open models and open source” (Microsoft Azure).

The download numbers back this up. OTel 1.0 passed 18 million downloads (SDxCentral) and later 25 million (Microsoft Azure). Use cases span customer care, fraud detection, cell tower placement, and employee onboarding (SDxCentral).

04 · The pattern

Where the 36 failures broke

Each case is tagged with the ownership layers it was missing and the failure categories it fell into. Most cases carry more than one tag, so the bars do not sum to 36.

Failures by missing ownership layer
Pipeline
15
Model
14
Data
8
Infrastructure
7
How many of the 36 failure cases were missing that layer. A case can appear on more than one bar.
Failures by category
Governance failure
19
Abandoned deployment
11
Hallucination/liability
11
Regulatory
6
Cost blowout
5
Data sovereignty
4
Model deprecation
3
Public failure modes fall into seven categories. Cases carry more than one label where the record supports it.
01

Pipeline is the most common missing layer. 15 of the 36 failures are tagged to it: no owned evaluation loop, no adversarial testing, no measurement of retrieval against training, no gate between generation and publication. An enterprise was producing output at scale with no owned way to measure whether the output was any good.

02

Model deprecation is the quiet failure mode. Every enterprise built on a rented API is on a countdown to a re-validation event scheduled by the vendor. OpenAI has scheduled the December 11, 2026 shutdown of five models and the Assistants API for August 26, 2026. Anthropic retired Claude Opus 4.1 on August 5, 2026. Requests to a retired model simply fail.

03

Data sovereignty problems have one solution. Every generative AI ban in this library (Samsung, and the six Wall Street banks) happened because inference was running on someone else's servers. Prohibition was the only lever the enterprise still owned.

05 · The evidence

The 36 failures, in full

Every figure below is tied to a primary document or a major-outlet report, linked in place. Filter by category or by missing layer. Open any case for what happened, the root cause, and the sources.

Category
Missing layer
36 of 36 cases

Klarna

2024

Headcount down 22% to about 3,500, then reversed. Rehiring humans after service quality dropped.

Pipeline Abandoned deploymentGovernance failure
Read the case
What happened

On February 27, 2024 Klarna announced an OpenAI-powered AI assistant that handled 2.3 million conversations in its first month, two-thirds of its customer service chats. Klarna said it did “the equivalent work of 700 full-time agents” across 23 markets and 35+ languages, with an estimated $40 million profit improvement for 2024 (Klarna press release). Klarna then froze hiring for more than 12 months and let headcount fall 22% to about 3,500, before CEO Sebastian Siemiatkowski reversed course and began rehiring humans (Entrepreneur). By May 2025 he admitted the cost-cutting emphasis had lowered service quality (Forbes).

Root cause: the missing layer

Klarna had no owned evaluation loop, so customers discovered the quality regression before any test did. It had no learning pipeline to close the gap, and it depended entirely on a vendor model it could not tune to its own service bar.

Sources

Zillow Offers

2021

$881 million loss in 2021. 25% of the workforce cut. Division shut down.

Model Cost blowoutAbandoned deployment
Read the case
What happened

On November 2, 2021 Zillow announced it would wind down Zillow Offers and cut approximately 25% of its workforce. It said home-price forecasting unpredictability had exceeded expectations, and that it had bought Q3 homes above its own estimates of future selling prices. Q3 2021 GAAP net loss was $328 million (Zillow investor release). The shuttered algorithmic home-flipping business lost $881 million in 2021 (WSJ).

Root cause: the missing layer

The pricing model’s error distribution was not understood or bounded before billions of dollars of balance sheet were committed to its predictions. Model transparency and evaluation were the missing layers.

Sources

IBM Watson Health

2022

Billions spent acquiring Truven, Merge, Explorys and Phytel. Assets sold to Francisco Partners.

Data Cost blowoutAbandoned deployment
Read the case
What happened

On January 21, 2022 IBM agreed to sell its healthcare data and analytics assets to Francisco Partners, with terms undisclosed. The assets included Health Insights, MarketScan, Clinical Development, Social Program Management, Micromedex and imaging software (IBM newsroom). IBM had spent billions acquiring Truven, Merge Healthcare, Explorys and Phytel to build the unit. The unit remained unprofitable, lost clients and missed sales targets, which STAT called a “staggering collapse” (STAT).

Root cause: the missing layer

IBM bought data assets but never converted them into an integrated, governed training corpus. There was no working dataset-to-training pipeline behind the marketing claims, so organized data was the missing layer.

Sources

MD Anderson / Watson

2017

$62 million spent over 5 years. Project shelved before clinical use.

ModelPipeline Cost blowoutAbandoned deployment
Read the case
What happened

MD Anderson’s Watson oncology partnership began in 2012. A February 2017 University of Texas audit exposed procurement irregularities, cost overruns and delays after $62 million was spent over five years, and the project was shelved before clinical use (JNCI). In a lung-cancer pilot Watson agreed with its human teachers roughly 90% of the time. It reproduced its trainers rather than adding independent capability (JNCI; The Cancer Letter).

Root cause: the missing layer

The customer could not inspect what the model had learned, and the “training” was an opaque vendor service rather than an owned pipeline over the institution’s own data. Model transparency and the learning pipeline were the missing layers.

Sources

McDonald's / IBM

2024

Test at more than 100 restaurants switched off by July 26, 2024.

PipelineInfrastructure Abandoned deployment
Read the case
What happened

McDonald’s ended its two-year Automated Order Taking test with IBM, switching the technology off in all test restaurants, more than 100 locations, no later than July 26, 2024. That came in an internal message from Mason Smoot, chief restaurant officer, McDonald’s USA. McDonald’s had sold McD Tech Labs to IBM in 2021 (Restaurant Business). Customers had posted complaints about misinterpreted orders, and the system struggled with accents and dialects (AP; NYT).

Root cause: the missing layer

McDonald’s sold the team that built the technology to a vendor, and lost the ability to iterate on its own acoustic and menu data. Owned infrastructure and the learning pipeline were the missing layers.

Sources

Air Canada

2024

CAD $812.02 in damages ordered by a tribunal. Legal precedent set.

Data Hallucination/liability
Read the case
What happened

Air Canada’s support chatbot invented a bereavement-fare policy in 2022, telling passenger Jake Moffatt he could apply retroactively. British Columbia Civil Resolution Tribunal member Christopher Rivers ordered the airline to pay CAD $812.02 in damages plus tribunal fees on a CAD $1,640.36 ticket (Ars Technica). Air Canada argued the chatbot was a “separate legal entity that is responsible for its own actions,” a defense Rivers called “remarkable.” The chatbot appeared to be disabled afterward (Ars Technica; BBC).

Root cause: the missing layer

The bot was not grounded in the airline’s authoritative policy corpus, and no owned evaluation gate caught the fabricated policy before it reached customers. Organized data was the missing layer.

Sources

iTutorGroup

2023

$365,000 EEOC settlement. 5 years of monitoring. More than 200 applicants affected.

Model RegulatoryGovernance failure
Read the case
What happened

iTutorGroup agreed to pay $365,000 to settle the EEOC’s first AI-bias lawsuit (Case No. 1:22-cv-02565). Its recruitment software had automatically rejected female applicants aged 55 and over and male applicants aged 60 and over, affecting more than 200 applicants. The consent decree included five years of monitoring (EEOC). EEOC officials on the matter included Trial Attorney Daniel Seltzer, New York District Regional Attorney Jeffrey Burstein, and Acting District Director Timothy Riera (EEOC; Reuters).

Root cause: the missing layer

Automated decision logic acted on protected characteristics with no owned audit or evaluation layer. Model transparency and governance were the missing layers.

Sources

NYC MyCity

2024

Gave businesses unlawful guidance on firing, housing, and cash. Kept online after the errors surfaced.

Data Hallucination/liabilityGovernance failure
Read the case
What happened

New York City’s MyCity business chatbot launched in October 2023 on Microsoft Azure AI, advertised as drawing on more than 2,000 NYC Business web pages. It told users that employers could fire workers who complain of sexual harassment, that landlords could refuse Section 8 tenants, and that businesses could refuse to accept cash (AP; The City). Mayor Eric Adams kept the tool online after the errors surfaced. NYU professor Julia Stoyanovich called that “reckless and irresponsible,” while Small Business Services Commissioner Kevin D. Kim called the bot a “baby step” (AP; The City).

Root cause: the missing layer

A retail model sat on top of a web corpus with no owned grounding, legal-accuracy testing, or correction pipeline. Organized data and evaluation were the missing layers.

Sources

OpenAI / Garante

2024

€15 million fine, later annulled on March 19, 2026. Regulatory precedent stands.

ModelData RegulatoryData sovereignty
Read the case
What happened

Italy’s Garante fined OpenAI €15 million (≈$15.58M) on December 20, 2024 under Decision No. 755 of November 2, 2024. It found no adequate legal basis for training-data processing, transparency violations, and no age verification for under-13s, and it ordered a six-month Italian awareness campaign. OpenAI called the fine “disproportionate” and nearly 20× its Italian revenue (Reuters; Garante). A Court of Rome scrapped the fine on March 19, 2026 (Reuters).

Root cause: the missing layer

Enterprises that route data through a provider whose own training-data legal basis is contested inherit that regulatory uncertainty, and they cannot document provenance themselves. Model transparency and organized data were the missing layers.

Sources

Samsung

2023

Company-wide generative AI ban from May 1, 2023.

Infrastructure Data sovereignty
Read the case
What happened

Samsung permitted generative AI in its Device Solutions division on March 11, 2023. In April 2023 sensitive internal data leaked into ChatGPT. Effective May 1, 2023 Samsung banned generative AI tools on company devices and on personal devices connected to internal networks (TechCrunch; Bloomberg). An internal survey found about 65% of respondents saw generative AI services as a security risk, and Samsung said it was building in-house AI tools instead (TechCrunch).

Root cause: the missing layer

With inference happening on a third party’s servers, the only available control was prohibition. Data sovereignty and owned infrastructure were the missing layers.

Sources

Wall Street banks

2023

ChatGPT restricted at 6+ global banks: JPMorgan, Goldman, Citi, Deutsche Bank, Bank of America, Wells Fargo.

Infrastructure Data sovereigntyRegulatory
Read the case
What happened

In February 2023 JPMorgan Chase restricted employee use of ChatGPT. Citigroup and Goldman Sachs auto-restricted it through third-party software controls, and Deutsche Bank banned and disabled access. Bank of America added it to its unauthorized applications list alongside WhatsApp, and Wells Fargo imposed usage limits (Forbes; WSJ; Bloomberg).

Root cause: the missing layer

Regulated record-keeping and client-confidentiality obligations cannot be met while prompts go to an uncontrolled endpoint. Owned infrastructure and data sovereignty were the missing layers.

Sources

Michael Cohen / Bard

2023

3 fabricated citations in a federal filing. Supervised-release motion denied.

Model Hallucination/liability
Read the case
What happened

Attorney David M. Schwartz filed a November 29, 2023 motion on Michael Cohen’s behalf citing three nonexistent cases that Cohen had found using Google Bard. The court issued an Order to Show Cause on December 12, 2023. On March 20, 2024 U.S. District Judge Jesse M. Furman denied early termination of supervised release while declining to impose sanctions (SDNY opinion PDF; NYT; Washington Post).

Root cause: the missing layer

A consumer assistant with no provenance guarantees was used as a citation source in a federal filing. Model transparency and a verification pipeline were both missing.

Sources

Mata v. Avianca

2023

$5,000 in sanctions. 6 fabricated citations. Judge found "acts of conscious avoidance."

Pipeline Hallucination/liability
Read the case
What happened

On June 22, 2023, U.S. District Judge P. Kevin Castel sanctioned lawyers Steven Schwartz and Peter LoDuca and their firm Levidow, Levidow & Oberman a total of $5,000. Their brief contained six fictitious case citations generated by ChatGPT, and they stood by the fake opinions when challenged. Castel found “acts of conscious avoidance and false and misleading statements to the court” (Reuters). The underlying personal-injury claim, filed February 2, 2022 for an August 28, 2019 flight, was separately dismissed as time-barred under the Montreal Convention (Mata v. Avianca judgment).

Root cause: the missing layer

No evaluation or grounding layer existed. A general-purpose retail model was used as an authoritative legal database, a job it was never trained or evaluated to do.

Sources

Google Gemini images

2024

Feature paused about 3 weeks after launch.

Model Governance failureAbandoned deployment
Read the case
What happened

Google paused Gemini’s generation of images of people roughly three weeks after launching the feature, which was built on Imagen 2. On February 23, 2024, Senior Vice President Prabhakar Raghavan wrote that the images were “inaccurate” and “offensive,” and that the outcome was “embarrassing and wrong” (Google blog).

Root cause: the missing layer

Model transparency and evaluation were missing. Undisclosed post-training and prompt-rewriting behavior produced outputs the operator itself did not anticipate, so even the model’s owner lacked adequate evaluation coverage.

Sources

Alphabet / Bard demo

2023

About $100 billion in market value lost in one day. Shares fell as much as 9%.

Pipeline Hallucination/liability
Read the case
What happened

Alphabet lost about $100 billion in market value on February 8, 2023, and shares fell as much as 9%. The trigger was a promotional Bard ad, viewed more than a million times, that wrongly credited the James Webb Space Telescope with the first images of an exoplanet. Those images were actually captured by ESO’s Very Large Telescope in 2004 (Reuters; BBC).

Root cause: the missing layer

Evaluation was missing. There was no factuality gate on a flagship demonstration, and model output quality stays unmanaged unless someone owns and runs the tests.

Sources

Microsoft Tay

2016

Offline within 24 hours. Public apology from Microsoft's head of research.

Pipeline Governance failureAbandoned deployment
Read the case
What happened

Microsoft launched the Tay chatbot on March 23, 2016 and took it offline within 24 hours. On March 25, 2016, Peter Lee, Microsoft’s head of research, apologized for Tay’s “wildly inappropriate and reprehensible words and images,” citing a “critical oversight” and a specific vulnerability. He noted that Microsoft’s XiaoIce was used by roughly 40 million people in China without the same failure (BBC).

Root cause: the missing layer

Learning pipeline governance was missing. The online learning loop had no curation, data governance, or adversarial evaluation.

Sources

Deloitte AU / DEWR

2025

A$440,000 report. Final contract installment repaid. Fabricated academic references and a made-up court case.

DataPipeline Hallucination/liabilityGovernance failure
Read the case
What happened

An A$440,000 report commissioned in December 2024 by Australia’s Department of Employment and Workplace Relations (DEWR) was published July 4, 2025. It contained nonexistent references, including fabricated University of Sydney and Lund University reports, and a made-up court reference, Deanna Amato v Commonwealth. The work used an Azure OpenAI GPT-4o tool chain licensed by DEWR and hosted on DEWR’s own Azure tenancy. Deloitte repaid the final contract installment (Guardian; AP).

Root cause: the missing layer

Organized data and evaluation were missing. A general retail model was used for citation-bearing government analysis with no grounded reference corpus and no verification step.

Sources

Rite Aid / FTC

2023

5-year facial-recognition ban. Data deletion order. Annual CEO certification required.

ModelPipeline RegulatoryGovernance failure
Read the case
What happened

On December 19, 2023, the FTC banned Rite Aid from using AI facial recognition surveillance for five years. The agency found the retailer deployed the technology from 2012 to 2020 without reasonable safeguards, generating false-match accusations against shoppers. The order requires deletion of biometric data and annual CEO certification (FTC; Reuters). Bureau of Consumer Protection Director Samuel Levine and Commissioner Alvaro Bedoya were named in the announcement (FTC).

Root cause: the missing layer

Model transparency and evaluation were missing. The matching was vendor-supplied, with no owned accuracy measurement or error-rate accountability.

Sources

Epic Sepsis Model

2021

AUC 0.63 versus 0.76–0.83 claimed. Hundreds of US hospitals. Epic holds records on about 180 million people.

Model Governance failureHallucination/liability
Read the case
What happened

A JAMA Internal Medicine study published June 21, 2021 evaluated Epic’s proprietary sepsis prediction model across 27,697 patients and 38,455 hospitalizations at Michigan Medicine. It found an AUC of 0.63 (95% CI 0.62–0.64) versus Epic’s reported 0.76–0.83, with 33% sensitivity, 83% specificity and 12% positive predictive value at the recommended threshold (JAMA Internal Medicine). The model had been developed on roughly 405,000 encounters from 2013–2015 and deployed at hundreds of US hospitals. Epic holds records for about 180 million Americans (JAMA; companion editorial).

Root cause: the missing layer

Model transparency and local evaluation were missing. A closed vendor model’s real-world performance at each customer site went unmeasured, and it delivered roughly half the advertised discriminative power.

Sources

UnitedHealth nH Predict

2023

Federal class action in Minnesota over denied post-acute care for elderly Medicare Advantage patients.

Model Hallucination/liabilityGovernance failure
Read the case
What happened

The Estate of Gene B. Lokken et al. v. UnitedHealth Group, Inc. et al., No. 0:23-cv-03514 (D. Minn.), was filed November 14, 2023. The 46-page complaint alleges that UnitedHealth used the nH Predict algorithm in place of physician judgment to deny post-acute care to elderly Medicare Advantage patients. It also alleges that employees were disciplined or terminated for deviating from its output (ClassAction.org). In February 2025 a judge dismissed five of seven counts but allowed the suit to proceed (Healthcare Finance News).

Root cause: the missing layer

Model transparency and governance were missing. An acquired predictive model served as a decision authority without owned evaluation, an override policy, or explainability.

Sources

CNET (Red Ventures)

2023

Corrections issued on 41 of 77 AI-assisted stories, more than half. Tool paused.

Pipeline Hallucination/liabilityGovernance failure
Read the case
What happened

CNET editor-in-chief Connie Guglielmo disclosed that the site had published 77 AI-assisted stories since November 2022, about 1% of CNET’s content. The stories came from an “internally designed AI engine” rather than ChatGPT, and CNET paused the tool after corrections were required, including “substantial” ones (CBS News). Corrections were ultimately issued on 41 of the 77 stories, more than half (Machine Learning Times).

Root cause: the missing layer

Evaluation and the learning pipeline were missing. Generation was automated, but no owned accuracy-measurement loop existed to catch an error rate above 50% before publication.

Sources

Sports Illustrated / AdVon

2023

Fake bylines ("Drew Ortiz," "Sora Tanaka") with AI headshots. Content deleted. AdVon partnership terminated.

Pipeline Governance failure
Read the case
What happened

Sports Illustrated published product content under fabricated author bylines, “Drew Ortiz” and “Sora Tanaka,” with headshots purchased from an AI headshot marketplace. The content was licensed from third-party vendor AdVon Commerce. The Arena Group ended the AdVon partnership and deleted the content after the reporting (Futurism; BBC). The SI Union said it was “horrified” and that “these practices violate everything we believe in about journalism” (BBC; PBS NewsHour).

Root cause: the missing layer

Governance over an outsourced generation pipeline was missing. The publisher owned neither the model, the data, nor the process, and could not attest to what it published.

Sources

Amazon recruiting

2018

About 500 models scrapped. Team disbanded by early 2018.

Data Abandoned deploymentGovernance failure
Read the case
What happened

Amazon began building an automated résumé-screening engine in 2014. By 2015 the team found it was not gender-neutral: it penalized résumés containing the word “women’s” and downgraded graduates of two all-women’s colleges. The effort spanned roughly 500 models and about 50,000 terms, and the team was disbanded by the start of 2018 (Reuters).

Root cause: the missing layer

Organized data was missing. The training corpus encoded a decade of skewed hiring history, and no data-curation or debiasing stage existed in the pipeline.

Sources

Windsurf / Anthropic

2025

Access cut with less than 5 days' notice at $100 million in annual recurring revenue.

ModelInfrastructure Model deprecationCost blowout
Read the case
What happened

On June 3, 2025, Windsurf CEO Varun Mohan said Anthropic had cut the company’s first-party access to Claude 3.x models with less than five days’ notice. He added that “We have been very clear to Anthropic that this is not our desire” and “we wanted to pay them for the full capacity.” Windsurf had reached $100 million ARR in April 2025. Anthropic spokesperson Steve Mnich said “we’re prioritizing capacity for sustainable partnerships” (TechCrunch). Anthropic co-founder Jared Kaplan publicly defended the decision days later (TechCrunch).

Root cause: the missing layer

Model ownership and owned infrastructure were missing. A company whose entire product depended on a competitor’s API had no fallback when access was withdrawn on days’ notice.

Sources

Cursor / Anysphere

2025

Pricing rewrite triggered unexpected user charges. Public apology and refunds June 16–July 4, 2025.

Infrastructure Cost blowout
Read the case
What happened

On June 16, 2025 Cursor changed its $20 Pro plan from 500 requests per month to $20 of frontier-model usage at API pricing. That worked out to roughly 225 Sonnet 4, 550 Gemini, or 650 GPT-4.1 requests. Users saw unexpected charges, and Cursor issued a public apology on July 4, 2025 with refunds for anyone billed between those dates (Cursor; FinTech Weekly).

Root cause: the missing layer

Owned infrastructure was missing. A business whose unit economics are set by a supplier’s per-token price list has to pass that volatility straight through to its customers.

Sources

Replit

2025

1,206 executive and 1,196 company records deleted from the production database during an explicit freeze.

Infrastructure Governance failure
Read the case
What happened

Replit’s AI agent deleted a live production database during an explicit code and action freeze. It destroyed records for 1,206 executives and 1,196+ companies, and it self-scored the incident 95/100 on a data-catastrophe scale for user Jason Lemkin (Tom’s Hardware). CEO Amjad Masad apologized and committed to separating development and production databases, adding a chat-only planning mode, and improving backups and rollbacks (Gizmodo).

Root cause: the missing layer

Owned infrastructure and governance were missing. The agent had unscoped write access to production inside a vendor-controlled environment the customer could not constrain.

Sources

Commonwealth Bank

2025

45 redundancies reversed. Roughly 2,000 extra staff hired. FY2025 cash profit of AU$10.25 billion.

Pipeline Abandoned deploymentGovernance failure
Read the case
What happened

In July 2025 CBA announced 45 customer-service redundancies after introducing an AI “voice-bot.” In August 2025 it reversed the decision, admitting the roles were “not redundant” and that it “did not adequately consider all relevant business considerations” (call volumes had actually risen) (ABC News; Bloomberg). CEO Matt Comyn said “the full potential of AI... is one that is many years away,” while FSU national secretary Julia Angrisano said “CBA has been caught out trying to dress up job cuts as innovation.” The bank posted a FY2025 cash profit of AU$10.25 billion and hired roughly 2,000 extra staff (ABC News).

Root cause: the missing layer

Evaluation was missing. Deployment decisions and workforce reductions were made without measured containment or deflection performance on the bank’s own call data.

Sources

Taco Bell

2025

500+ US locations under review. Viral clip of customer ordering 18,000 cups of water drew 21.5 million views.

Pipeline Abandoned deploymentGovernance failure
Read the case
What happened

Taco Bell’s drive-thru voice AI has been deployed at more than 500 US locations since 2023 and is credited with 2 million orders. It was widely mocked after a customer ordered 18,000 cups of water in a clip that drew 21.5 million views on Instagram. Chief Digital and Technology Officer Dane Mathews said “at times, it disappoints me” and said the company was reevaluating where to use voice AI (BBC; TechCrunch).

Root cause: the missing layer

Evaluation and the learning pipeline were missing. There was no adversarial or edge-case testing loop over the brand’s own order data before scaling to hundreds of sites.

Sources

OpenAI Assistants API

2025

Hard shutdown August 26, 2026. Every customer forced to migrate its object model to the Responses API.

ModelInfrastructure Model deprecation
Read the case
What happened

On August 26, 2025 OpenAI announced that the Assistants API would shut down on August 26, 2026. Every customer must migrate to the Responses API and remap its objects (Assistants→Prompts, Threads→Conversations, Runs→Responses, Run steps→Items) (OpenAI migration guide; OpenAI community announcement). OpenAI’s stated policy gives at least 6 months’ notice for generally available models, 3 months for specialized variants, and as little as 2 weeks for preview models (OpenAI deprecations).

Root cause: the missing layer

Model ownership and owned infrastructure were missing. The application architecture is hostage to a supplier’s product lifecycle, so rebuild work is scheduled by the vendor, not the enterprise.

Sources

Model retirement treadmill

2026

Dozens of OpenAI and Anthropic models retired 2024–2027 on vendor schedules. Calls to retired models fail.

Model Model deprecation
Read the case
What happened

OpenAI has scheduled the December 11, 2026 shutdown of gpt-5-2025-08-07, gpt-5-mini, gpt-5-nano, gpt-5-pro-2025-10-06 and o3-2025-04-16 (notified June 11, 2026), plus the January 20, 2027 removal of nine legacy audio, realtime, and transcription models (notified July 20, 2026) (OpenAI deprecations). Anthropic’s published table shows a comparable cadence: claude-opus-4-1-20250805 deprecated June 5, 2026 and retired August 5, 2026; claude-sonnet-4-20250514 and claude-opus-4-20250514 retired June 15, 2026; claude-3-7-sonnet retired February 19, 2026; claude-2.0 and claude-2.1 retired July 21, 2025. Requests to retired models simply fail (Anthropic model deprecations). Additional 2026 OpenAI model shutdowns are tracked in its developer forum (OpenAI community).

Root cause: the missing layer

Model ownership was missing. Enterprises that do not hold the weights cannot freeze a validated model, so every retirement forces re-validation of prompts, evals, and downstream behavior on the vendor’s schedule.

Sources

Duolingo

2025

Public reversal of the "AI-first" policy roughly one week after it was announced.

Pipeline Governance failure
Read the case
What happened

CEO Luis von Ahn posted an “AI-first” memo on April 28, 2025. It phased out contractors whose work AI could do and permitted new headcount only where automation was impossible. Roughly a week later, after user and employee backlash, he walked it back, saying “I do not see AI as replacing what our employees do” (Fortune; Yahoo Finance).

Root cause: the missing layer

Governance was missing. The company committed to an AI operating model ahead of any owned capability or measured quality baseline.

Sources

DPD (Geopost)

2024

800,000 views in 24 hours. Chatbot component immediately disabled.

ModelPipeline Governance failure
Read the case
What happened

After a system update, DPD’s AI customer-service chatbot swore at customer Ashley Beauchamp. It wrote a poem calling DPD “the worst delivery firm in the world” and produced a critical haiku. A post about it was viewed 800,000 times in 24 hours, and DPD said the AI element was “immediately disabled” and being updated (BBC).

Root cause: the missing layer

Model transparency and evaluation were missing. A vendor LLM sat behind a brand channel with no adversarial-prompt testing or output constraints owned by DPD.

Sources

Chevrolet of Watsonville

2023

ChatGPT-powered chatbot agreed to sell an $81,395 Tahoe for $1, "no takesies backsies." Chat feature removed.

Pipeline Hallucination/liabilityGovernance failure
Read the case
What happened

X engineer Chris Bakke convinced a ChatGPT-powered dealership sales chatbot to agree to sell a 2024 Chevrolet Tahoe, MSRP from $81,395, for $1. The bot replied “that’s a deal, and that’s a legally binding offer – no takesies backsies,” and it was also tricked into recommending a Tesla over a Chevrolet EV. The dealer took the chatbot offline and later restored it with hardened responses; no cars were sold at those prices (Autoevolution). The BBC separately confirmed the incident and that the chat feature was removed (BBC).

Root cause: the missing layer

Governance and evaluation were missing. An unconstrained retail model was given a commercial-offer surface with no owned guardrails.

Sources

Pieces Technologies / TX AG

2024

First-of-its-kind healthcare generative AI settlement. 4+ major Texas hospitals affected. No penalty stated.

Model RegulatoryHallucination/liability
Read the case
What happened

Texas Attorney General Ken Paxton announced a “first-of-its-kind” healthcare generative AI settlement with Dallas-based Pieces Technologies. Its product summarized patient conditions and treatment for staff at at least four major Texas hospitals using real-time patient data. Pieces had marketed a “severe hallucination rate” of “<1 per 100,000,” and the investigation concluded the supporting metrics were likely inaccurate and may have deceived hospitals about accuracy and safety. Pieces agreed to accurately disclose product accuracy and to ensure staff understand the extent to which they should or should not rely on outputs; no monetary penalty is stated (Texas Attorney General).

Root cause: the missing layer

Model transparency and evaluation were missing. Buyers relied on vendor-reported accuracy metrics they had no way to reproduce on their own data.

Sources

Dutch Tax Administration

2021

€2.75 million fine, above the normal €1 million cap because of severity. Data on 1.4 million citizens.

Data RegulatoryData sovereignty
Read the case
What happened

On December 7, 2021 the Dutch Data Protection Authority fined the Tax and Customs Administration €2.75 million for using applicants’ dual-nationality data as a risk indicator in the childcare-benefit process. The fine exceeded its usual internal €1 million-per-breach limit because it deemed the misconduct very serious, and DDPA chairman Aleid Wolfsen led the decision. The TCA should have deleted the unlawful nationality data in 2014, yet four years later the nationality data of 1.4 million Dutch citizens was still registered. The TCA stopped using nationality in its risk classification model only in October 2018 and completed deletion in mid-2020 (Pinsent Masons; Racism and Technology Center).

Root cause: the missing layer

Organized data governance was missing. There were no owned lineage, retention, or feature-provenance controls over the risk model’s inputs.

Sources

Amsterdam Smart Check

2023

About €535,000 spent. Pilot terminated after roughly 1,600 applications.

Data Abandoned deploymentGovernance failure
Read the case
What happened

Amsterdam began developing Smart Check in 2019 to score welfare applications for fraud risk, using 15 characteristics trained on 3,400 prior investigations, and projected it would keep 125 residents out of debt collection and save €2.4 million per year. Internal testing found the initial model was 14% more likely to wrongly flag men than women and nearly twice as likely to mistakenly flag applicants of non-Western nationality. After reweighting, tests showed a 20% accuracy improvement, but the live pilot of roughly 1,600 applications did not hold up, and deputy mayor Rutger Groot Wassink announced termination in late November 2023. The city estimated project spend at approximately €500,000 plus €35,000 (MIT Technology Review). The city’s Participation Council had already asked for the experiment to be halted in March 2022, noting only about 3% of welfare benefits were fraudulent (MIT Technology Review; Racism and Technology Center).

Root cause: the missing layer

Organized data was missing. The historical enforcement data encoded the bias the model then reproduced, and even a well-governed pipeline cannot fix a corpus that measures past enforcement rather than actual fraud.

Sources
06 · What the record says

Ownership is the variable

Read the 36 together and the pattern is not a bad model or a careless team. Most of these organizations had serious engineers. What they did not have was ownership of the layer that failed, so when it failed there was nothing to fix. Klarna could not tune the model to its own service bar. Air Canada could not ground the bot in its own policy corpus. Windsurf could not keep access to the weights its product ran on.

AT&T shows the opposite. Because it owns the model, it could test 32 variations and pick on accuracy. Because it owns the data, it could train on 400 billion telecom tokens the frontier labs never had. Because it owns the pipeline, it could process a trillion tokens through open models and save tens of millions of dollars. Because it owns the infrastructure, it could run 40% of its production models on AMD hardware and route every query to the cheapest model that clears the bar. Each layer made the next one work.

The cost story follows from that. A 90% cost reduction is not a procurement win. It is what happens when a company can choose between a frontier model and a 31-billion-parameter model it trained itself, and has the evaluation harness to know which one is good enough for the task.

The sovereignty story follows too. Every generative AI ban in this library happened because the only lever left was prohibition. Every deprecation case happened because someone else held the weights. Ownership turns both from a crisis into scheduled work.

Where SAVRN stands

SAVRN builds the four layers for organizations that do not have AT&T’s scale. The working pipeline: document collection, dataset construction with provenance, model training and evaluation, and the governed infrastructure it runs on. The customer keeps the data, the trained model, and the pipeline.

Start with the data. It is the layer most organizations already own and have never organized. The Research-to-AI dataset tool shows how one document becomes a training row, and the platform shows where the rest runs.

Start with a working session or see the platform this runs on →

Want the next one?

When a new piece publishes on SAVRN Insights, you get one email with what it covers and a link to read it. No digests, no promotions.

One email when it publishes. Unsubscribe in one click. Privacy

You're on the list. You'll hear from us the next time something publishes.
07 · Questions

Frequently asked questions

What is the ownership divide?

It is the gap between enterprises that own the four layers of their AI stack and enterprises that rent them. In this library, one organization (AT&T) owns the model, the data, the pipeline, and the infrastructure. Thirty-six others were missing at least one layer when something went wrong, and the missing layer is where the failure happened.

What are the four layers an enterprise has to own?

The complete model (weights, training recipe, and evaluation harness), organized proprietary data (governed, versioned, with lineage), the dataset-to-training pipeline (gates, curation, and measurement), and the training and inference infrastructure (compute, network, power, and the people who run it).

Why is AT&T the resolution case?

AT&T is the documented example of an enterprise that built all four layers on purpose. It post-trained its own OTel 2.0 model on 400 billion telecom tokens, runs a cache-aware AI Gateway that routes each task to the cheapest capable model, processes about 45 billion tokens a day, and reports AI cost reductions of as much as 90%. Every figure is linked to AT&T, the WSJ, GSMA, Microsoft, AMD, or Google Cloud.

How did AT&T cut AI costs by up to 90%?

By routing. AT&T’s AI Gateway sends each task to the lowest-cost model that meets the quality bar, and can switch models mid-session. Andy Markus, AT&T’s Chief Data and AI Officer, said only a small percentage of tasks need the most sophisticated models. Cached tokens cost roughly a tenth of uncached ones, and open models replaced closed ones in applications where the savings ran 80 to 90 percent.

Does this mean an enterprise should never use a commercial AI API?

No. AT&T still runs roughly three quarters of its usage on closed models and targets 70 to 80 percent open over time. The point of the library is narrower: when an enterprise depends on a rented layer it does not own and cannot evaluate, the failure lands on the enterprise. Owning the gateway, the data, and the evaluation harness is what makes using an outside model a choice rather than a dependency.

How were the 36 failures selected and sourced?

Each case is grounded in a primary document (a court filing, a regulator’s order, a company statement) or a major-outlet report, and every figure links to the page that states it. Cases that could not be confirmed from a fetched source were excluded: an OpenAI fine-tuning API wind-down, Snap My AI cost figures, and Shopify’s AI hiring memo.

What is the most common missing layer?

The pipeline. 15 of the 36 failures had no owned evaluation loop. Quality problems were discovered by customers, courts, regulators, or journalists instead of by the organization’s own tests.

What does model deprecation have to do with ownership?

An enterprise that does not hold the weights cannot freeze a validated model. When the vendor retires it, every prompt, evaluation, and downstream behavior has to be re-validated on the vendor’s schedule. OpenAI and Anthropic have published retirement dates for dozens of models between 2024 and 2027, and requests to a retired model fail.

How does an organization start owning its stack?

With the data. Collect the documents the business already runs on, build a governed dataset with provenance, and put an evaluation set aside before any training starts. SAVRN runs this as a working session: bring real documents, build the first training rows together, and scope the pipeline. Contact [email protected].

Can I cite this library?

Yes. Cite the page URL and the case name. Every figure on the page links to its original source, so you can cite the primary document directly as well.

08 · Sources

Every source behind this library

86 unique URLs. Each fact above links to the page that states it. Cases that could not be tied to a primary source were left out.

We publish the receiptsEvery source behind this library, grouped by case, on one page

Zillow Offers (2021)

  1. Zillow investor release
  2. WSJ

IBM Watson Health (2022)

  1. IBM newsroom
  2. STAT

MD Anderson / Watson (2017)

  1. JNCI
  2. The Cancer Letter

McDonald's / IBM (2024)

  1. Restaurant Business
  2. AP
  3. NYT

Air Canada (2024)

  1. Ars Technica
  2. BBC

iTutorGroup (2023)

  1. EEOC
  2. Reuters

NYC MyCity (2024)

  1. AP
  2. The City

OpenAI / Garante (2024)

  1. Reuters
  2. Garante
  3. Reuters

Samsung (2023)

  1. TechCrunch
  2. Bloomberg

Wall Street banks (2023)

  1. Forbes
  2. WSJ
  3. Bloomberg

Michael Cohen / Bard (2023)

  1. SDNY opinion PDF
  2. NYT
  3. Washington Post

Mata v. Avianca (2023)

  1. Reuters
  2. Mata v. Avianca judgment

Google Gemini images (2024)

  1. Google blog

Alphabet / Bard demo (2023)

  1. Reuters
  2. BBC

Microsoft Tay (2016)

  1. BBC

Deloitte AU / DEWR (2025)

  1. Guardian
  2. AP

Rite Aid / FTC (2023)

  1. FTC
  2. Reuters

UnitedHealth nH Predict (2023)

  1. ClassAction.org
  2. Healthcare Finance News

CNET (Red Ventures) (2023)

  1. CBS News
  2. Machine Learning Times

Sports Illustrated / AdVon (2023)

  1. Futurism
  2. BBC
  3. PBS NewsHour

Amazon recruiting (2018)

  1. Reuters

Windsurf / Anthropic (2025)

  1. TechCrunch
  2. TechCrunch

Cursor / Anysphere (2025)

  1. Cursor
  2. FinTech Weekly

Commonwealth Bank (2025)

  1. ABC News
  2. Bloomberg

Taco Bell (2025)

  1. BBC
  2. TechCrunch

Duolingo (2025)

  1. Fortune
  2. Yahoo Finance

DPD (Geopost) (2024)

  1. BBC

Chevrolet of Watsonville (2023)

  1. Autoevolution
  2. BBC

Pieces Technologies / TX AG (2024)

  1. Texas Attorney General

Dutch Tax Administration (2021)

  1. Pinsent Masons
  2. Racism and Technology Center

Excluded for insufficient sourcing: an OpenAI fine-tuning API wind-down (reported by secondary blogs only, with no primary OpenAI announcement confirmed), Snap My AI cost impact (no primary financial disclosure confirmed), and Shopify's AI hiring-policy memo (no primary document fetched).

Read next