NeuroAI NEUROAINEUROAI.SITE
China AI, reported clearly
1,300+ verified stories on models, robotics, BCI and the business of AI.
Browse stories
ESC

The Model Never Failed. The Login Did, for Three and a Half Hours

OpenAI's authentication layer failed for 211 minutes on Nov 21, with request failures peaking at 75 percent after both Postgres read replicas went down in one maintenance window.

2026-11-22 · 4887 words · NeuroAI
The Model Never Failed. The Login Did, for Three and a Half Hours

At 1:48 p.m., logins began to fail.

Pacific time, November 21. That is the starting point recorded in the incident write-up. Service came back at 5:21 p.m. In between: 211 minutes, or three hours and 31 minutes.

Most of the people who use the service in Asia were asleep, which is why the Chinese internet barely noticed. But by Pacific time this was a Saturday afternoon, one of the lightest load windows of any week. A weekend, off-peak, and the work was planned maintenance.

Low load, a weekend, maintenance only. It still went down.

What Actually Stopped

Look at which layer stopped.

Not inference. Not model weights. Not a GPU queue. What stopped was login, the step every single request passes through on its way in.

The write-up says the failure was triggered by maintenance on a PostgresDB service used by OpenAI's Primary API service, and that the maintenance caused both read replicas to fail at the same time.

The Primary API service is not a peripheral system. It handles user authentication and service-to-service authentication across the whole stack, by the company's own description. That sentence matters because it translates a database problem into a total-product problem. Embeddings through the API, voice mode in the app: everything walks through the same door, and when the door stops working, a faster model behind it changes nothing.

One leg holding up the building.

Nine Endpoints, Five Features

The blast radius can be counted.

Nine on the API side: Chat Completions, Fine-tuning, Embeddings, Images, Batch, Audio, Moderations, Realtime, and the Files endpoint.

Five in ChatGPT: login, conversations, search, file uploads, voice mode.

Nine plus five. Fourteen surfaces.

What makes the list worth reading is how far apart those surfaces sit. Embeddings, fine-tuning, image generation, speech synthesis, file transfer, batch jobs. Different codebases, different teams, different release cadences. They should not share a dependency. Yet one authentication step strung all of them onto the same rope, so a single database service sneezing gave fourteen surfaces a fever.

Fourteen surfaces. One door.

Three-Quarters Is Not a Full Outage

The 75 percent needs care.

That figure is a peak, not an average across the incident. And a 75 percent peak failure rate reads, in reverse, as roughly one request in four still getting through.

That is harder to handle than a clean outage.

A clean outage tells everyone to start fixing, gives support a script, and leaves users in no doubt. With a quarter of traffic still succeeding, monitoring keeps producing successful samples, alerts can sit below a threshold, and what users feel is not "broken" but "flaky." Refresh twice, it works, most people stop reporting.

The people debugging end up waiting for a stable failure that never arrives.

The Replicas Would Not Come Back

The order of the repair is more useful than the conclusion.

Step one was trying to revive the two failed replicas. It did not work.

Step two was trying other regions. That did not work either. On paper this is the most standard move available, the first line of any disaster-recovery runbook. In practice it depends on where the data lives, whether a replica can be grown in place, and whether a new one has to replay everything from the start.

Step three was bringing two new replicas into service in the same region, working with the infrastructure provider. That held.

At 5:21 p.m., service returned.

The provider is not named. And there is a more interesting line in the same write-up: the company does not explain why one maintenance operation took down both replicas at once.

Six Fixes, Three Urgent

The remediation list counts to six.

Three could be done the same day: pause similar maintenance, retune the database to cut connection churn, and ship a feature flag that can move traffic off the replicas by percentage. None of those change the architecture; they lower the odds of a repeat.

The other three take longer: a review process before maintenance, enough hot standby capacity held during maintenance windows, and new defenses against replica lag. The slow three are the ones that actually remove "both replicas die together" from the space of possibilities.

The percentage-based traffic flag is the most revealing item, because its existence implies it did not exist before. When the replicas failed, the traffic had nowhere else to go, so the only option was to wait for the replicas to return.

The Second One in Three Days

This was not the week's only incident.

On November 19, error rates rose for gpt-6 Luna API requests. By UTC, the record shows a report at 16:01 and a resolution at 16:58, fifty-six minutes.

Two days later: 211 minutes.

Fifty-six against 211 is a factor of 3.8. The difference is not only duration. The first was one model wobbling; the second was authentication for the whole stack failing. The first hit some callers; the second hit everyone who needed the door.

The Next Maintenance Window

As a structural matter, there is one question.

The model layer got multiple replicas, multiple regions and multiple suppliers over several years, because redundancy there is common sense. The authentication layer had two replicas, and that was enough, until it was not. Two standards inside one company, and the reason is not hard to guess: the model is the product, authentication is the plumbing. Nobody adds budget to plumbing until it leaks.

That asymmetry is not unique to one company. Any product that treats a single login as the front door sits on the same chain. The more concentrated the request path, the simpler the system, and the more brittle.

When the next maintenance window of this kind arrives, the status page will say so.

The six-item list will be the exam.

Disclaimer: This article is based on public information and is for reference only; it is not investment advice.

Related coverage

Cite this article

NeuroAI (脑机网), "The Model Never Failed. The Login Did, for Three and a Half Hours," 22 November 2026, https://neuroai.site/a/na-app-login-outage-1122

Machine-readable: BibTeX · RIS · JSON · Markdown

More in “AI in Action” → · Back to home · Markdown version

China AI, reported clearly
1,300+ verified stories on models, robotics, BCI and the business of AI.
Browse stories