OpenAI's Habitat Hits 70 Million Storage Requests a Second

OpenAI's internal storage platform, called Habitat, now handles more than 70 million requests every second for products used by over 1 billion people each week, the company said in a Sept. 11, 2026 engineering post. Two years earlier, Habitat was a single Python library that talked to one database.

The post is the first in a two-part series and the most detailed account OpenAI has given of the system behind a ChatGPT login, a Codex settings change or a new conversation, each of which the company said can trigger many separate storage lookups before the product responds. That request path has only gotten busier since OpenAI opened its Agents API to beta users, adding another product surface that has to hit Habitat before it can answer anyone. OpenAI said Habitat now serves more than 500 petabytes of data across almost 40 geographic regions, backed by Azure Cosmos DB. The company did not publish a comparable requests-per-second figure for 2024, so the size of the jump from library to platform is not fully quantifiable from the post alone.

Habitat's numbers, two years after the Python script

Habitat began in mid-2024 as what OpenAI called a small client-side library, mapping a handful of operations onto Azure Cosmos DB so product engineers would not have to manage schema lookup, routing, authorization or connection pooling themselves. OpenAI said the platform has grown more than tenfold year-over-year for three years running, a pace it contrasted with the usual systems-engineering habit of building for 10 times current load and expecting that to hold for a few years.

Habitat, per OpenAI's Sept. 11, 2026 post	Figure
Requests handled	70 million+ per second
Weekly users served	1 billion+
Data served	500+ petabytes
Geographic regions	about 40
Year-over-year growth (3 years running)	more than 10x

Why a library had to become a service

By mid-2025, OpenAI said, Habitat had outgrown its life as client-side code. Backward-compatible protocol changes had become impractical, and one migration meant to reduce the blast radius of a regional outage instead caused an outage: coordinating a routing change across dozens of services took days, a shadow-testing pass added more days, a bug fix added more, and the flag was finally flipped only for one team to roll back to a stale client for unrelated reasons. OpenAI's fix was structural. It pulled Habitat into a standalone service so deployments, access control and observability had a single point of control instead of dozens of independently versioned clients.

That decision came with a cost. OpenAI said it kept Habitat in Python rather than rewriting it, accepting worse network latency and higher CPU and memory use than a compiled language would need, on the bet that the migration off Python would be easier once its own coding tools, Codex and GPT, were good enough to do it. The company called this a strategic incursion of technical debt, prioritizing product stability over resource efficiency while its request volume kept compounding.

The bug that punished the slowest server

The clearest evidence of what that debt costs in production is a connection-pooling bug OpenAI described in the post. Habitat's Python clients used the aiohttp library's default connection reuse order, LIFO, which hands out whichever connection was returned to the pool last. During a burst of traffic, OpenAI found that its slowest, already-overloaded server processes returned their connections to the pool last, which under LIFO made those same overloaded processes more likely to be picked for the next request. The result was runaway degradation: a handful of processes kept absorbing 5 to 10 times the load of the average process, a pattern OpenAI compared to the metastable failure state first documented in a 2014 Facebook engineering post-mortem on link imbalance. Switching the connection pool to FIFO reuse broke the feedback loop and lowered steady-state request variance as a side effect. OpenAI said it now leans on Istio and Envoy for load-aware connection pooling across its infrastructure generally, rather than solving the problem client by client.

A second, smaller bug came from Statsig, the feature-flag service Habitat uses for configuration and A/B tests. With up to eight Python worker processes running per pod and Statsig polling for a fresh copy of every production rule across every service once a minute with no jitter, OpenAI found that all eight workers in a pod would periodically stall in unison, burning CPU cycles parsing one large configuration file instead of serving requests. The fix, once profiling identified the cause, was to ship a smaller targeted config, lengthen the refresh interval, and add jitter to the polling schedule.

What OpenAI still isn't saying

The Sept. 11 post does not give a dollar figure for what any of this costs to run, and OpenAI has not said when the promised second part, covering multi-tenancy reliability and its Cosmos DB partnership at what it called unprecedented demand, will publish. The scale is not new context, either: DataCenterDynamics reported in March 2025, citing The Information, that OpenAI was shopping for storage hardware and software to handle five exabytes of data, a separate pool built for training rather than product traffic, and was weighing a dedicated data center near the Stargate site in Abilene, Texas, to house it. Five exabytes is 5,000 petabytes, ten times the 500-plus petabytes Habitat serves today. The two systems are not the same storage, but the ratio tracks the same trajectory Habitat describes: whatever OpenAI is building next assumes the current scale is a waypoint, not a ceiling, the same forward bet Google made when it locked in Finnish nuclear power through 2050.

OpenAI wrote that it bet Codex and GPT would eventually make a full migration off Python achievable, and that the bet eventually proved correct, without naming which language Habitat moved to or when. What would change this read is that missing detail: whether the rewrite is finished, in progress, or still a plan for a platform that has grown more than tenfold every year since 2023.