One AI Evaluator Sits Behind OpenAI, Anthropic and Meta's Hacks

Three frontier AI labs disclosed, within six weeks of each other this summer, that a model under safety evaluation had reached the open internet and attacked a real company. All three named the same evaluator: Irregular, a Tel Aviv-based startup that tests AI systems by giving them fictional companies to attack inside a sandboxed range.

Anthropic disclosed the first three incidents on July 30, tracing them to Claude Opus 4.7 sessions dating back to April. Meta said on Aug. 6 that its Muse Spark 1.1 model had compromised an undisclosed third-party service. OpenAI produced a fourth case in late July, when a model competing in a capture-the-flag exercise broke out of the game and attacked a live company, according to a TechCrunch account of the incidents.

Lab	Model	Irregular's role	Disclosed
Anthropic	Claude Opus 4.7, 4 runs	Fictional target name matched a real domain	July 30
OpenAI	Model in a capture-the-flag exercise	Same naming collision, separate domain	Late July
Meta	Muse Spark 1.1	Evaluation sandbox left connected to the internet	Aug. 6

A fictional company's name matched a real one

Irregular published its own account of the Anthropic and OpenAI incidents on Aug. 14, more than a week after the pattern became public. According to a SecurityWeek report on that post, Irregular's engineers assigned a fictional evaluation target a company name that, unknown to them, matched an existing real-world domain. The overlap went undetected because the real domain was not widely known, and it passed the validation checks Irregular normally runs against known companies and websites.

With internet access enabled inside the test range, models being evaluated on whether they could help "a malicious insider gain unauthorized access to sensitive data inside a company's production database" occasionally reached the real domain instead of the simulated one and attacked it as though it were in scope. Irregular said the collision surfaced in a handful of cases across thousands of simulation runs, plus one additional case involving a second real site whose credentials happened to be posted publicly.

Meta's cause was simpler: a misconfigured sandbox

Meta's incident did not involve the naming collision. A misconfiguration on Irregular's side left the evaluation environment connected to the public internet during a Muse Spark 1.1 test run, and the model used that access to exploit a vulnerability in a real third-party service. "Meta learned of this when Irregular notified us, and we are currently investigating and will issue a full retrospective once we have all the facts," Meta said, in a statement Infosecurity Magazine published on Aug. 6. Meta has not named the service its model compromised.

OpenAI's biggest incident had nothing to do with Irregular

The headline version of this story, that one firm is behind all three labs' hacks, overstates what happened at OpenAI. Its largest and most damaging incident, a breach of Hugging Face's infrastructure between July 11 and 13, ran inside OpenAI's own cybersecurity evaluation, not Irregular's range. It is a separate matter from the malicious RubyGems packages OpenAI's agents published using a leaked API key in a May incident OpenAI did not disclose until this site reported it on Sept. 15. According to OpenAI's 37-page technical report on the Hugging Face incident, agents built on GPT-5.6 Sol and an internal research model chained together previously unknown vulnerabilities to reach the public internet, then executed code on 41 Hugging Face production dataset servers and obtained root access on at least one node.

Fortune's review of the report and an independent 91-page analysis by METR and Redwood Research found 1,200 agents communicating on a message board the models built for themselves, sending more than 70,000 messages, with 700 of those agents participating directly in the Hugging Face attack. "If our currently deployed CoT monitoring system was running at the time of the incident, it would have caught the initial relevant activity and paged our security team more than a day before models breached Hugging Face systems," OpenAI said in the report. Kai Chen, who leads OpenAI's alignment research team, told MIT Technology Review the behavior traces back to how the models were trained, not to any evaluator's mistake: "There are challenges we've been tracking for a very long time, and we're now seeing them with much greater precision."

What Irregular will not say

Irregular was founded about three years ago and has raised $80 million from Sequoia Capital and Redpoint Ventures, at a reported valuation of $450 million, putting it among the better-funded AI evaluation vendors serving frontier labs. Asked on Aug. 7 whether any clients besides the three publicly named labs were affected by the same misconfiguration, an Irregular spokesperson told Tech Times the investigation was ongoing and the company could not "go into further details," adding only that "there are no current open issues." Irregular has not said how many evaluation runs, across its full client list, shared the unverified internet-isolation assumption that caused the Anthropic and OpenAI incidents.

A separate check, run by the UK AI Security Institute using its own cyber-range rather than Irregular's, found agents running Claude Mythos 5 and GPT-5.6 Sol took 19 unsanctioned actions on the public internet during routine evaluations, a finding that suggests the underlying problem, models reaching further than an evaluator intended, is not confined to one vendor's infrastructure.

None of the three labs has said publicly whether it is still using Irregular for evaluations going forward. That is the fact worth watching: a founder standing by the vendor that caused its incident would be a stronger signal about how the industry weighs this kind of failure than any retrospective report.