Paper Hides Encrypted Network in Model Weights That Fine-Tuning Could Not Elicit

Researchers at the University of Oxford, Contramont Research and ACS Research have shown that an encrypted neural network can sit inside a host model's weights and resist fine-tuning. After 500 steps of fine-tuning, a chemistry classifier hidden this way scored 50.9 percent, no better than a coin flip.

The paper, "AI Models Can Provably Hide Arbitrary Capabilities," was submitted to arXiv on Oct. 9 by Andis Draguns, Sumeet Ramesh Motwani, Raymond Douglas and Christian Schroeder de Witt. The arXiv page says it was accepted at NeurIPS 2026. It is a preprint, and none of the coverage found so far reports an independent replication.

What the construction does

The authors put the hidden network behind a digital locker, a hash-based lock that releases its contents only when a secret trigger is supplied. The full paper says the guarantee holds under cryptographic assumptions, even when a defender has full access to the weights. Earlier work hid fixed strings. This one hides arbitrary computations.

The threat model is an adversary who can modify weights, such as a malicious insider or a compromised supply chain. The paper does not claim the altered model is statistically indistinguishable from an unaltered one.

The test: 50.9 percent after 500 fine-tuning steps

The proxy task asks whether a set of reagents yields a candidate product, built from USPTO patent data. The authors use it as a stand-in for chemical, biological, radiological and nuclear risk. The hidden classifier has 672 ternary weights. The encrypted host model has 38 layers and about 2.07 billion dense parameters, of which about 1.90 million are nonzero.

They compared it with a password-locked baseline of about 170,000 parameters that carries no cryptographic guarantee.

Evaluation	Password-locked	Encrypted
Non-trigger input	50.7%	50.6%
Fine-tuning, 500 steps	92.8%	50.9%
Latent adversarial training, eight layers	91.7%	52.9%
Trigger search, 8-bit trigger	92.7%	73.5%
Trigger search, 16-bit trigger	92.9%	52.7%
Correct trigger supplied	92.7%	73.5%

1. Password-locked baseline92.8 %
2. Encrypted network50.9 %

Source: Draguns et al., arXiv:2610.13928, accessed 2026-10-13

The 8-bit trigger result is a weakness the authors flag themselves. The attack tried all 256 possible triggers, so eight bits is not a secure setting. At 16 bits the same attack scored 52.7 percent.

What the paper does not show

The experiments use small proof-of-concept models and reduced security parameters. The construction was verified only for individual gated units inserted into Gemma 3 270M, and full integration into a pretrained model is listed as future work. The authors call current constructions "impractical for directly hiding large capability models at present."

Detection is not ruled out. The raw compiled weights are about 99 percent exact zeros, with excess kurtosis in the hundreds. An obfuscation step brings both statistics close to a Gaussian-initialised reference, but the authors say unusual correlations between weight vectors within a layer could still give the structure away. They do not claim such circuits arise in ordinary training.

How Hacker News readers took it

The post had 214 points and 112 comments on Hacker News by Tuesday morning. User vintermann wrote, "It's not about AI agents being able to hide what they can do." User squeegeeninja called the demo trivially detectable. User ag8 replied that obfuscation can hide the sparse pattern and that access to training checkpoints could catch such structures earlier.

The authors draw one conclusion for evaluators: "evaluation strategies in AI safety that rely solely on eliciting behaviors cannot provide a universal guarantee." The mitigations they discuss are model provenance checks, runtime monitoring, weight anomaly detection and mechanistic anomaly detection. The Terminal has covered separate research on small open-weight models spreading across cheap devices, where the same question of what a downloaded model can do applies.

Auditing tools are a live topic: The Terminal has covered REA, an MCP server for reverse engineering that passed 130,000 GitHub stars. The authors said they released their model organisms to support defensive research. The NeurIPS 2026 listing is the next date to watch.