Hindsight Wants to Give AI Agents a Real Memory

Hindsight, an open-source memory system for AI agents built by Boulder, Colorado-based Vectorize, was the single most-starred repository on GitHub's trending page on Sept. 24, gaining more than 1,600 stars in one day. The pull is a specific number: 91.4 percent accuracy on LongMemEval, a benchmark for testing whether an AI agent can recall and reason over information from earlier in a long conversation.

Vectorize published that figure on Dec. 16, 2025, alongside its MIT-licensed release of the project, and says the score has since been reproduced independently by researchers at Virginia Tech's Sanghani Center for Artificial Intelligence and Data Analytics and by an applied machine learning scientist at The Washington Post. "Agent memory is one of the most critical unsolved problems in AI right now," Andrew Neeser, the Post's applied machine learning scientist, said in Vectorize's own announcement. Naren Ramakrishnan, who heads AI and machine learning for Virginia Tech's Institute for Advanced Computing, put the failure mode more bluntly in the same release: agents "will execute a task flawlessly once, then get it wrong the next."

What 91.4 percent actually measures

LongMemEval scores an agent's ability to hold up across multiple sessions, not just one long context window. On Vectorize's own breakdown, a full-context baseline answered multi-session questions correctly 21.1 percent of the time; Hindsight raised that to 79.7 percent. Temporal reasoning questions went from 31.6 percent to the same 79.7 percent, and questions requiring the agent to recognize an old fact had been superseded by a new one went from 60.3 percent to 84.6 percent. The 91.4 percent headline number came from running Hindsight on top of Gemini 3 Pro Preview as the underlying model; Vectorize also reported strong results pairing Hindsight with GPT-OSS 120B, OpenAI's open-weight model.

Task type	Full-context baseline	With Hindsight
Multi-session recall	21.1%	79.7%
Temporal reasoning	31.6%	79.7%
Knowledge updates	60.3%	84.6%

Those are vendor-reported figures, run on Vectorize's own benchmark harness, though the two outside citations give them more weight than a typical vendor chart. Neither Virginia Tech nor The Washington Post has published its own independent methodology or a separate score alongside Vectorize's; both are quoted only inside Vectorize's release.

How it stores what an agent has seen

Hindsight organizes what an agent learns into four categories: world facts, individual experiences, observations, described as "consolidated, evidence-backed beliefs," and mental models, its term for synthesized understanding built up over time. New information is meant to strengthen, weaken or extend an existing belief rather than silently overwrite it. Retrieval runs four strategies at once: vector similarity search, BM25 keyword matching, a graph of entities and temporal and causal links between them, and plain time-range filtering. The system stores this in Postgres with the pgvector extension, or in Oracle AI Database 23ai, and Vectorize says it supports more than 25 LLM providers.

The project's own numbers show real production use, not just benchmark chasing. Vectorize said in an April 22 post marking Hindsight's first 10,000 GitHub stars that the project had reached that mark in four and a half months, with 598 forks, 49 releases and 77 contributors, and that 90 percent of the issues filed against it were bug reports rather than feature requests, a ratio the company read as evidence people were running it against real workloads rather than just trying the demo.

What ships free and what does not

The core project is MIT-licensed, meaning the code itself carries no restriction on commercial use or modification. Vectorize's paid layer is Hindsight Cloud, a hosted version billed on usage with free credits to start rather than a fixed monthly or per-seat fee, according to the project's own documentation. The most recent open-source release, version 0.10.1 on Sept. 21, was mostly maintenance: fixes to attachment handling, memory redaction, OAuth flows and a deadlock that could occur when a delete and a re-ingest ran at the same time. The prior release a week earlier replaced the tiktoken tokenizer with a faster one called quicktok.

What is missing from Vectorize's own materials is any comparison naming specific competing memory systems by their scores on the same benchmark, or a public breakdown of what the reproduction work at Virginia Tech and The Washington Post actually consisted of. Vectorize has also not said what, if anything, changed in the underlying technique between its December announcement and the version now trending, beyond the incremental bug fixes visible in the changelog. For a project whose selling point is that it remembers, the most recent releases have mostly been about not forgetting how to run.

The same appetite for agent tooling that pushed Hindsight to the top of GitHub's trending list on Sept. 24 is visible elsewhere in the ecosystem: Cua's Computer-Use 2.0 shipped its own open-source take on giving agents persistent state, and AWS's push to make Pydantic AI agents durable on Lambda addresses a related problem, keeping an agent's progress intact across a crash rather than across a conversation.