Zerodha's Blobasaur Stores Kite Layouts in Sharded SQLite, and Gives Up Durability to Do It
Software / news
Zerodha's Blobasaur Stores Kite Layouts in Sharded SQLite, and Gives Up Durability to Do It
The Rust, AGPLv3 blob store speaks the Redis protocol, shrank sampled production writes about 11 times, and has no live rebalancing or automatic failover yet.
Zerodha published the design of Blobasaur on Nov. 19, 2026, an open-source blob store that keeps JSON in sharded SQLite files and speaks the Redis protocol. It replaces SSDB for the user interface preferences and workspace layouts on Kite, the broker's trading platform, according to the post on the Zerodha Tech Blog by Sarat Chandra, a software engineer there.
The code is written in Rust and licensed under AGPLv3. Zerodha's production cluster is three nodes with 4 vCPUs and 8 GB of RAM each, and the post gives p99 latency of about 2 ms across a trading session.
Why SSDB had to go
Chandra writes that SSDB was unmaintained, written in C++ outside Zerodha's stack, and ran on a single node. Latency degraded on large payloads, which forced a 1 MB cap on blobs. The blobs are mostly JSON, from a few kilobytes to several megabytes, and about half the traffic is writes.
The team looked at ScyllaDB, PostgreSQL, YugabyteDB, RocksDB, Ceph, SeaweedFS, S3 with MinIO and raw filesystems. The post says each failed on some mix of large-blob performance, disk growth, operational complexity, TTL handling or cost. Zerodha built on SQLite instead.
How the sharding works
Keys map to SQLite shard files by multi-probe consistent hashing. Each shard has one dedicated writer that batches writes into BEGIN IMMEDIATE ... COMMIT transactions, plus a pool of read-only connections. Across machines it implements the Redis Cluster protocol with 16,384 CRC16 hash slots, so existing Redis clients work unchanged. Zerodha's clients are mainly Go and Python.
Resharding is offline. Consistent hashing moves about 20% of keys when a shard is added, against about 80% with modulo hashing, but the migration command still needs the server stopped. The GitHub README confirms that: shard migration "requires stopping the server."
The numbers, and what they do and do not show
The benchmark ran on a 16-core laptop with tmpfs storage, 64 connections and 64 KiB JSON payloads over 100,000 keys. The load generator and the server shared the machine, so the figures describe the design, not a deployment.
- 1 shard7740 ops/s
- 16 shards25K ops/s
Source: Zerodha Tech Blog, Blobasaur post, 2026-11-19
Write-only throughput rose from 7,740 to 25,324 operations a second, and p99 latency fell from 10.7 ms to 8.3 ms. A 50/50 mix went from 14,227 to 29,538. Read-only barely moved: 67,128 against 70,884.
The read result is the telling one. Each shard has its own writer, which fits the write gains, while reads use separate read-only connections and barely changed. That is a reading of the numbers, not a claim the post makes. The post also reports that compression took a one-hour sample of production writes from an average of 63.6 KiB to 5.5 KiB, roughly 11 times smaller.
What it gives up
| Choice | Cost in the post or README |
|---|---|
WAL with synchronous = NORMAL | Recent writes can be lost on power failure |
| Static slot assignment | No live rebalancing or automatic failover yet |
| SQLite files | They do not shrink on delete; incremental auto-vacuum added |
async_write mode | Needs tombstones for read-your-writes |
Chandra says the team accepted the durability trade because the data is non-critical interface state. That is the line to take seriously before copying the design. Blobasaur is built for data that can be regenerated or re-saved by a user, not for ledgers.
The repository has 204 stars, 22 forks, four open issues and no published release in the Releases section, though the README's install example uses v0.8.0. The README gives no benchmark numbers of its own. Everything in the performance section comes from Zerodha's post.
The post says Rust was chosen over Go because Go's SQLite driver requires CGo. The Terminal covered the wider Rust question in Stenberg's weighing of two curl rewrites. A pairing of headline speedup and disclosed cost also showed up in Google's ABFS checkout numbers.
Live rebalancing and automatic failover are the items the post lists as not yet built.
Sources
More in Software
- 01OpenAI's WebRTC Voice Stack Draws a 30 ms Objection That Its Own Engineers Never AddressedLuke Curley says WebRTC drops audio older than 30 ms. OpenAI's engineers wrote about port ranges and routing, which is a different problem.
- 02Superpowers Passes 322,000 GitHub Stars as a Set of 15 Prompted Skills for 16 Coding AgentsJesse Vincent's MIT-licensed framework tells agents to brainstorm, plan and test first. Its clone count is large, and its evidence for better results is one author's view.
- 03GitHub Disables 144 Picofly Repositories After a Nintendo Notice Names OneThe November 6 notice is the first on GitHub aimed at Switch modchip firmware, and it asked for forks to be taken down without listing a single one.
- 04Chrome's Gemini Nano Download: What Google's Docs Say, and the Numbers They Leave OutGoogle's page on built-in model management gives triggers, a 30-day rule and full-download updates, but no file size, no disk threshold and no consumer off switch.