MiniLake Fakes the Databricks API on Localhost With DuckDB, and Accepts Any Token

MiniLake 1.9.1, an MIT-licensed emulator of the Databricks REST API, lets the Databricks SDK, CLI and Terraform provider talk to http://localhost:8000 instead of a cloud workspace, according to its project page. It reached Hacker News on Oct. 29 with 94 points.

The Terminal did not install it. Everything below comes from the project page, the GitHub repository and Databricks' own documentation. The page says the project is not affiliated with or endorsed by Databricks, Inc.

What problem a Databricks emulator solves

Databricks' own guidance for testing is to run pytest against real Databricks compute. Its testing guide builds a get_spark helper on DatabricksSession.builder.getOrCreate(), which connects to remote compute, and says Databricks Connect and PySpark are mutually exclusive. A continuous-integration job that follows that advice needs a workspace, a credential and a bill.

MiniLake removes all three. The project page lists pip install minilake && minilake --port 8000 and a Docker image at ghcr.io/dmux/minilake:latest, with Python 3.11 or newer and Docker as requirements. It says the image downloads nothing at runtime and can be checked with docker run --network none.

The free tier is the other option, and it has walls. Databricks' Free Edition limits allow one SQL warehouse at the 2X-Small size and a maximum of five concurrent job tasks per account. Going over quota shuts compute down for the rest of the day, and in extreme cases the rest of the month. Free Edition is also non-commercial use only.

What MiniLake actually runs

The project page says statement execution runs on DuckDB, an embedded analytics database, with each catalog stored as its own DuckDB file so that three-part names such as catalog.schema.table resolve natively. Unity Catalog SQL functions become DuckDB macros.

Delta tables are handled differently. EXTERNAL Delta tables are real Parquet files with a _delta_log/ directory, and INSERT, UPDATE and DELETE on them run through a generated Spark job in a sibling container. Reads go through DuckDB's delta_scan().

The page counts more than 30 API groups, 67 tools on a built-in Model Context Protocol server, and more than 378 tests that it says are driven by the real Databricks SDK rather than mocks. Those are the project's own figures. The MCP server is off until MINILAKE_MCP=1 is set, and the page gives a claude mcp add --transport http line for pointing an agent at it.

1. Tests (more than)378 count
2. MCP tools67 count
3. API groups (more than)30 count

Source: MiniLake project page, dmux.github.io/minilake, accessed 2026-10-30

What is real and what is a stub

The project page sorts its coverage into four groups, and the split decides whether a passing test means anything.

Area	Status per project page	What it means
SQL statements, EXTERNAL Delta, Jobs, Unity Catalog	Real	Runs actual queries and tasks
Grants, policies, SCIM, tokens	Unenforced CRUD	Stored and read back, never checked
Clusters	State machine only	No Spark compute on clusters
DLT, Model Registry, Vector Search, Lakeview, Repos	Not implemented	Return 501 NOT_IMPLEMENTED

The README calls these gaps "deliberate, not oversights," because the project targets one developer running it locally. It also says jobs.py, sql_statements.py and unity_catalog.py are mostly tested on happy paths.

The three limits that bite first

Authentication is the first. The README says "No real authentication — any token is accepted," and the page warns against exposing the server beyond localhost. A test that checks a permission-denied path will pass or fail on nothing real, because the Permissions API is always allow-all.

Dialect is the second. DuckDB SQL is not Spark SQL, so a query that runs in a MiniLake test can still fail on a real warehouse. The project page says its MCP server ships a dialect guide for exactly this reason.

Managed tables are the third. They are DuckDB tables with no files behind them, so Spark cannot see them, and anything that needs Spark has to use EXTERNAL Delta. DuckDB is also single-writer, so concurrent load serializes on a lock.

How big the project is

The repository shows 125 stars, 15 forks, 237 commits on main and one open issue. The author is the GitHub user dmux; neither the page nor the repository page names a person. Neither shows a release date for 1.9.1.

That is a one-maintainer project with a short public record, and a team deciding to put it in CI is betting on that. It is a similar bet to the one behind firmware simulators, which The Terminal covered in Simantic's Renode-based product, where the fake exists so that tests can run before the real thing is available. Agent tooling that talks to such fakes is the subject of Cursor's Agent SDK release.

The number to watch is the 501 list. DLT, the Model Registry and Lakeview are the pieces a larger Databricks project leans on, and the repository's FEATURES.md is where the maintainer says status changes will be recorded.