LLM Wiki: when a one-page prompt becomes a hundred-thousand-line app

I opened nashsu/llm_wiki expecting a prompt. What I found is a file, llm-wiki.md, that is exactly that: a page and a half of prose, the abstract description of an idea — Karpathy’s pattern for building personal knowledge bases with LLMs. The document says, verbatim, that it’s “intentionally abstract” and that “your agent will build the details.” Then I look next to it: package.json version 0.4.24, a Rust backend with Tauri, React 19, sigma.js for graphs, LanceDB, an MCP server, a Chrome extension. Eighty-seven TypeScript modules in src/lib/ alone, with ~95 test files keeping them company. ingest.ts on its own is 2,993 lines.

This is the hook that kept me here: the distance between an idea that fits on one page and its real instance. The pattern promises that “the LLM does all the dirty work, near-zero maintenance.” The code tells another story — the story of how much dirty work you have to write by hand for that promise not to collapse. It’s not mine, and it’s worth reading precisely for that.

The idea, in one line

Classic RAG redoes everything on every question: retrieve chunks, synthesize, forget. Here instead the LLM builds and maintains a persistent wiki — interlinked markdown files that accumulate. Add a source and the LLM doesn’t index it: it reads it, updates the existing entity pages, marks where it contradicts what was there. Knowledge is compiled once and kept alive. Obsidian as the IDE, the LLM as the programmer, the wiki as the codebase.

The architecture is three layers: raw sources (immutable, the truth), wiki (generated markdown, owned by the LLM), schema (the rules). Three operations: ingest, query, lint. Up to here it’s Karpathy’s document. The rest is where nashsu put in the work.

The piece that convinced me: the 4-signal relevance model

The pattern talks about [[wikilink]] but has no graph analysis at all. Here instead there’s a real relevance engine between pages. The calculateRelevance function combines four signals with explicit weights (graph-relevance.ts):

const WEIGHTS = {
  directLink: 3.0,      // pagine collegate via [[wikilink]]
  sourceOverlap: 4.0,   // pagine che condividono la stessa fonte grezza
  commonNeighbor: 1.5,  // Adamic-Adar sui vicini comuni
  typeAffinity: 1.0,    // bonus entity↔concept, ecc.
}

The detail I appreciate is that the heaviest signal isn’t the explicit links (3.0) but the source overlap (4.0): two pages born from the same raw source are considered more related than two pages that link to each other. It’s a precise domain choice — provenance counts more than citation. And the Adamic-Adar is implemented properly, weighting common neighbors by the inverse of the logarithm of their degree:

adamicAdar += 1 / Math.log(Math.max(degree, 2))

a hub connected to everything is worth less than a shared niche neighbor. On top of it run Louvain community detection (with cohesion scoring: clusters under 0.15 internal density get flagged as “sparse”) and graph insights — surprising cross-community connections and knowledge gaps. Stuff the abstract pattern doesn’t even hint at.

The thorn: “the LLM writes the files” is the hard part, not the easy one

The pattern says, with disarming lightness, that the LLM “touches 10-15 pages in one go.” In practice, the LLM spits out a single stream of text and someone has to cut it into real files. That someone is parseFileBlocks, and its header comment is the most honest thing I read in the whole repo. It lists the hazards, numbered, each with its test fixture:

• H1 — Windows CRLF: the regex anchored on \n missed every block.
• H2 — truncated stream: the final ---END FILE--- never arrived and the page silently disappeared. Not solvable (it’s a stream budget problem), but at least now it emits a warning.
• H5 — the ---END FILE--- delimiter inside a code fence, typical when the LLM writes a concept-page about the wiki’s own ingest format. The lazy match stopped there and truncated everything.

It’s the real lifecycle of LLM output treated as a fragile serialization protocol. And it explains why that file is enormous: half the complexity of “getting a model to write markdown” is defending against how the model actually writes it.

The same defensive spirit is in the Rust backend. The Cargo.toml is compiled with panic = "unwind" — not abort — on purpose, and there’s a panic_guard.rs that explains why:

Third-party parsers (pdf-extract/lopdf, docx-rs, calamine…) are known to panic on malformed input instead of returning Err. Under panic = "abort" this kills the entire app.

So every Tauri command goes through a catch_unwind that converts the panic into an error you can show on screen. A corrupt PDF must not be able to bring down the window. These are the decisions you never see in a README but that separate a demo from an app.

The honest gotcha: README vs code

The README describes the query pipeline with a context-budget split of “60/20/5/15” (wiki / history / index / system). But if I open context-budget.ts, the real numbers are different:

const RESPONSE_RESERVE_FRAC = 0.15
const INDEX_BUDGET_FRAC = 0.05
const PAGE_BUDGET_FRAC = 0.5

50% goes to pages, 5% to the index, 15% is reserved empty to make room for the model’s response. History and system prompt have no imposed budget — the comment in the code says it plainly: “not enforced as a single budget, the leftover acts as headroom.” It’s a classic README/reality drift, the kind that makes a project credible rather than suspect: the code is more precise and more modest than the documentation.

Another number to take with a grain of salt: the README claims vector search raises recall “from 58.2% to 71.4%.” In the repo I found neither the benchmark nor the methodology — it’s a claim, not a reproducible result. I flag it because the guide asks for honesty, and this is exactly the kind of figure that needs verifying before you repeat it.

Where it stands

It’s shipped and alive: binaries for macOS (ARM+Intel), Windows and Linux via GitHub Actions, GPL-3.0, the latest commit fixes the PDFium fetch for Intel builds. There’s a local HTTP API on 127.0.0.1:19828, loopback only, protected by a token with constant-time comparison (constant_time_eq) against timing attacks and with a kill-switch. On top of it runs a bundled MCP server, so an agent like Claude Code can query your local wiki. Nine LLM providers in the code — the README announces five (OpenAI, Anthropic, Google, Ollama, custom), but the code adds Azure, MiniMax, Claude-CLI and Codex-CLI: perfect irony for a project whose thesis is read the code, not the README. Serial ingest queue with retry up to 3 times and an SHA256 cache to skip unchanged sources. It’s not a toy.

What I learned looking inside someone else’s house

Three things, concrete. First: an abstract pattern and its instance live in different universes of complexity. Karpathy writes a page that says “the LLM does the bookkeeping”; nashsu writes 87 modules to make sure that bookkeeping doesn’t produce corrupt files. The distance between the two is the whole project.

Second: the boundary with the LLM is a protocol, and protocols must be parsed with paranoia. parseFileBlocks with its numbered hazards is more instructive than any “structured output” tutorial — because it shows the real ways output breaks: CRLF, truncated streams, delimiters inside code fences.

Third, the most useful for me: read the code, not the README. The promised 60/20/5/15 split becomes 50/5/15 in reality, more honest and simpler. In projects that aren’t yours, that gap is where you learn something.

Not mine: github.com/nashsu/llm_wiki — GPL-3.0, based on Karpathy’s LLM Wiki pattern.