An agentic knowledge system
Keeping a personal knowledge base coherent as it grows, by letting an agent do the reconciling and a human make the final call.
On this page
Knowledge Compiler is an agentic knowledge system. It keeps the notes you accumulate organised, current and consistent over time, instead of letting them rot into a pile you never reopen. The hard part of personal knowledge was never writing things down; it's everything that has to happen afterwards.
A note is an input to be compiled into knowledge, not a finished document to be filed and forgotten.
Agentic workflow
A knowledge base decays in predictable ways as it grows. There's soon too much of it to hold in your head, so you stop revisiting it. Older notes quietly go out of date without anyone noticing. The same topic ends up scattered across a dozen half-finished notes, and every so often a new note flatly contradicts an old one. On top of that, every time you add something, reconciling it against everything you've already written is slow and tedious, so most people simply don't, and the base rots.
The fix is to hand that reconciliation to an agent. When you add a raw note, which the system calls a source, the agent runs a small, guarded tool loop to work out how that note should land in the base. It can load the raw source (get_source), resolve the note's concepts against the concept index with optional fuzzy matching (lookup_concepts), and search the existing knowledge for related material (search_blocks), since the base is stored as blocks rather than whole notes. When it needs more detail it can read a specific block or its edit history (get_block and get_block_history), and it can check that a draft is genuinely grounded in the source before committing to it (verify_grounding).
The loop only ends when the agent reaches a decision. Two terminal tools close it out: draft_proposal classifies the outcome as a merge, an update or a new entry and drafts the change, while finish_without_proposal stops cleanly when a complete proposal can't be made.
What keeps this reliable is that the order is enforced rather than hoped for. Each tool carries a guard, so nothing runs before the source is loaded, and a block can't be fetched unless an earlier search actually surfaced it, which stops the agent acting on an id it made up. Just as important, whatever the loop produces is only a proposal. It never writes to the knowledge base directly. You review the merge, update or new entry and decide what to apply, so the agent does the tedious work of finding what's related and drafting the change while you keep authorship and the final call.
The plumbing underneath is deliberately boring. The agent runs on the OpenAI Agents SDK, behind a clean-architecture Express and TypeScript backend where routes stay thin, services hold the business logic without touching raw SQL, and repositories own the database queries. Data lives in PostgreSQL, and the client is React with Vite.
Hybrid retrieval and eval
Answering a question reuses the same knowledge through a different path. Retrieval is hybrid and runs mostly at the block level: a query fans out to several retrievers at once and their results are combined. A Postgres full-text search acts as a baseline fallback. BM25 lexical ranking, through ParadeDB, handles keyword relevance. For meaning rather than wording, the query is embedded and matched by cosine similarity through pgvector. There is also a note-level match against the concept index, though that one is currently dormant, because the query-side concept resolver isn't wired up yet, so it contributes no candidates for now.
From there the pipeline is fixed. The separate rankings are merged with Reciprocal Rank Fusion (k=60), then expanded one hop across approved note links to pull in directly connected notes. That produces the top N blocks, each carrying its citation, which are handed to an answerer running an exact-question scoping prompt so the reply stays tied to what was actually retrieved.
Evaluation falls naturally out of this shape. Because every answer is assembled from cited blocks, and because the compile-time agent's drafts have to pass a grounding check, the system is built to be judged on whether an answer is genuinely supported by its sources rather than merely plausible.
Future work
There's plenty I still want to do. The most concrete item is wiring up the query-side concept resolver so the dormant concept-index retriever actually contributes candidates at query time. Beyond that I want the agent to get sharper at spotting genuine contradictions and at proposing a right-sized merge instead of over-merging, to expose the concept graph so the base can be browsed by concept rather than only searched, and to make the review step fast enough that accepting or rejecting a batch of proposals is effortless.
Conclusion
An agentic knowledge system earns its keep by doing the reconciliation you would otherwise skip. It finds what's related by concept and proposes what to merge, update or add, then it stops and lets you decide. The agent proposes and the human approves, and that division of labour is what lets a knowledge base keep growing without quietly falling apart.