Third-party source review · September 3, 2026

The graph is not the trust layer.

Blake McCarn's Paperless knowledge graph is interesting because it treats evidence, freshness, and failure as parts of retrieval—not because it adds a chat box to scanned documents.

Evidence label: source review, not field test

We reviewed McCarn's case study, inspected the current public repository, and checked related projects. We did not connect the stack to his private archive, reproduce the reported corpus results, or score answer accuracy. There is no commercial relationship or endorsement.

The pipeline worth copying

Paperless source documents
  → baseline OCR → selective enhanced OCR
  → classify → extract → verify → relate
  → graph + vector + keyword indexes
  → quick | deep | timeline | strict query
  → cited answer + claim ledger + trace

The project reports more than 800 documents, roughly 7,000 graph nodes, 25,000 relationships, and 6,000 searchable chunks. Those are author-reported operating results, not Software Sausage measurements.

Five decisions make it more credible than generic document chat

1. Preserve the source system. Paperless remains the document authority. Enhanced OCR is a separate companion layer, so the archive is not held hostage by the AI pipeline.
2. Compare identities, not counts. Freshness checks compare exact document ID sets and hashes across Paperless, the graph, and vector chunks. Equal counts can still conceal a missing document and a stale replacement.
3. Route by question shape. Entity lookups use the graph; deeper questions combine vector, keyword, and graph retrieval; timeline questions retain explicit dates; strict mode can refuse weak evidence.
4. Leave inspectable artifacts. The repository contains evidence-pack, claim-ledger, trust-dimension, trace, and answer-repair paths. A separate model pass can catch inconsistencies, but it is not independent ground truth and can share the drafter's blind spots.
5. Keep model routing outside the application. LiteLLM centralizes aliases, keys, limits, and cost visibility. That reduces provider coupling; it does not remove the need to test every provider's authentication and response behavior end to end.

What the repository proves—and does not

The source contains the advertised evidence helpers, exact-drift audit, API smoke checks, and an evaluation harness for fixed questions. Its published container workflow is currently green, but it builds images rather than gating publication on those checks.

The Python dependency file is mostly unpinned. The README says MIT, while the repository had no LICENSE file and GitHub detected no license when we reviewed it. Until that is corrected, treat the code as publicly readable—not as reusable MIT material.

Most importantly, no public fixture demonstrates the high-stakes accuracy implied by medical, tax, financial, and legal query modes. Strict refusal and visible citations are good controls; they are not a substitute for source review.

The same process appears across Blake's work

Portable Brain applies the portability boundary to knowledge work: plain Markdown, YAML, and relative links remain usable without a runtime, while source evidence stays separate from synthesized wiki notes. We ran its current test suite locally: 75 tests passed. A full-repository lint also found one executable-bit issue in an optional Paperless integration that the project's narrower CI lint scope does not cover.

His Herdr write-up separates terminal topology from provider logic: one worktree per implementation, distinct review and test surfaces, and status labels used for routing rather than proof. His LiteLLM operating note makes a similar boundary explicit for model traffic, budgets, and credentials.

The field test we should run

Use a synthetic, non-sensitive Paperless archive with ordinary text, tables, handwriting, duplicate entities, contradictory dates, and one deliberately stale index. Freeze a dozen questions and score source coverage, exact-value accuracy, unsupported claims, refusal behavior, elapsed time, and model cost. Then remove a document without changing the total count and require the freshness check to catch it.

That would turn a strong architecture story into a reproducible Software Sausage recipe. Until then, the honest label is community source review.

Follow the verification run

One maintained workflow at a time. Explicit opt-in, no vendor-paid ranking, unsubscribe any time.

Inspect Blake's repository ↗Read our methodology