Architecture

<!-- translations:start -->

한국어 · 中文 · 日本語 · Русский · Español · Français · Deutsch

<!-- translations:end --> Tesserae is a context engine. It reconstructs a self-improving knowledge base from your project and hands it to agents as ready-to-use context. It runs on three pillars: (1) session monitoring — watch live agent/work sessions and capture findings as they happen; (2) autonomous, proactive knowledge ingestion — a pipeline + supervisor loop that pulls in and re-extracts knowledge continuously, improving the base rather than waiting to be told; (3) on-demand docs/context — user-requested artifacts compiled from that same base. The typed graph, the markdown vault, and the static site are projections of the knowledge base; the engine is the loop that keeps them fresh and feeds agents.

Underneath, Tesserae turns a directory of source material into a controlled, typed knowledge graph and projects that graph through a durable markdown wiki layer into a static, AI-friendly website. The April 2026 redesign reorganised the projection side around a Karpathy three-layer model: raw evidence stays raw, a typed graph governs ontology, and a markdown wiki layer sits between the graph and any rendered output. The static site is a renderer of that wiki layer rather than a direct dump of the graph, with the controlled ontology in MD0 as the schema. The v0.5.0 milestone (June 2026) added the engine spine that drives all three pillars — see Engine spine and On-demand context compiler below.

The Karpathy three-layer model

Andrej Karpathy's framing for LLM-friendly knowledge bases distinguishes three layers, each with its own durability guarantee:

The schema sits across all three layers as a separate axis: ResearchGraph in graph.json is the controlled ontology that L2 pages link against, and the ResearchNodeType / edge whitelist in MD3 is the source of truth for what types exist at all.

The redesign added L2 explicitly. Before April 2026 the static site was projected straight from graph.json; the wiki layer existed only inside the Obsidian vault export. Splitting it out gave us:

• A single human-editable surface (open .tesserae/wiki/ in Obsidian or any markdown editor).
• Idempotent rebuilds: re-running project compile produces zero file diffs unless source content changed.
• An evolution log: synthesis pages accumulate over time and let the project narrate itself.

Pipeline

data/, docs/, src/                                    (L1 raw)
        │
        ▼  project compile  (tesserae/project.py)
┌───────────────────────────┐
│ ResearchGraphExtractor    │   deterministic + selective Claude
│ + canonicalization        │
└───────────┬───────────────┘
            │
            ▼
┌───────────────────────────┐
│ ResearchGraph (graph.json)│   schema: research_graph.py
└───────────┬───────────────┘
            │
            ├──▶ WikiLayerProjector   (one page per L1/L2 node)
            ├──▶ SynthesisProjector   (pulse, daily, weekly, topic, …)
            │
            ▼
┌───────────────────────────┐
│ .tesserae/wiki/  (L2 md)  │   sources/, concepts/, entities/,
│                            │   papers/, repos/, topics/,
│                            │   syntheses/, questions/
└───────────┬───────────────┘
            │
            ▼  StaticSiteBuilder.write_site
┌───────────────────────────┐
│ .tesserae/site/  (L3 html)│   index.html, <kind>/index.html,
│                            │   <kind>/<slug>.html,
│                            │   per-page .txt + .json siblings,
│                            │   llms.txt, llms-full.txt,
│                            │   graph.json, graph.jsonld,
│                            │   search-index.json,
│                            │   sitemap.xml, rss.xml,
│                            │   robots.txt, ai-readme.md,
│                            │   manifest.json
└───────────────────────────┘

Every step is incremental. The graph extractor uses manifest.json content hashes to skip unchanged source files. WikiPageStore.write_page returns False (and skips the write) when the body hash matches what's already on disk. StaticSiteBuilder wipes and rewrites .tesserae/site/, but its output is deterministic — see "Idempotence story" below.

Context compiler dataflow

The on-demand context compiler (MD0) is Pillar 3's headline path. Given a query and/or explicit seed node ids, compile_context builds a tailored, cited markdown bundle straight from the graph and returns it in memory — it writes nothing under .tesserae/.

query / seeds
     │
     ▼  1. Seed resolution
        explicit seeds (kept iff they exist) + hybrid_search() hits, deduped, stable order
     │
     ▼  2. PPR expansion
        retrieval.ppr.personalized_pagerank ranks the depth-bounded k-hop neighbourhood;
        empty result (disconnected seeds) → fall back to seed order (bundle is never empty)
     │
     ▼  3. Budget-bound selection
        walk PPR order, include each node's cited body until the next would overflow
        `budget` chars (budget <= 0 = uncapped; over-budget marker on a word boundary)
     │
     ▼  4. Cited markdown assembly
        one section per selected node + a trailing `## Citations` block.
        Body text prefers the projected wiki page (when a store + public wiki kind exist),
        else the node description, else a minimal stub. The no-LLM body embeds NO
        wall-clock timestamp → byte-identical for the same (graph, query, seeds, depth, budget).
     │
     ▼  5. Optional LLM synthesis  (only when synthesize=true AND ANTHROPIC_API_KEY is set)
     ▼
   ContextBundle { query, seeds_used, ranked_nodes, selected_nodes,
                   citations[ContextCitation], body, synthesized,
                   char_budget_used, char_budget_total }

Defaults: depth=2, budget=32000. The deterministic assembly (steps 1–4) is the contract; LLM synthesis is purely additive. The same pipeline backs the project context CLI command, the compile_context MCP tool, and the topic-scoped export slices (slice_export_context_for_topic, topic-scoped llms.txt).

Module map

Wiki + synthesis (L2)

Graph + ontology

Site renderer (L3)

Pipeline orchestration

Engine spine (v0.5.0 — pillars 1 & 2)

The engine spine is the in-process loop that drives session monitoring and autonomous re-ingestion. The same Pipeline.run() is the single refresh path that the CLI, the supervisor daemon, and (later) the MCP server all call.

Self-improvement memory (v0.5.0 — pillar 2)

Phase 5 activated persistent self-improvement. Mutable per-node state lives in a node_memory SQLite sidecar (inside .tesserae/sqlite.db), separate from the immutable node_provenance.first_seen_at first-seen stamp (a Phase-4 sidecar). The compile drives a set of deterministic passes over the graph.

Recurrence confidence is numeric in the output: the temporal projection stamps each fact's confidence from NodeMemoryRow.confidence (text in SQLite, surfaced via temporal.py), falling back to infer_confidence only when no stored value exists.

Retrieval (v0.5.0 — pillars 2 & 3)

On-demand context compiler (v0.5.0 — pillar 3 headline)

Persistence ports + graph stores

External adapters (unchanged this round)

Project workspace layout

.tesserae/
  config.json                 project name, source kind, source list
  graph.json                  validated ResearchGraph (incl. Synthesis nodes)
  manifest.json               per-source content hashes (input dedup)
  sqlite.db                   SQLite graph store; also owns the node_provenance
                              (first-seen, Phase 4) and node_memory (decay /
                              confidence / superseded, Phase 5) sidecar tables
  temporal_facts.jsonl        Graphiti-style temporal projection (numeric recurrence confidence)
  graphiti_episodes.jsonl     dependency-free Graphiti episode export
  report.md                   graph quality / summary
  competitive_report.md       comparison vs. MegaMem / Graphiti / others
  markdown_projection/        flat human-readable markdown
  obsidian_vault/             Obsidian projection w/ .obsidian/, raw/assets/
  agent_harness/              Claude Code / Codex / etc. harness files
  harness_sessions/           imported local Claude Code/Codex sessions
  wiki/                       L2 markdown wiki — see below
  site/                       L3 static site — see below

.tesserae/wiki/ (L2)

wiki/
  sources/<slug>.md           raw documents from data/ + docs/, with frontmatter
  concepts/<slug>.md          Concept / TechnicalTerm / Algorithm / etc.
  entities/<slug>.md          Model / Dataset / Benchmark / Metric / Org / Person
  papers/<slug>.md            Paper hub
  repos/<slug>.md             Repository / Project / CodeProject
  topics/<slug>.md            ResearchField / ResearchTopic / ApproachFamily / Trend
  syntheses/<slug>.md         pulse, daily_digest, weekly, topic, comparison, field_overview
  questions/<slug>.md         OpenQuestion

Each file is editable by hand; the next compile honours user edits as long as the body hash differs from what the projector would write. (Editing only the body wins; editing the frontmatter loses on next compile because frontmatter is regenerated.) Obsidian users can open .tesserae/wiki/ directly; the existing obsidian_vault/ adapter is a separate projection, not a substitute.

.tesserae/site/ (L3)

site/
  index.html                  home + project pulse
  about.html                  schema, build info
  assets/{style.css,app.js}   single CSS bundle + single JS bundle
  sources/index.html
  sources/<slug>.html
  sources/<slug>.txt          AI sibling — plain text
  sources/<slug>.json         AI sibling — structured record
  concepts/…  entities/…  papers/…  repos/…  topics/…  syntheses/…  questions/…
  sessions/index.html          imported harness-session index
  sessions/<project>/<id>.html session detail: summary, metadata, turn rail, markdown turns, collapsed tools
  timeline/index.html
  graph/index.html            interactive 2D + 3D force layout
  graph.json                  full graph payload (incl. code nodes, for tooling)
  graph.jsonld                schema.org Dataset, wiki-layer nodes only
  search-index.json           palette + page search; wiki-layer kinds only
  llms.txt                    llmstxt.org — short index
  llms-full.txt               llmstxt.org — every page body, capped 5MB
  sitemap.xml                 every emitted route
  rss.xml                     last 30 syntheses
  robots.txt                  permissive (crawl + index)
  ai-readme.md                machine-readable site map
  manifest.json               sha256 + size for every emitted file

What's deliberately excluded

The redesign drew an explicit line: code-class and code-function nodes stay in graph.json (so MCP and Graphiti consumers still see them) but never get HTML pages, never appear in search-index.json, and never appear in the navigation. That's the user-facing contract — the wiki is a document-first knowledge base, not a function browser.

Concretely, StaticSiteBuilder skips any node whose type is not in the L2 wiki kind map (tesserae/wiki_projector.py::_KIND_FOR_TYPE):

• Excluded from L2 + L3: CodeClass, CodeFunction, CodeModule, Dependency, EvidenceSpan, SourceFile, all Claim variants (Claim, ContributionClaim, PerformanceClaim, ComparisonClaim, LimitationClaim, CausalClaim).
• Surface where they still appear: as bullets, badges, neighbour counts, or evidence excerpts inline on related wiki pages, and in graph.json for downstream tooling.

If you need code-level browsing, point an LSP / call-graph tool at the source tree directly — that's a different problem from "wiki of what this project knows."

OKF v0.2 export/import

MD0 projects the graph into a Google **OKF v0.2** bundle — a directory tree of Markdown files with YAML frontmatter whose only required key is a non-empty type. tesserae export okf writes v0.2; tesserae export okf --import DIR reads v0.1 and v0.2. The bundle is a pure projection of graph.json: no wall clock, no os.stat(), no environment, so two exports of one graph are byte-identical.

What Tesserae emits, and where each value honestly comes from:

index.md follows §8 (frontmatter is exactly okf_version: '0.2', the one place §12 permits it) and log.md follows §9 (no frontmatter, ## YYYY-MM-DD groups, newest first). On the Tesserae project graph (5197 nodes / 15284 edges) that is 5195 files carrying generated on all 5193 concepts, sources on 3934, usage_window on 1264, description on 1749, resource on 822, and status/stale_after on 25.

Deliberately not emitted. No verified key (§5.2), and therefore no trust tier above unverified (§5.3): nothing in the compiled graph is a recorded verification event with an actor and a timestamp. verify_claim and re-grounding are query-time functions over the graph, and lint --verify-claims is an LLM judge, which MD4 itself says is not evidence. Edge provenance classes describe how strongly the graph licenses a triple; OKF's trust family is a per-concept confirmation. Mapping one onto the other would put a machine-confirmed tier on content nothing confirmed, so generated.by can never begin with human: — a test pins it. Likewise no Attested Computation family (§10): Tesserae has no sanctioned computations, executor, receipt, or attester ABI, and §10.5 tells a consumer to gate on attestation, so empty scaffolding would advertise a contract that cannot be honoured. Also absent for want of an honest source: tags (no node-level tag field — aliases are alternate names, not categories), per-claim [^id] footnotes, status: draft (metadata.confidence is extraction confidence, not review state), and any stored credibility score (§5.1 records signals, not verdicts). last_modified comes from in-graph document dates, never file mtime — the obvious-looking os.stat() shortcut is exactly the environment leak that has broken byte-idempotence here before.

Reading. Per §11 the importer rejects nothing: unknown type values, unknown frontmatter keys, missing optional families, broken cross-links, and a missing index.md are all tolerated; only a file with no non-empty type is skipped. Tesserae's own bundles round-trip losslessly through x_tesserae. Foreign bundles map type → the matching node kind or Concept, body links → references edges, and every unrecognised frontmatter key into metadata.okf (§4.1's round-trip SHOULD), with a bare verified mapping normalised to a one-element list (§11 MUST). v0.1 fallbacks (§13.1): a legacy timestamp lands in metadata["updated_at"] (a rung the timestamp ladder already reads) and a legacy body # Citations list becomes metadata["okf"]["sources"], stripped out of the description instead of being swallowed as prose. On re-export the preserved bucket is merged over anything Tesserae derived, so re-exporting someone else's bundle never overwrites their provenance or trust claims with ours; --import prints a trust-tier histogram so a mixed bundle is visible rather than silent. Tiers are inferred at read time by okf_trust_tier, never stored.

Breaking changes from Tesserae's v0.1 output. name: becomes title: (name was never an OKF key in either version; the reader still accepts it, behind title). index.md and log.md lose their type: / name: frontmatter (§8, §9), so a consumer that treated them as typed concepts loses two phantom entries — which is the point; relatedly they are now reserved at any level of the hierarchy (§3.1), not only the bundle root. Every concept file's bytes change, so the first v0.2 export rewrites the whole bundle.

Known limits. usage_count counts distinct agent/work sessions whose transcript touched the document, not human page reads — §5.1 already warns the signal is coarse; read it as liveness, not popularity. The lifecycle family fires only for nodes targeted by a supersedes edge (25 of 5197 here); real coverage would need the temporal validity intervals TemporalFactProjector derives at query time, and running that inside the exporter over 15k edges was rejected as scope. generated.by uses process:tesserae-<extractor> rather than §7's <producer>/<version> on purpose: a version-bearing actor would rewrite all ~5200 concept files on every release for no semantic change. No path-valued OKF field (resource, sources[].resource) ever carries an absolute path — one that cannot be made project-root-relative is omitted rather than emitted raw, since §6.2 would have a consumer read it as bundle-relative — though absolute paths can still appear inside x_tesserae.source_path (the node's real identity, which a foreign consumer ignores) and inside node content that happens to quote one.

Idempotence story

The redesign aims for byte-identical output across two consecutive project compile runs over unchanged inputs. The pieces:

1. Source extraction uses manifest.json content hashes; unchanged files are skipped, so the graph remains stable.
2. Wiki layer writes are idempotent at the body level. WikiPageStore.write_page reads the existing file, strips frontmatter, sha256s the body, and short-circuits if the new body hashes the same — even if the new frontmatter has a different generated_at timestamp. This is the key trick that keeps git diffs tight on rebuild.
3. Synthesis output carries a content_hash: sha256-… in its frontmatter. The body hash is computed without generated_at so repeated compiles on the same graph produce the same hash, and Synthesis nodes carry the same content_hash in graph metadata.
4. Site rendering wipes site/ at the start of write_site, then writes deterministically: routes are sorted, dictionaries dumped with sort_keys=True, manifest.json walked via sorted(rglob("*")). Two runs produce byte-identical files including the manifest.

This is verified by tests/test_site_pages.py and the end-to-end smoke in tests/test_project_e2e_redesign.py (compile twice, diff sites, expect zero file deltas).

Scaling notes

• Graph view node cap. MD0 bounds the page-embedded payload for the interactive force layout. Beyond ~1500 nodes the browser-side simulation gets sluggish on mid-range hardware, so the page drops the lowest-degree wiki-layer nodes first when the count exceeds the cap. The exported graph.json is unaffected — it always contains the full graph. Code nodes are filtered out before the cap is applied.
• llms-full.txt cap. A 5 MB safety cap applies in MD1; the file ends with a [TRUNCATED — see graph.jsonld for the full set] marker if the cap is hit. graph.jsonld is uncapped because JSON-LD consumers expect the full set.
• Search index. Wiki-layer kinds only. Code-graph nodes never enter search-index.json; the redesign target is < 500 KB for the dogfood corpus and we're well under that today.
• Per-page byte budget (rule of thumb). Each detail page < 60 KB gz HTML, shared CSS < 30 KB, shared JS < 25 KB, sigma vendor on the graph page only (~60 KB). The graph view uses 3D-force-graph + Three.js loaded once; all other pages stay vanilla.
• Compile time on dogfood. ~300 markdown files extract in under 5 s on a recent dev machine; site render adds another ~2 s. The wiki layer's idempotence means subsequent compiles touch only the changed paths.

Frontend interaction surface

• Search palette — cmd+k / ctrl+k / /. Fuzzy match over search-index.json, scoped to wiki kinds. Recent pages persisted in localStorage.
• Theme toggle — top-right button; data-theme="dark" is stored in localStorage and applied before paint to avoid flash.
• Sticky right TOC — desktop only; collapses to a <details> drawer on mobile. Generated from <h2> / <h3> in the page body.
• Activity heatmap — 26-week SVG with month + weekday labels. Cells link to the day's digest.md source page when one exists. (Per-day timeline detail pages — /timeline/<YYYY-MM-DD>.html — are an explicit follow-up; the inline notice in render_timeline flags it. ⚠ in-progress.)
• Graph view — /graph/. 3D force layout (3d-force-graph + Three.js) with hover tooltips, edge labels, cursor-anchored zoom, and a 2D fallback view. Node colors come from ResearchNodeType.
• Mobile shell — drawer rail, bottom nav, fluid type, touch-safe hit targets (≥ 44 px).

Testing strategy

• Unit — tests/test_wiki_store.py, tests/test_synthesis.py, tests/test_site_components.py, tests/test_site_pages.py, tests/test_site_exports.py, tests/test_relevance.py.
• Engine spine — tests/test_pipeline.py, tests/test_refresh_pipeline.py, tests/test_daemon_core.py, tests/test_daemon_sources.py, tests/test_cli_engine.py.
• Self-improvement memory — tests/test_memory_sidecar.py, tests/test_decay_supersede.py, tests/test_supersede_suppression.py, tests/test_mcp_supersede_suppression.py, tests/test_memory_contradiction_reinforce.py.
• Retrieval + embeddings — tests/test_hybrid_search.py, tests/test_ppr.py, tests/test_real_embeddings_phase6.py.
• Context compiler — tests/test_context_compiler.py (shape, citation integrity, determinism, budget, PPR fallback), tests/test_cli_context.py, tests/test_mcp_server_context.py.
• Incremental compile (experimental) — tests/test_incremental_compile.py, tests/test_incremental_parity.py, tests/test_provenance_readiness.py, tests/test_sqlite_provenance.py.
• Idempotence — tests/test_project_e2e_redesign.py compiles twice and asserts zero diffs in wiki/ and site/.
• Link integrity — tests/test_frontend.py parses every emitted HTML for hrefs and asserts every internal link resolves to a generated file. No nodes/codeclass-*.html is produced.
• AI siblings — for every path/foo.html, the test suite asserts path/foo.txt and path/foo.json exist; the JSON parses and contains {title, kind, body, links}.
• No Playwright — vanilla pytest under PYTEST_DISABLE_PLUGIN_AUTOLOAD=1.

Related docs

• Quickstart — minimum path from project init to a browsable site.
• Frontend redesign walkthrough — annotated tour of every route.
• Feature map — what's shipped, what's in-progress, with file pointers.
• Self-dogfood demo — running Tesserae against its own repo.