obsidian-sync-mechanism

Parent: corpus · Children: protocol · vault-ingestion · rendering-and-extensibility · Status: both directions built — full SyncVault (raw + writing + wiki/subjectivity + .obsidian CSS); reverse export writes corp notes back as genre folder + tree + folder-notes since 2026-07-29 (export_corpus.go, 9f1ffcf13)

Bidirectional sync between StandMeet and an Obsidian vault. Shaped like Quartz / obsidian-importer: two buttons, each a batch job, owner-triggered manually — no file watcher, no live sync.

Endpoints (backend/internal/routes/admin/obsidian.go:152)

• GET /api/admin/obsidian/export → a zip (writings/<slug>.md + attachments/<id>.<ext>, plus wiki/… · subjectivity/… · output/… as genre folder + tree); each note is rendered as frontmatter + body
• POST /api/admin/obsidian/import → multipart upload of the whole vault (browser webkitdirectory, 200 MB cap); each file carries its vault-relative path
• GET /api/admin/obsidian/state → when the last import ran (obsidian_state.go, 6cf4e2b08, 2026-08-21) — before this the screen had no past tense at all

Sync pipeline (backend/internal/corpus/obsidian/sync.go · SyncVault)

Import is one batch pass over the uploaded vault. classifyVault (sync_classify.go:65) splits files by top-level folder into three buckets — corp, writing, css (hidden paths and _templates filtered, except .obsidian CSS which is harvested). raw/ is no longer its own bucket: since d925d9081 (2026-07-09) it folds into corp as genre='raw', frontmatter-exempt (whole file = body), and goes through the same node-tree materializer. The corp bucket additionally runs the tree → ambiguity check → reconcile → link-resolution → prune stages. This flowchart is the authoritative overview; the prose below details each stage.

Reverse direction (export) is a separate batch — a node with children flattens back to foo/foo.md + children under foo/ (export_corpus.go); writings go out flat as writings/<slug>.md; see writings-import-export.

Writing-branch import detail (backend/internal/corpus/obsidian/import.go)

1. Partition .md vs non-.md; attachments are indexed by basename (matches how Obsidian resolves [[image.png]] by basename regardless of subfolder)
2. For each .md:
   • Parse frontmatter; publish != true → skip outright (import.go:146 — the writings branch is still ingest-gated; the corp branch is not, see the invariants below)
   • Image refs in the body → look up bytes in the attachment index → mint pending-<uuid>, rewrite the body ref to standmeet-asset:pending-<uuid>
   • Find an existing writing by obsidian_source_path, else by slug (upsertFromVault, import.go:252 / :279) → update; none → create. There is no web-wins guard any more (84080bac5, 2026-07-15, F-L-6): the vault is the single live source; a web edit is kept only by exporting it back before the next sync
   • SetObsidianMeta stamps imported_at
3. Return { created, updated, skipped, errors } (writings never delete)

Folder notes (convention A) — must be specially recognized

The vault represents a node that has children as a folder foo/ whose own content lives in foo/foo.md (the folder note; enforced by a pre-commit hook). The sync must special-case this so the folder note maps to the node itself, not to a duplicate child:

• Rule: if basename(file, ".md") == basename(dirname(file)), the file is a folder note → its node path = the directory path (a/b/foo), not the naive filesystem derivation a/b/foo/foo.
• So market/market.md ⇒ the market node (path …/market); a sibling market/two-scarcities-arbitrage.md ⇒ a child of market (parent_id → market). One node per folder, no duplicate foo/foo slug.
• Inverse on export: a node with children is written as foo/foo.md + its children under foo/.

Implemented for the corp path

buildDesiredTree (sync_tree.go) collapses folder notes for the wiki/subjectivity sync — market/market.md becomes the market node, siblings become its children (parent_id), no duplicate foo/foo. Still open: the writings branch derives slug as plain basename(path) (pickSlug, see protocol) and does not collapse folder notes yet.

Key invariants

• publish is a visibility gate, not an ingest gate (F-L-8, 84080bac5, 2026-07-15; sync.go:13): every routed corp .md persists regardless; publish only feeds the DB's published = visible to anonymous visitors / in the sitemap; published=false means "needs a code", not "doesn't exist". An absent publish key keeps the current value — silence is not a no (F-L-22, 8bb2e1c00, 2026-08-08; keepPublish, sync.go:229). The writings branch alone still skips publish != true outright
• The vault is the single live source (F-L-6, replaces "web edits win"): sync makes destination equal source, no timestamp comparison; an authoritative (whole-vault) sync prunes vault-imported notes the vault no longer has (sync_prune.go), a partial upload never deletes
• Body [[slug]]/[[Title]] → edge tables: link relationships come from the body, not frontmatter — corp notes write note_refs (resolved by title and frontmatter aliases), writings write writing_refs
• Which fields come from the template vs. from elsewhere → see protocol