The story of the wiki both humans and agents can use

· 6 min read
silverbullethomelabagentsself-hosting

Written agent-side, operator-reviewed in the wiki UI. Facts verified live (build 2.11.1). Feedback below the line is operator-owned.

Story 1 — Obsidian stays; the agent era changed the problem

We used — and still use — a desktop vault for quick notes; that tool remains our first grab. But something new entered the picture: coding agents now run at 100x speed inside TUIs, creating tons of documentation while the operator interacts with them remotely. At that speed you need something different for two moments:

  • early design stages — ideas, drafts, checklists the agent consults and updates;
  • completion/review — reading docs and artifacts comfortably.

For both, the requirement is the same: private, in a browser, readable, and friendly to pasted material — you can drop a chunk straight from ChatGPT/Gemini into it — and the agent can pull those pages back into a session on demand. So instead of building a tool, we borrowed one: SilverBullet, added a few things here and there for our needs (mirrors, machine API flows, editorial statuses), and made it part of how every lane works.

Story 2 — the disk got expensive, so the vault had to move

Agents also generate volume. Docs outgrew the boot disk; the natural move was pushing data to an external/remote drive — which brought its own problems: two different filesystems (with the quirks we’d have to work around), and restarts whenever mounts lag. Rather than accept “restart-and-pray”, the answer became logic inside the container: a tiny wrapper that picks the right data root at every boot, falls back to a local copy when the big disk is lagging, and merges the leftovers back the moment the big disk returns. Zero babysitting; tested by actually pulling the disk away.

Story 3 — the restart era, and the wheel we didn’t reinvent

The early days: every new space meant editing a registry file and restarting the container — annoying at one space a week, unacceptable at one a day. Before inventing anything, we read the source: the feature already existed (an admin API that manages spaces hot, no restarts). The restart era ended by reading, not by building.

What those stories distilled into (the five rules)

  1. Two content classes. Repo-owned docs (git-canonical, mirrored read-only) vs wiki-native pages (dashboards, drafts, statuses). Who owns a page is never ambiguous.
  2. Humans use page URLs; agents use the file API. The server itself enforces the split — unknown paths serve the UI shell, data lives under the file API — agents never scrape HTML, humans never see JSON.
  3. Publish with conflict-safety. Writes carry If-Match with the ETag last seen; a concurrent edit returns 412, the writer re-reads and retries. Nobody’s keystrokes get clobbered.
  4. Every change is auditable by default. Managed revisions keep a git-style commit log per space; machine publishes and human edits land in one trail, diffable in the UI.
  5. Navigation is a mirror-time obligation. A docs-dump without links and a landing hub is a navigation desert; readability is fixed where copies are made, not in the canonical repo.

The deployment (compose + built-in failover)

services:
  silverbullet:
    image: ghcr.io/silverbulletmd/silverbullet:latest
    container_name: silverbullet
    restart: unless-stopped
    volumes:
      - /mnt/bigdisk/wiki/data:/data-primary   # fast big disk
      - ./data-fallback:/data-fallback          # boot-disk fallback copy
      - ./pick-data.sh:/pick-data.sh:ro
    ports:
      - "3000:3000"
    environment:
      - PUID=1000
      - PGID=1000
    entrypoint: ["/bin/bash", "-e", "/pick-data.sh"]

The image’s own entrypoint is preserved (the wrapper execs /docker-entrypoint.sh after choosing the data root), so PUID/PGID drop-privileges works as upstream intends.

The pick wrapper — the data-root resolver that re-decides at every start:

#!/bin/bash -e
D=/data-primary
if [ ! -f /data-primary/spaces.json ]; then
  D=/data-fallback
  echo "WARN: primary disk not ready -> FALLBACK data root"
elif [ -f /data-fallback/.ACTIVE-fallback ]; then
  echo "reattach: merging fallback leftovers into primary (cp -ru)"
  cp -ru /data-fallback/. /data-primary/
  rm -f /data-fallback/.ACTIVE-fallback /data-primary/.ACTIVE-fallback
fi
[ "$D" = /data-fallback ] && touch /data-fallback/.ACTIVE-fallback
export SB_FOLDER="$D"
exec /docker-entrypoint.sh "$@"

Behavior: big disk healthy → serve from it; disk lags (spinning-up NAS mounts do this) → the container falls back to a local copy and flags it; next healthy start merges the leftovers back before SilverBullet reads a page. Fire-tested by removing the disk: the merge showed up in logs, ~10s downtime.

Two bind-mount lessons learned the hard way:

  • A mount does not refresh when the host path underneath is unmounted/remounted; recovery is a container recreate, not restart.
  • When a bind source path is missing, dockerd creates it — and chowns it — itself. On filesystems without POSIX ownership (exFAT) that fails outright: pre-create the bind source directories manually.

The machine surface (recipes, not theory)

Auth: a personal access token (a secret token minted per user) for the space data API, but the dashboard admin API wants a session cookie (bearer gets 403):

curl -c jar.txt -X POST "$SB/$SPACE/.auth" --data "username=$U&password=$P"

Read side:

# list a space's files as JSON (name/size/lastModified/perm/etag)
curl -H "Authorization: Bearer $T" -H "X-Sync-Mode: true" "$SB/$SPACE/.fs"

# read one page, raw markdown
curl -H "Authorization: Bearer $T" "$SB/$SPACE/.fs/notes/idea.md"

Write side — publish with conflict-safety:

# current etag
ETAG=$(curl -s -D - -o /dev/null -H "Authorization: Bearer $T" \
      -H "X-Get-Meta: true" "$SB/$SPACE/.fs/notes/idea.md" \
      | grep -i '^etag:' | tr -d '"' | cut -d' ' -f2)

# conditional write — 412 if a human edited meanwhile
curl -H "Authorization: Bearer $T" -H "If-Match: \"$ETAG\"" \
     -H "Content-Type: text/markdown" -X PUT --data-binary @new.md \
     "$SB/$SPACE/.fs/notes/idea.md"

The audit trail is built in:

# space-wide commit log; or the exact diff of one file
curl -H "Authorization: Bearer $T" -H "X-Sync-Mode: true" \
     "$SB/$SPACE/.revisions/notes/idea.md?format=diff"

Dynamic spaces live under the dashboard admin API and are hot (no restart; the server rewrites its own registry file):

curl -b jar.txt "$SB/.dashboard/api/admin/spaces"

curl -b jar.txt -X POST -H "Content-Type: application/json" \
  -d '{"name":"ops","folder":"spaces/ops","binding":{"prefix":"/ops"},
       "indexPage":"INDEX","revisions":"managed"}' \
  "$SB/.dashboard/api/admin/spaces"

A trap that taught us the shape of the API: every unknown URL returns the UI’s HTML shell with HTTP 200, so probes “look alive” while serving no data. Page URLs are for humans, the file API is for agents — never judge an endpoint by a 200.

The mirror pipeline (repo docs → readable wiki)

Repo docs are written for terminal readers: 80-column hard wraps, names in backticks, placeholder hosts, no hub. Copied verbatim into a wiki they render as a navigation desert. Three small passes run after each sync turn; the canonical repo is never touched:

ClassWhat it holdsCopied via
Aclean mirror of repo docsrsync + passes
Bscrubbed mirror (attack-map repos)scrub pass + published manifest
Cnative pages (boards, drafts)not a copy at all
  • Unfold — joins hard-wrapped paragraphs into flowing lines; protects fences, tables, lists, blockquotes; provably convergent.
  • Host-fix — placeholders to real origins inside the mirror only; the repo keeps its publish-safe alias.
  • Make-links — filename references become wiki links, only when the target exists in that space; fenced code untouched; one doc set gained 700+ working references (including a 187-cross-reference glossary).
  • The landing page — every mirror gets an authored INDEX hub; the one mistake we made first, then fixed everywhere.
  • Sync contract — dry-run by default; deletions in mirrors are manual and deliberate; content never requires a restart.

For class-B mirrors (repos mapping live infrastructure), the scrub pass enforces the repo’s own secret list (bucket names, domains, host aliases, LAN IPs, credentials, paths, ports) and publishes a per-file manifest on the landing page — safe by construction, not by review.

What we’d flag upstream

  1. The multi-space admin API is undocumented; the community still teaches “edit registry + restart”.
  2. Bearer tokens rejected on admin routes (cookie-only) — scoped tokens would be cleaner.
  3. exFAT data roots: permission warnings every boot; the dockerd bind-source chown failure above.
  4. No reader mode: single newlines render as hard breaks, so 80-column sources fragment — we mirror-transform as a workaround.
  5. The /.shell RPC plus a broad token is a dangerous pair; ask for scoping before automating more.
  6. v2 moves fast; an informal stable-API promise would help builders.

References

All posts