Architecture

SideshowDB treats Git as the canonical event store and treats every other surface — local indexes, document projections, browser views — as a disposable derived view over Git history.

Layers

flowchart TD
  git["Git Repository<br/>canonical truth<br/>refs/sideshowdb/..."]
  local["Local Materialization<br/>indexes / projections / caches"]
  read["Read Surfaces<br/>CLI / WASM / playground"]

  git -->|"pull / merge / rebase"| local
  local -->|"derived"| read

  git@{ shape: rect }
  local@{ shape: rect }
  read@{ shape: rect }

Core Invariants

These are non-negotiable. Breaking any of them invalidates the design.

1. Events are append-only.
2. Events are immutable.
3. History is reconstructible by replay.
4. Git stores canonical truth.
5. Local databases are disposable.
6. Merges happen on events, not state.
7. Projections never write back to truth.

A more detailed treatment lives in docs/development/specs/sideshowdb-spec.md .

Design rationale (ADRs, RFCs, and vocabulary) is indexed from the design hub and summarized on the Design hub docs page.

Storage Boundaries

Every section of state lives under its own ref:

refs/sideshowdb/<section-name>

Examples in current and planned use:

refs/sideshowdb/documents      # current document slice (CLI doc put/get)
refs/sideshowdb/events         # planned event log
refs/sideshowdb/projections.*  # planned derived projections

This namespace owns its tree exclusively, so SideshowDB data cannot collide with the user's refs/heads/*, tags, or remotes.

The RefStore Interface

storage.RefStore is a small vtable-style "interface" struct (the same shape as std.mem.Allocator or std.Io.Writer). It exposes four operations on a section-scoped key/value store:

• put(gpa, key, value) -> VersionId
• get(gpa, key, version?) -> ?ReadResult
• delete(key)
• list(gpa) -> [][]u8
• history(gpa, key) -> []VersionId

Two concrete implementations live in the codebase:

• storage.GitRefStore shells out to the user's git binary and produces real commits per write. It is gated to non-freestanding targets.
• The wasm32-freestanding build resolves storage.GitRefStore to void so the browser surface can compile without subprocesses.

New implementations should pass the contract tests in tests/git_ref_store_test.zig to be considered conforming.

Write-Through Composite

storage.WriteThroughRefStore is a RefStore that fronts a canonical RefStore with one or more cache RefStores. Every operation is exposed under the same vtable contract, so the composite is itself just another RefStore to the caller. Every successful put / delete blocks until canonical accepts — there is no asynchronous queue today.

flowchart TD
  operation["put / delete / get"]
  composite["WriteThroughRefStore"]
  caches["cache RefStores"]
  canonical["canonical RefStore<br/>truth in Git ref"]

  operation --> composite
  composite -->|"reads try first"| caches
  caches -->|"miss"| canonical
  composite -->|"writes stage then commit"| canonical
  canonical -->|"read hit refills"| caches

The full contract — write order, read fall-through, refill, recovery, and the EARS-tagged failure semantics — lives in docs/development/specs/write-through-store-spec.md . The deliberation that produced this primitive (rather than a "real" write-behind cache with a durable WAL) is recorded in docs/development/decisions/2026-04-29-caching-model.md .

Two practical reasons for this layer:

1. Speed. A local cache can answer reads without round-tripping the canonical Git engine. With multiple caches in the chain (e.g. an in-memory hot cache in front of a LevelDB warm cache), the cheapest tier serves the common path.
2. Backend swap. Cache backends — LevelDB, RocksDB, IndexedDB — plug into the same composite without changing the canonical layer. A future native deployment can mix-and-match without re-deriving the read/write semantics.

flowchart TD
  get["get(key)"]
  cache0["cache 0"]
  cache1["cache 1"]
  more["more caches"]
  canonical["canonical"]
  cacheHit["return value<br/>cache version-id"]
  refill["refill caches<br/>return canonical version-id"]
  missing["return null"]

  get --> cache0
  cache0 -->|"hit"| cacheHit
  cache0 -->|"miss"| cache1
  cache1 -->|"miss"| more
  more -->|"miss"| canonical
  canonical -->|"hit"| refill
  canonical -->|"miss"| missing

flowchart TD
  put["put(key, value)"]
  cache0["cache 0 stage"]
  cache1["cache 1 stage"]
  more["more caches stage"]
  canonical["canonical commit"]
  version["return canonical version-id"]

  put --> cache0 --> cache1 --> more --> canonical
  canonical --> version

The composite degenerates cleanly:

• Zero caches → thin pass-through to canonical.
• One cache → traditional read-through / write-through cache. Expected to be the common steady-state shape.
• N caches → fan-out structure that becomes useful for benchmarking and tiered-cache experimentation once on-disk cache backends and an instrumentation hook land. Today every cache is in-memory; the multi-cache topology mostly exercises the composite's failure semantics.

WriteThroughRefStore is not thread-safe; callers needing concurrent access must serialize externally. Same posture as every other RefStore implementation in the codebase.

Sibling caching primitives are filed for separate design and shipping when the use cases land:

• WAL + batched canonical flush — the genuine "write-behind" pattern; layers over write-through.
• Write-around — writes bypass cache entirely, cache populated only on read fall-through. Useful when single-cache deployments want to skip the speculative-cache window.
• Offline writes — caller-visible success while canonical is unreachable, with durable buffering and reconnect-flush. The feature SideshowDB's local-first posture ultimately needs.

DocumentStore on Top of RefStore

document.DocumentStore is the first end-to-end slice. Documents are addressed by an Identity of (namespace, doc_type, id) and stored as JSON envelopes that include identity plus a data payload. The canonical key is computed by document.deriveKey as <namespace>/<doc_type>/<id>.json.

Errors surface as document.Error variants (ConflictingIdentity, InvalidDocument, InvalidIdentity, MissingIdentity, VersionIsOutputOnly).

Transport adapters live in document_transport for JSON wire-format usage by CLI and WASM bridges.

Local-First Operation

There is no always-on server. Every SideshowDB consumer:

• Reads canonical state from a Git working copy or git fetch.
• Writes canonical state by producing commits on refs/sideshowdb/<section>.
• Builds derived state (indexes, projections) on demand and may delete it at any time.

Pulling, branching, merging, and rebasing are normal Git operations because the store layout is a normal Git tree under a SideshowDB-owned ref.

Browser Constraints

The WASM client builds against wasm32-freestanding, so storage.GitRefStore is unavailable in the browser. Browser playground code therefore:

• Fetches public GitHub data with the Fetch API.
• Calls into the loaded WASM module for projection logic.
• Treats the result as an explanatory derived view, not a writable database.

See the Projection Walkthrough for the end-to-end mapping from a real public repo into SideshowDB concepts.

Where to Look in the Reference

• sideshow — top-level module
• sideshowdb.storage
• sideshowdb.document
• sideshowdb.document_transport
• sideshowdb.event