DreamDB Specification — 0008: Versioning & Collaboration

Status: Draft. Builds on 0000–0007. This document defines the versioning model: Manifest DAG history, branching, merging, multi-writer reconciliation, and conflict-resolution semantics. Resolves OQ-5 (manifest distribution) and OQ-35 (explicit parent for Publish).

1. Purpose

0001 introduced Manifests as immutable, content-addressed snapshots of a Space. 0002 defined the parent-pointer field on Manifests. 0006 defined Publish as the verb that mints a new Manifest. This document defines what those parent-pointers form: a history DAG that supports branching, merging, and multi-writer reconciliation in a Git-like model — with a few critical differences imposed by DreamDB's architecture.

By the end of this document, the following are concrete:

• The shape of the Manifest DAG: linear chains, branches, merges.
• The branching workflow — how a writer forks from an old Manifest without a Ref advance.
• The merge workflow — how two divergent histories are reconciled.
• Multi-writer reconciliation — the lock-free pattern when two writers race on a Ref.
• Conflict-resolution rules by Track Kind and modality.
• Manifest distribution beyond Refs (resolves OQ-5) — pull-only spaces, manifest exchange, federation hints.
• Pruning and history compaction — how to bound DAG growth without violating immutability.

What this document does not define:

• The byte format of Manifests — 0002 §7.2.
• The verbs that operate on the DAG (Open, Publish) — 0006.
• Backend push Subscribe and durable change streams — out of scope per 0006 §9. Optional pull-based Ref Watch is defined in 0006 §4.1.2.

2. The Manifest DAG

A DreamDB Space's history is a directed acyclic graph of Manifests, with edges given by the parents field.

                     (root Manifest M_0)
                            │
                       parent
                            │
                           M_1
                            │
                           M_2 ←──────┐
                          /  \         │
                         M_3  M_4      │
                              │        │
                              M_5      │
                              │        │
                            (merge: parents = {M_3, M_5})
                              │
                             M_6 ──── refs/main

Properties:

• Immutable. Once published, a Manifest's bytes (and therefore its hash, parent pointer, and tracks) never change.
• Single root per Space. The first Manifest of a Space has parents: [] (empty array). Subsequent Manifests have at least one parent.
• Acyclic. Cycles are impossible by content-addressing — a Manifest's hash depends on its parent's hash, so a cycle would require a hash to reference itself transitively (cryptographically impossible).
• DAG, not tree. A Manifest MAY have multiple parents (a merge commit). The parents field is an array, not a single hash.
• Refs point at "tips." A Ref names a single Manifest; readers resolve the Ref to find the current "live" point in the DAG.

2.1 Manifest schema (recap)

Per 0002 §7.2, a Manifest's CBOR has a parents field — an array of multihashes:

{
   "parents":   [<multihash-of-prev-Manifest>, ...] | [],   ;; empty array for root
   "timelines": [...],
   "tracks":    ...,
   "ts":        ...,
   "writer":    ...,
   "registry":  ...,
}

Cardinality of parents:

• Empty array [] — the root Manifest of a Space (no ancestors), including a Manifest produced by re-ingesting data from an older Manifest form, which is a root and MUST NOT name its predecessor.
• One-element array [h] — linear advance (the common case).
• Multi-element array [h_a, h_b, ...] — merge Manifest combining work from divergent ancestors (§5).

Order is significant and MUST NOT be sorted (0002 §7.2): for a merge, parents[0] is the trunk and parents[1] is the branch, as declared by the merging writer. Because a Manifest is content-addressed, the order changes the Manifest's hash, so implementations that do not agree on it will mint different addresses for the same merge.

Arity, and what is defined for it. The ordering rule above defines parents[0] and parents[1] only. Positions parents[2..] have no defined order and no defined three-way resolution, so no writer may rely on them:

• Readers and the schema retain the general array. Historical Manifests with more than two parents exist, MUST remain decodable, and MUST be traversable for reachability and history walks. Nothing here narrows what a reader accepts.
• A Lineage-v1 writer MUST emit 0, 1 or 2 parents. Zero for a root, one for a linear advance, exactly two for a merge.
• A merge MUST have exactly two parents, [trunk, branch].
• An N-way merge MUST be expressed as successive two-parent merges, each with its own baseline (§5.4), until a future revision defines N-way ordering and resolution. This is a restriction on writing, not on reading: it exists because a three-way rule stated for two sides does not generalise to N by itself, and guessing at the generalisation is how two implementations come to disagree.

0002 §7.2 is the canonical schema definition; this section restates the field for the DAG semantics that follow.

2.2 Why parents-as-array (not separate fields)

Alternative designs considered:

• Singular parent only. Forecloses merges; forces all collaboration through linear advance. Loses the Git-like fork/merge workflow that distributed teams need.
• Separate merge_parents field. Two fields = ambiguity about which one's traversed during history walks. One field with a list is simpler.
• Time-ordered single chain (no branches). Equivalent to "parent only," with the same constraint.

The DAG model with parents: [...] matches Git's CommitObject model, IPLD-DAG conventions, and most distributed-version-control systems. DreamDB's content-addressing makes the DAG operationally identical to Git's — a DAG of immutable, hash-named, parent-referencing nodes.

3. Linear Advance (the Single-Writer Case)

The simplest history: a single writer (or non-conflicting concurrent writers) advances the Ref through a linear chain of Manifests.

M_0 ← M_1 ← M_2 ← M_3 ← refs/main

Every Publish produces a new Manifest with parents: [<previous-tip>]. Refs/main advances by CAS (0005 §4.2). The DAG is a single chain; no merge logic is needed.

This is the dominant pattern for: single-writer pipelines, sequentially-coordinated multi-writer pipelines (where writers explicitly serialize), and workflows where contention is rare enough to round-trip through CAS retries.

For multi-writer Spaces with sustained concurrency, linear-advance breaks down — see §3.1.

3.1 Feature-branching: the recommended workflow for multi-writer Spaces

In environments with N concurrent writers (N > ~10), competition for the refs/main CAS produces a flood of 412 errors that stalls the pipeline:

• N writers attempt to advance refs/main simultaneously.
• One wins; N-1 receive 412.
• The N-1 losers retry; in the meantime, more new writers join the queue.
• Throughput collapses to ~1 successful Publish per round-trip, regardless of how many writers exist.

The recommended workflow: feature-branching. Writers Publish to per-writer or per-team Refs during normal operation, and a separate process (manual or automated) merges branches into refs/main periodically.

refs/users/alice/scratch     ← Alice's writes (no contention with Bob)
refs/users/bob/scratch       ← Bob's writes
refs/teams/eng/main          ← team-level integration branch
refs/main                    ← canonical timeline (merged periodically)

This shifts contention from the per-write CAS phase (every Append) to the merge phase (one CAS per logical batch of branch integrations). The Phase-1 Object writes (per 0006 §5.5) remain fully parallel and lock-free across all writers.

Conventional Ref namespaces (SHOULD, not MUST — operators MAY use other names):

Spec posture: feature-branching is a SHOULD for multi-writer Spaces; the protocol does not enforce it (writers may target any Ref name they have permission for). 0009's load-test conformance scenarios SHOULD include both patterns: 1000 concurrent writers on per-user Refs (the recommended workflow) is expected to sustain ≥10× the throughput of 1000 concurrent writers on refs/main directly.

The merge step from per-writer Refs into refs/main follows the standard merge workflow (§5). A periodic "integration coordinator" — a single writer or automated process — pulls each branch's tip, runs §6's conflict resolution, and Publishes the merge to refs/main. This concentrates CAS contention to one writer at a time on refs/main, eliminating the 412 storm.

4. Branching

A branch is a Manifest published with parent = some non-tip ancestor of refs/main.

M_0 ← M_1 ← M_2 ← M_3 ← refs/main
                 ↖
                  M_3' ← refs/feature   (M_3' has parent = M_2)

Branches arise naturally in two scenarios:

4.1 Concurrent writer who lost a CAS race

Per 0006 §6.3, a writer who loses the Ref CAS (412 Precondition Failed) typically rebuilds with the new winner as parent. But the writer MAY instead publish to a different Ref:

PUT refs/<my-feature-name>     If-None-Match: *

The losing writer's Manifest still references its (now-stale) parent. This is a branch.

4.2 Explicit fork

A writer who wants to experiment without affecting refs/main calls Publish with an explicit parent argument (resolves OQ-35):

# Conceptual API surface
session = Open(backend_url, ref="main")          # current tip = M_3
publish = Publish(session, manifest_spec={
   parents: [hash_of_M_2],                       # explicit fork: parent = old M_2, not the tip
   tracks:  [...],
   ref:     "feature-branch",                    # different Ref
})

The explicit-parent argument resolves OQ-35: Publish accepts an optional parents: [...] override. If absent, the parent defaults to the Session's loaded Manifest tip. If present, the writer is explicitly declaring what they're forking from.

This is the operational mechanism for "experimental tracks," "alternative analyses on the same source data," "snapshots before a destructive transformation," etc. — all distributed-version-control patterns familiar from Git.

4.3 Refs vs. branches

A "branch" in DreamDB is structurally just a Ref pointing at a Manifest whose parent isn't the current tip of another Ref. Refs are the named handles; branches are the structural pattern they create. refs/main, refs/feature/auth-rewrite, refs/experiments/larger-context-window — each is a Ref; each may point at a different DAG tip.

4.4 Immutable named snapshot tags (OQ-41)

A tag binds one backend-global name permanently to one Dataset Manifest. Its address is refs/tags/<name>, where the entire suffix after refs/ obeys 0002 §10's name grammar and 256-byte bound. The tag name is not automatically prefixed by a Dataset name; callers MAY use dataset/release to organize it. A tag body is canonical CBOR with exactly these two keys:

{
  "kind": "dreamdb.snapshot-tag.v1",
  "manifest": <33-byte multihash bstr>
}

This CLOSED record is a named root, not a content-addressed Object and not an ordinary 33-byte Ref. Unknown kinds, keys, missing fields, invalid hashes and noncanonical encodings MUST be refused. The target is a Dataset Manifest, not a Federation Manifest. No timestamp, tag metadata, update or delete operation is defined by this version.

Tag publication MUST use If-None-Match: *, never If-Match or unconditional replacement. The target Manifest must already be durable and hash-verified. Referenced pending writes must be flushed before publishing the root; root durability must be established before returning success. An existing canonical tag naming the identical Manifest is idempotent success, not a conflict. A different target is a conflict; a malformed occupant is corruption. Transport failure can leave the tag published: it is not proof of absence. Retry by name and compare the exact target; a 412 alone is not proof of identity (0005 §4).

The reference Rust API is Dataset::create_tag(name) for the handle's current published Manifest (not staged samples), and Dataset::open_tag(name, connector) to resolve once and open that exact hash as a detached Dataset. Branch this handle under an ordinary Ref to write elsewhere. Changing the source branch does not change old or newly opened tag readers. These operations verify the Manifest, not the entire object closure; they require the same external source retention and GC coordination as ordinary Ref creation. No Python/JavaScript tag binding is claimed by this initial API.

Ordinary Ref publication, creation, branching, snapshot-label creation and deletion MUST refuse tags and tags/ names, whether or not such a root exists. In particular, deletion must use the namespace rule rather than a read-then-delete check of a mutable body. The shared protocol Ref mover applies this rule even to a manually bound Session. Dataset::snapshot and old snapshot labels remain ordinary, advanceable Refs; they are not retroactively promoted to tags.

Collectors MUST enumerate refs/tags/ along with other refs/ roots, decode tag bodies and mark their exact Manifest closures using the normal retained-root policy. A tag retains its snapshot independently of source branch movement or removal; it does not promise unlimited ancestor history. Tag roots themselves MUST NOT be swept. An unreadable/unsupported tag aborts the run before sweep.

Compatibility and rollout: the tags/ subspace is newly reserved, not a reinterpretation of historical bytes. Existing 33-byte Refs there remain ordinary read-only legacy roots and are retained by GC; tag open refuses them. Before further ordinary mutation, operators must relocate them outside that subspace. Inventory it and upgrade/fence all generic mutators before creating tags. Older 33-byte-only collectors see a new tag as a malformed Ref and must refuse sweep, not skip it; current collectors understand it. Upgrading the collector is therefore required for GC availability after tag creation. The single-root benchmark collector does not implement multi-root retention and explicitly refuses a backend containing refs/tags/ entries.

Immutability is a protocol/API guarantee. Old generic deletion clients, direct Connector writes/deletes and privileged backend operators can bypass it; deployment must exclude them or enforce equivalent backend policy. This feature provides neither WORM storage nor an authentication scheme, and the name convention alone is not its enforcement mechanism.

5. Merging

A merge is a Manifest with two or more parents — typically combining work from two divergent branches.

   M_2 ←─┐
         │
         M_3 ← refs/main         ┐
              ↖                  │
                M_3' ← refs/feature
                                  │ ← (merge step)
                                  ▼
                                  M_4 (parents = [M_3, M_3'])
                                  │
                                  refs/main (advanced via CAS)

Publish accepts a multi-element parents: [...] array. The merging writer:

1. Resolves both parents (fetches and validates each Manifest).
2. Computes the merged Track set by three-way resolution against a common baseline (§5.4), with conflict resolution (§6). A plain union of both parents' Tracks is NOT correct: it resurrects bindings the other branch deleted.
3. Computes the merged registry by registry-specific reconciliation — NOT a blind union. Concatenating two registries can mint duplicate bindings for one modality, which a strict publish-time validator refuses and which, if published, makes a lookup return the first match and silently answer with the wrong index's results. Until domain reconciliation is specified, the rule is: the two registries MUST be canonically equal as a whole, or the merge is refused. A production merging transform MUST NOT ship before that reconciliation is defined.
4. Builds the merge Manifest with parents: [M_3, M_3'].
5. PUTs the new Manifest, advances the Ref.

5.1 What a merge "is" in DreamDB

DreamDB merges are conceptually different from Git merges:

• Git merges combine textual changes via line-based 3-way merge with manual conflict resolution.
• DreamDB merges combine sets of immutable, content-addressed Tracks via deterministic union and rule-based conflict resolution (§6).

A merge does NOT merge the bytes of any Object. Tracks from both parents remain referenced; the merge Manifest declares the combined set. Conflict-resolution rules (§6) handle cases where both parents have layered the same logical track (e.g., two corrections to the same title.text).

5.2 Fast-forward merges

If one parent is an ancestor of the other (e.g., M_3 is an ancestor of M_3'), the merge is a fast-forward — the result is just M_3' (no new Manifest needed). The Ref advances directly to M_3'.

The SDK SHOULD detect fast-forward opportunities by walking the parent DAG before constructing a merge Manifest. If detected, no new Manifest is created.

5.3 Layered-merge vs. fused-merge of overlapping Tracks

Format scope. The choice below is an older-format capability. It is NOT available to a Lineage-v1 writer: Lineage-v1 forbids two active bindings sharing one logical binding, so a layered result — two query-visible versions of the same binding — is not a shape it can express. A Lineage-v1 merge of a both-changed binding MUST therefore produce one legal fused binding, or refuse; it never has two options. The read-equivalence contract below is unaffected, because layered and fused are defined to return identical results: Lineage-v1 simply has one legal representation of that result rather than two. See design/0012 "Layered merge — v1 refuses".

When two parents reference Tracks for the SAME (timeline_id, modality) pair (the common case under sharded ingest, where N workers each append to the same embedding modality from their own branch), the merging writer has TWO valid options:

Layered-merge (§6's default): both parents' TrackObjects are kept as separate TrackEntrys in the merge Manifest. Each entry retains the role it already carried — the merge neither synthesises nor rewrites a role, and there is no merge-specific role value. Physical layers are distinguished by full TrackEntry identity: two entries for the same (timeline_id, modality) that differ in any field are both kept. role therefore carries only the meaning 0002 §7.2.1 gives it (base, or layer-of:<Track-address> provenance) and never encodes merge structure. Readers union candidate sets across all layers. Storage: O(parents) Track Objects per modality, accumulating per merge. Query cost: scales with layer count.

Fused-merge: the merging writer produces ONE new TrackObject per modality whose object_index is the cell-by-cell union of the parents' bucket entries. For SpatialBucket tracks, cells touched on both sides require fetching both buckets and writing a new consolidated bucket with the union of records (deduplicated by time_anchor); cells touched on only one side reuse that side's bucket address. The new Manifest references ONE TrackObject per modality. Storage: bounded by total live records (no layer accumulation). Query cost: independent of merge history.

A conformant SDK MAY implement either or both — the choice does not change what a read returns (see the read-equivalence contract below), only the post-merge shape and performance. The reference implementation (dreamdb-dataset crate, MergeStrategy::UnionTracks) ships fused-merge for all inline index kinds — SpatialBucket (embeddings), Fragment (CMAF video/audio), and ScalarBucket (0011 scalars) — taking the content-addressed set-union of each Track's index entries (deduped, re-sorted canonically). Constants are not unioned; they resolve by the §6.3 lexicographically-greatest-address tiebreak. (Cross-Timeline merge remains out of scope — §5.1.)

Read-equivalence contract (normative). Layered-merge and fused-merge MUST produce identical query results for every read over the merged Manifest. They differ only in physical representation — and therefore performance — never in what a read returns. The merged result is defined by the §6 conflict resolution applied to the two parents: set-union per Track Kind (§6.1–§6.4), the §6.3 lexicographically-greatest-address tiebreak for Constants, and a MUST-refuse on SpatialIndex-hash disagreement (§6.5). Equivalently, fused-merge ≡ layered-merge followed by 0021 compaction: fusion is that union physically consolidated, not a separate algorithm with its own semantics. Whether an SDK fuses eagerly, lazily (layers, compacted later), or never (pure layered) is a representation / performance choice (0009 §4.4) and is invisible to readers. The §6 resolution — not a particular strategy — is the conformance contract every implementation MUST honor; it is verified by 0009 §8 dag.merge-equivalence.

Both approaches MUST refuse a merge whose two parents disagree on the modality's SpatialIndex hash (§6.5; conformance: 0009 §8 dag.spatial-index-incompat).

The performance trade-off is operator-policy:

• Layered-merge is cheaper to write (no bucket reconciliation), but query cost scales with accumulated layer count until compaction.
• Fused-merge is what production sharded-ingest wants (N parallel workers consolidating back into trunk; layered-merge would leave N parallel TrackObjects per modality and degrade query latency by N×). It is the layered result with the union physically materialized.

design/0007-sharded-ingest.md documents the fused-merge orchestration recipe — LCA walk, per-cell reconciliation, bucket-conflict bytes-level merge, refusal scenarios. The normative merge result is the §6 resolution defined here (and tested by 0009 §8 dag.merge-equivalence); that recipe is an informative implementation guide, and a future revision may promote it to its own spec number.

5.4 Three-way resolution: add, change and delete

A merge MUST be resolved per logical binding against a common baseline — the most recent common ancestor of the two parents. Writing the baseline as A, the trunk as T and the branch as B:

Why a baseline is required. Without one, "present on exactly one side" is ambiguous between added there and deleted on the other side. Treating it uniformly as a carry — which a plain union does — resurrects every binding either branch deleted, and the deletion is silently undone with no error. This is the single most damaging way to get a merge wrong, because the result is a valid Manifest that quietly contains data a writer removed on purpose.

The two delete/modify rows MUST refuse rather than choose: when one side deleted a binding the other side modified, no automatic outcome preserves both intents, and picking either discards a writer's explicit action.

Equality means the complete declaration, not the content address alone: the binding reference together with its kind, coverage, role, and its provenance edge if it has one. Two parents naming the same address with different declarations MUST be refused — they disagree about what that Track is, and publishing either claim under the other's provenance would corrupt the record while appearing consistent.

Deleting a binding never deletes provenance directly. A deletion removes a binding from the query-visible set; whether its Object remains referenced is then decided by reachability, not by the deletion.

A merging writer can only resolve relative to the baseline it is given, and cannot itself prove that baseline is the true common ancestor, since a Manifest carries no ancestry beyond its immediate parents. Establishing the true common ancestor is the responsibility of the layer that walks the DAG.

6. Conflict Resolution

Format scope. This section defines the logical merge of the retained result. Under an older format that result MAY be represented as layered entries; under Lineage-v1 it MUST NOT, because two active bindings may not share one logical binding. Where the rules below speak of "keeping both parents' Tracks", a Lineage-v1 writer keeps the merged binding — fused, or the merge refuses. The reader-visible answer is the same either way (§5.3's read-equivalence contract); only the representation differs.

§5.4 decides presence; this section decides presentation. Apply §5.4 first: it determines, per logical binding, what the merged set contains — including that a deletion on one side wins, and that a delete/modify conflict is refused. This section then governs how a pair the merged set retains is presented. A binding §5.4 resolved as deleted is not a conflict for this section to resolve, and MUST NOT be reintroduced by the union rules below.

When two parents both reference retained Tracks for the same (timeline_id, modality) pair, the merge Manifest must decide how to present them. Resolution depends on Track Kind:

6.1 Continuous Signal Tracks: union

For continuous signals (video, audio, embeddings), both parents' Tracks are kept as separate layered Tracks in the merge Manifest. They become two layers on the same (timeline, modality).

Readers querying the merge Manifest see Items from both Tracks (per 0006 §4.3 query semantics — multi-Track queries union candidate sets). For embedding tracks with overlapping coverage, the union is automatic; for video tracks with the same time-bucket, both Fragments coexist (different content hashes → different Objects).

6.2 Discrete Event Tracks: union

Same as Continuous Signal: both parents' Tracks are kept; readers see the union of events. Time-collisions between events from different parents are not conflicts — events at the same instant from different writers are expected (per 0000 §5.3), disambiguated by content hash.

6.3 Global Constant Tracks: lexicographically-greatest layer wins

Per 0007 §8.2: when both parents reference layered Constant Tracks of the same modality, the lexicographically-greatest layer-Track address wins. The other is shadowed (still present in the Manifest's tracks list, but the reader's title.text resolution returns the winning layer's value).

Why deterministic tiebreaker, not "most recent":

• Wall-clock ts fields aren't trusted (0003 §11 — writer clocks vary).
• Hash comparison is content-addressable and reproducible across implementations.
• Writers who want "their" correction to win can adjust the layer Track's content slightly until its hash sorts greater. This is rare in practice; explicit ordering by Publish sequence is the better workflow.

6.4 Genesis and Timeline conflicts

Conflicts on Genesis Objects or Timeline IDs are impossible by construction — Genesis Objects are content-addressed and the Timeline ID is the Genesis hash (0001 §5.1). Two Manifests referencing different Timeline IDs have different Timelines; they cannot conflict.

6.5 Schema/Registry conflicts — fatal, MUST refuse

This is a catastrophic-failure-on-silent-merge case. If two branches use different SpatialIndex Object hashes for the same modality (one writer regenerated the SpatialIndex with a new seed; one branch upgraded to a new dim; etc.), the embeddings produced by one are physically incompatible with the other. They occupy different bucket addresses; they hash queries differently; they are semantically different spaces masquerading as the same modality.

A naive auto-merge that unions the two registries' Track lists would produce a Manifest where:

• Some Buckets in the modality were built with SpatialIndex S₁.
• Other Buckets in the same modality were built with SpatialIndex S₂.
• Queries hash via whichever SpatialIndex the registry currently advertises (one of them) — and the other SpatialIndex's Buckets become invisible because their spatial keys are computed from a different projection.

This is silent data loss. It would not be caught by 0007 §6.1.1's lineage validation alone, because the SDK's "current SpatialIndex" matches whichever one the merge happened to keep — it just never tries to fetch the Buckets built with the other.

Therefore, the merger MUST detect SpatialIndex registry incompatibility and refuse the merge. Specifically:

1. Compare registry[modality].spatial_index arrays (per 0004 §3.3) across all parent Manifests.
2. If, for any modality, two parents declare different hash arrays (different SpatialIndex Objects, or different counts in the multi-table case), the merge MUST be aborted.
3. The SDK MUST surface this as a fatal MergeRefused error to the application. The error MUST name the conflicting modality and the conflicting SpatialIndex Object hashes from each parent.
4. Auto-merge logic that does not perform this check is non-conformant. This is not a soft default — it is a normative requirement.

Resolution paths the application MAY take:

1. Explicit selection. Choose one SpatialIndex hash; discard or quarantine the other branch's spatial-bucket data (manifest references remain immutable; they're just no longer reachable from the chosen merge path).
2. Re-indexing. Re-derive one branch's embedding Track using the other branch's SpatialIndex. Lossless if the source vectors (the Track being embedded — e.g., the parent video Track for an embedding layer) are still available. Requires running the embedding model again; cost depends on item count.
3. Coexist as distinct modalities (recommended default when re-indexing isn't feasible). Bump the modality parameter to disambiguate: embedding.f32.dim=768.bucketed.v1 and embedding.f32.dim=768.bucketed.v2. Both parents' Tracks remain visible; each is queryable via its own SpatialIndex. This loses the "single semantic space" property (queries must explicitly choose which version to search) but preserves both branches' work without re-computation.

Why this discipline matters at the protocol level: the lineage validation in 0007 §6.1.1 catches incorrect Bucket reads, but it can only catch what the SDK attempts to read. Manifest-merge time is the only point where the SDK can comprehensively detect an incompatible-SpatialIndex situation BEFORE it produces queries that miss data. Catching at merge time produces an actionable error; catching at query time produces silent under-recall.

The protocol does not silently auto-resolve schema/registry conflicts — they indicate a real semantic mismatch that requires explicit application-level decision.

7. Multi-Writer Reconciliation (Ref CAS Workflow)

The lock-free pattern from 0000 §5.2 made operationally concrete.

    Writer A                              Writer B
    ────────                              ────────

    Open(ref="main") → M_3                Open(ref="main") → M_3
    
    -- Phase 1 (parallel, no contention)
    PUT leaf objects                      PUT leaf objects
    PUT index pages                       PUT index pages
    PUT track objects                     PUT track objects
    
    -- Phase 2 (race for Ref)
    PUT manifests/<M_a>                   PUT manifests/<M_b>
    GET refs/main → ETag E_3              GET refs/main → ETag E_3
    PUT refs/main If-Match E_3            PUT refs/main If-Match E_3
        → 200 OK                              → 412 Precondition Failed
    
    Writer A wins. Writer B retries:
    
                                          GET refs/main → ETag E_a
                                                         → manifest = M_a
                                          
                                          -- Decision point: rebase or merge?
                                          
                                          (rebase path)
                                          Build new Manifest M_b' with
                                              parents = [M_a],
                                              tracks  = M_b's tracks +
                                                        any tracks from M_a
                                                        not in M_b.
                                          PUT manifests/<M_b'>
                                          PUT refs/main If-Match E_a
                                              → 200 OK
                                          
                                          (merge path)
                                          Build new Manifest M_merge with
                                              parents = [M_a, M_b],
                                              tracks  = union per §6,
                                              registry = union per §6.5.
                                          PUT manifests/<M_merge>
                                          PUT refs/main If-Match E_a
                                              → 200 OK

The losing writer's Phase 1 work is preserved — content-addressed Objects don't disappear. The rebuilt Manifest references them as before; nothing was wasted.

7.1 Rebase vs. merge

Rebase (recreating Writer B's Manifest with Writer A's tip as parent) gives a linear history. Suitable when:

• Writer B's changes are small.
• The application prefers linear history for human readability.
• Conflict resolution is trivial (no overlapping Tracks).

Merge gives a branching history with explicit reconciliation. Suitable when:

• Both writers' changes are substantial.
• Branching history aids accountability ("who contributed what?").
• Conflict resolution is non-trivial and the merge logic should be explicit.

The SDK provides both as Publish-time options; the application chooses.

7.2 Bounded retries

A writer's CAS-loss-then-retry loop SHOULD have a bounded retry count (default 5–10) with exponential backoff between attempts. Unbounded retry under sustained contention can starve writers.

If the retry budget is exhausted, the SDK surfaces a PublishConflict error to the application; the application decides whether to retry, abandon, or escalate.

7.3 Monotonic timestamp progression (observability rule)

ts is never trusted for conflict resolution (per 0003 §11 — writer clocks vary; lex-greatest-hash and explicit Publish ordering are the deterministic tiebreakers). But ts remains valuable for:

• Audit logs: "when was this Manifest published?"
• Compliance: "in what order did these regulated events get committed?"
• Debugging: "did the failure happen before or after the intervening Publish?"
• Topological-by-time displays: presenting the DAG to humans in approximate write order.

To preserve ts for these forensic uses without compromising correctness, writers SHOULD obey the Monotonic Progression Rule:

new Manifest's ts  ≥  max(parents' ts)

In words: a child Manifest's timestamp is at least as late as the latest parent's timestamp.

Writer-side enforcement (SHOULD):

When constructing a new Manifest:

• Compute min_ts = max(parents.ts).
• If the writer's wall clock wall_now ≥ min_ts: set ts = wall_now.
• If wall_now < min_ts (writer's clock is behind a parent's claimed time — clock skew): set ts = min_ts + 1ns. This produces a monotonically advancing chain even if individual writers' clocks are off.
• The SDK SHOULD log a warning when this fix-up occurs, naming the parent whose ts was ahead.

Reader-side observability (SHOULD):

When walking the DAG, the SDK SHOULD detect "Timeline Jump" violations — Manifests where ts < max(parents.ts). These are NOT failures (still valid Manifests; correctness doesn't depend on ts); they are forensic flags indicating clock skew or possibly malicious timestamp manipulation. The SDK SHOULD surface them via:

• Audit logs (every Timeline-Jump occurrence noted).
• Optional API surface (Manifest.has_timeline_jump: bool or similar) so applications can present the warning to operators.

Spec posture: SHOULD, not MUST. Writers with badly-skewed clocks shouldn't refuse to publish (that would break liveness for forensic-only signal); they should fix up ts to maintain monotonicity. Readers shouldn't reject Manifests with backward ts (that would break correctness on a forensic-only signal); they should log and proceed.

The rule is observability, not correctness. No logic in the protocol — conflict resolution, ordering, validation — depends on ts monotonicity. Writers and readers are explicitly permitted to operate on Manifests that violate the rule; the rule's purpose is to keep ts useful for humans inspecting history, not to make ts load-bearing.

8. Manifest Distribution Beyond Refs (resolves OQ-5)

Refs (per 0000 §5.2) handle in-band distribution when the backend is RefStore-Conformant. Three additional distribution modes apply when Refs are unavailable or insufficient:

8.1 Hash-addressed distribution (mandatory; works on every backend)

Writers share Manifest hashes out-of-band — chat messages, email, configuration files, another data system. Readers Open(<backend>/<manifest-hash>) directly. No Ref needed; works on ContentStore-only backends.

8.2 Pull-only Spaces

A Space without Refs is pull-only: readers receive Manifest hashes externally and Open against them. New writes by the same Space's writers produce new Manifest hashes; readers must be told the new hash to see them.

This is the deployment model for: archival reads of frozen Spaces, content-distribution networks (where readers fetch a known-good Manifest by hash), audit trails (Manifest hashes embedded in compliance records).

8.3 Federation across backends

Content-addressed Dataset Manifests remain portable: a Manifest MAY be opened through any backend that holds its byte-identical transitive closure. Logical read federation is defined separately by 0012: a Federation Manifest names exact child Dataset Manifests on independently resolved backends, and the reader routes and merges their results with explicit quorum and completeness semantics.

Each child closure MUST have a retention root local to the backend serving it. Copying such a closure is operator-coordinated and uses ordinary hash-verified GET/PUT; DreamDB defines no federation-specific replication verb or cross-backend CAS.

8.4 Ref-based distribution (RefStore-Conformant backends only)

The default. Writers advance the Ref via CAS; readers fetch the Ref to find the current Manifest hash; periodic polling (0006 §4.1.1) detects updates.

9. History Pruning and Compaction

The Manifest DAG grows monotonically. Without pruning, a long-running Space accumulates millions of historical Manifests. Two pruning patterns:

9.1 Manifest GC (operator)

Same algorithm as Object GC (0006 §7.3), applied to Manifests:

1. Walk all live Refs → collect reachable Manifest hashes.
2. LIST manifests/ → for each Manifest not in the reachable set AND older than safety threshold → DELETE.

Deleted Manifests' content-addressed Objects (Tracks, Buckets, etc.) are GC'd in the next pass once they become unreachable.

9.2 Squash-merge

A long branch can be squash-merged into the main line: the merging writer publishes a single Manifest with parents: [<main-tip>] (single parent — implicitly, but the squash discards the branch's intermediate history). The branch's intermediate Manifests become unreachable from any Ref and are eligible for GC (§9.1).

Squash-merging is a writer-driven decision. DreamDB provides no automatic squash; the writer constructs the squash Manifest manually and Publishes.

9.3 Snapshot-roll-up

For Spaces with millions of historical Manifests forming a long linear chain, a writer MAY publish a snapshot Manifest: a new Manifest whose parents: [] (declares itself a root) but whose tracks field unions everything from the entire prior chain. Old Manifests become unreachable from refs/main (which is advanced to the snapshot); GC reclaims them.

Snapshot roll-up loses history; the snapshot looks like a fresh Space's first Manifest. Use cases: archival copies, license/compliance handover to a different operator, cost reduction in long-running Spaces.

This is lossy by design. Operators who need permanent history retention should not use snapshot roll-up.

10. Out of Scope for this Document

• Backend push / durable change-stream notification verbs — 0006 §9; pull-based SDK Watch follows 0006 §4.1.2.
• Cross-Timeline alignment — 0001 §11.
• Cross-Space federation primitives — §8.3 mentions; formalized in 0012.
• Conflict-aware merge UI — application concern; the spec defines deterministic resolution rules, not user-interaction patterns.
• Replication / mirroring across backends — operator concern; v0 provides no protocol verb.

11. Open Questions Surfaced by This Document

• OQ-39 (→ 0009 §8): RESOLVED as a category contract. DAG walk, fast-forward detection, incompatible-SpatialIndex refusal, Constant tiebreaks, bounded CAS retries and timestamp handling are specified there. Concrete reports name the cases that ran. Branch-per-writer throughput is a separate 0023 performance result, not a required 10× improvement for conformance.
• OQ-40 (→ 0012): Copying a Manifest's transitive closure between backends. Resolved: 0012 §7 uses ordinary hash-verified GET/PUT plus a backend-local retention root. v0.2 deliberately defines no federation-specific HTTP service verb or credential format.
• OQ-41: Resolved. §4.4 defines explicit create-only snapshot-tag records, ordinary Ref mutation refusal, pinned reads, retention and legacy deployment limits (#358). This is not a never-advanced ordinary Ref or a backend WORM claim.

Next: 0009-conformance.md — defines the conformance test suite. Pulls together every "OQ-NN → 0009" pointer accumulated across 0000–0008 into a coherent battery of test vectors that any conforming implementation MUST pass.