# DreamDB v0 — Spec Index

A single navigable entry point for the DreamDB v0 specification. Maps every Open Question (OQ) to its resolution and points at every spec document.

---

## The 21 Spec Documents

Read in order; each document inherits vocabulary and decisions from the ones before it.

| # | Document | What it pins down |
| - | -------- | ----------------- |
| 0 | [`spec/0000-overview.md`](spec/0000-overview.md) | Vocabulary, four first principles, three-layer architecture (App → SDK → HTTP → Object Store) |
| 1 | [`spec/0001-data-model.md`](spec/0001-data-model.md) | Five entities (Space, Timeline, Track, Item, Manifest); three Track kinds; Item ≠ Object |
| 2 | [`spec/0002-content-addressing.md`](spec/0002-content-addressing.md) | BLAKE3, deterministic CBOR, address grammar, modality-tag grammar, paged Index Pages |
| 3 | [`spec/0003-time-encoding.md`](spec/0003-time-encoding.md) | u64 ns ticks, integer-only arithmetic, 16-char hex, billion-scale sizing |
| 4 | [`spec/0004-spatial-indexing.md`](spec/0004-spatial-indexing.md) | `dreamdb.lsh-cosine`, ChaCha20 hyperplanes, multi-table & multi-probe levers |
| 5 | [`spec/0005-backend-interface.md`](spec/0005-backend-interface.md) | HTTP contract, Manifest Supremacy, ETag normalization, HTTP/2 mandate |
| 6 | [`spec/0006-protocol-operations.md`](spec/0006-protocol-operations.md) | Eight verbs, per-session cache, GC algorithm, Stream prefetch, Ref freshness |
| 7 | [`spec/0007-streaming-encapsulation.md`](spec/0007-streaming-encapsulation.md) | CMAF Fragments, Spatial Bucket layout, Index Page byte format, dynamic B-tree height |
| 8 | [`spec/0008-versioning-collab.md`](spec/0008-versioning-collab.md) | Manifest DAG, branching, merging, MUST-refuse-on-incompat, monotonic-ts |
| 9 | [`spec/0009-conformance.md`](spec/0009-conformance.md) | Conformance roles, test categories, meta-OQ resolutions (CBOR tags, multihash) |
| 10 | [`spec/0010-vector-compression.md`](spec/0010-vector-compression.md) | VectorCompressor Object, `dreamdb.qinco-cosine`, modality `compress=` suffix, re-rank contract — *draft, Phase 3* |
| 11 | [`spec/0011-scalar-indexing.md`](spec/0011-scalar-indexing.md) | Native scalar filtering: B-tree / bitmap algorithms, ScalarIndex Object — *draft, Phase 2* |
| 12 | [`spec/0012-federation.md`](spec/0012-federation.md) | Federation Manifest, `federate` verb (resolves OQ-40), cross-cluster scatter-gather — *draft, Phase 3* |
| 13 | [`spec/0013-graph-indexing.md`](spec/0013-graph-indexing.md) | Vamana / DiskANN — GraphIndex + GraphPage Objects; router GraphIndex for cross-shard — *draft, Phase 3* |
| 14 | [`spec/0014-streaming-extensions.md`](spec/0014-streaming-extensions.md) | Path A item chunking (ItemManifest), Path B CMAF renditions (RenditionPlaylist) — *draft, Phase 3* |
| 15 | [`spec/0015-hybrid-retrieval.md`](spec/0015-hybrid-retrieval.md) | TextIndex + MultiVectorIndex; HybridQuery verb; RRF default fusion; query planner cost model — *draft, Phase 4* |
| 16 | [`spec/0016-streaming-freshness.md`](spec/0016-streaming-freshness.md) | HotShard delta tier; `dreamdb.fresh-vamana-cosine`; index-health signal; bounded-staleness contract — *draft, Phase 4* |
| 17 | [`spec/0017-schema-evolution.md`](spec/0017-schema-evolution.md) | Multi-version modalities; `Reencode` verb; `compatible_with` planner hint — *draft, Phase 4* |
| 18 | [`spec/0018-multi-tenant.md`](spec/0018-multi-tenant.md) | Per-Space quotas; tenant_id in capability tokens; TenantUsageBatch; fair-share scheduling — *draft, Phase 4* |
| 19 | [`spec/0019-encryption.md`](spec/0019-encryption.md) | Convergent + per-Space modes; EncryptionMeta header; KMS abstraction — *sketch, Phase 4* |
| 20 | [`spec/0020-tombstones.md`](spec/0020-tombstones.md) | TombstoneListObject; per-Space `dreamdb.tombstones` registry; read-side suppression filter — *Phase 5* |

---

## Open Questions: Audit

**87 OQs total** across v0 + Phase-3 + Phase-4 drafts. v0 (OQ-1–43): 41 resolved actively, 2 deferred to v0.1+. Phase-3 (OQ-44–61): formally opened by drafts 0010/0012/0013/0014 — most deferred to first implementation pass. Phase-4 (OQ-62–87): formally opened by drafts 0015–0019 — most deferred to first implementation pass; the 9 OQs in 0019 are sketch-level and block the spec/0019 sketch→full-draft promotion.

### Resolved within v0 (29 OQs)

| OQ | Topic | Resolution |
| -- | ----- | ---------- |
| OQ-1 | Genesis-relative vs. absolute origin | `0003` §2 — Genesis-relative canonical, optional Unix-ns back-ref |
| OQ-2 | Spatial-indexing scheme | `0004` §5 — `dreamdb.lsh-cosine` |
| OQ-3 | Backend consistency model | `0005` §5 — strong RAW for content; linearizable CAS for refs |
| OQ-4 | Per-modality container | `0007` §5 — CMAF / fragmented MP4 |
| OQ-5 | Manifest distribution | `0008` §8 — Refs / hash-addressed / pull-only / federation |
| OQ-6 | Modality-tag grammar | `0002` §5 — dot-separated; reverse-DNS for user-defined |
| OQ-7 | Constant correction conflicts | `0008` §6.3 + `0007` §8.2 — lex-greatest layer wins |
| OQ-8 | Event bucketing default | `0007` §8 — Time-batch format; per-modality bucket-duration |
| OQ-9 | Spatial bucket-key derivation | `0004` §5 — ChaCha20-seeded LSH-cosine |
| OQ-10 | Default Fragment duration | `0007` §5.4 — 2 s default; 1–30 s range |
| OQ-11 | Concrete CBOR tag numbers | `0009` §3.2 — private-use range 65521–65525 |
| OQ-12 | Multihash algorithm tags | `0009` §3.1 — `0x1e` (IPFS-aligned) |
| OQ-13 | Spatial+time segment order | `0007` §6.5 — spatial-first |
| OQ-14 | Index Page fanout & size | `0007` §7.1 — B = 256, 16 KiB target |
| OQ-15 | Inline-vs-paged threshold | `0007` §7.2 — 1 MiB |
| OQ-16 | Per-modality byte layout | `0007` §6 — fixed-size + offset-table patterns |
| OQ-17 | CBOR tag for time anchor | `0009` §3.2 — `65524` |
| OQ-20 | Index Page delta encoding | `0007` §7.3 — deltas for time; positional arrays |
| OQ-22 | Local-FS Connector | `0005` §9 — HTTP-only in v0; MinIO for dev |
| OQ-23 | Vector storage layout | `0007` §6 — inline (default) vs. reference (multi-table) |
| OQ-24 | Bucket-splitting rule | `0007` §6.4 — 100 MiB threshold |
| OQ-27 | LSH conformance vectors | `0009` §5.3 — multi-arch f32 discipline |
| OQ-28 | Adaptive widening / speculative preload | `0006` §6.4 / §6.5 — implementation-defined |
| OQ-29 | HTTP version requirement | `0005` §6 — HTTP/2 mandatory |
| OQ-30 | HTTP test vectors | `0009` §6 — full battery |
| OQ-33 | Verb conformance vectors | `0009` §7 — full battery |
| OQ-35 | Explicit parent for `Publish` | `0008` §4.2 — `Publish(parents: [...])` |
| OQ-36 | Byte-format vectors | `0009` §5.4 — full battery |
| OQ-39 | DAG conformance vectors | `0009` §8 — full battery |

### Decided "no" in v0; may revisit in v0.1+ (2 OQs)

| OQ | Topic | v0 decision |
| -- | ----- | ----------- |
| OQ-19 | Negative time anchors | v0: forbidden (unsigned u64). v0.1 may add bias scheme. |
| OQ-21 | Byte-size delta encoding | v0: as-is, no delta. v0.1 may add if benchmarks justify. |

### Deferred to v0.1+ as feature additions (7 OQs)

| OQ | Topic | Rationale |
| -- | ----- | --------- |
| OQ-18 | Coarser time resolutions (`ms`, `s`) | v0 ns is sufficient; revisit if storage waste measured |
| OQ-26 | `dreamdb.lsh-l2` algorithm | L2-metric modalities; v0 cosine-only |
| OQ-31 | Multipart upload | v0 caps Bucket Objects at 100 MiB; multipart unneeded |
| OQ-32 | Presigned-URL Connector | Web/browser deployment pattern |
| OQ-34 | `Subscribe` / `Watch` verb | v0 is pull-only via Ref polling |
| OQ-37 | Additional media codecs | AV1, HEVC, FLAC, VP9 — all CMAF-compatible |
| OQ-41 | `tag` namespace | Immutable Refs (release tags) |

### Resolved in Phase-3 / Phase-4 spec drafts (4 OQs)

| OQ | Topic | Resolution |
| -- | ----- | ---------- |
| OQ-25 | `dreamdb.pq-ivf` algorithm | `0010` §7 — superseded by `dreamdb.qinco-cosine` (NeurIPS 2025); classical PQ sketched in `0010` §11.1 |
| OQ-38 | PQ-compressed vector layout | `0010` §8 — bucket header extends to 200 bytes carrying `vector_compressor_hash` |
| OQ-40 | `federate` verb | `0012` §4 — push + pull modes with capability tokens (Ed25519) |
| OQ-54 | Streaming graph updates | `0016` §4 — `dreamdb.fresh-vamana-cosine` ships append + background consolidation + tombstone deletes |

### Deferred to community / governance process (2 OQs)

| OQ | Topic |
| -- | ----- |
| OQ-42 | Public registry of conformance reports |
| OQ-43 | Conformance suite release cadence |

---

## Foundational Doctrines (cross-cutting)

The spec accumulates several disciplines that span multiple documents. Quick reference:

| Doctrine | Defined in | Why |
| -------- | ---------- | --- |
| **Time as the sole primary key** | `0000` §5.1 | No user-level IDs; everything joins via timeline |
| **Strict immutability for content; CAS for refs** | `0000` §5.2 | Lock-free data plane; refs are the only mutable thing |
| **Address = `<object-address> · <intra-object-locator>`** | `0000` §5.3 | Item ≠ Object; bucketing makes 1B-scale viable |
| **Streaming-native encapsulation** | `0000` §5.4 | Fragment-as-Item; index = seek pointer |
| **Manifest Supremacy** | `0005` §5.3.1 | Hot path uses cached `object_index`; LIST is bootstrap-only |
| **Universal byte-range URIs** | `0002` §6.5 | All `dreamdb://` URIs use `#bytes:<a>-<b>` form; opaque to readers |
| **Integer-only time arithmetic** | `0003` §4.1 | Zero f64 ops on the time path |
| **F32 left-fold determinism** | `0004` §5.4 | Bit-identical LSH across hardware paths |
| **Pure-scalar reference is canonical** | `0009` §5.3.1 | SIMD paths verified against scalar |
| **Lineage validation at decode time** | `0007` §6.1.1 | Bucket headers carry `spatial_index_hash` |
| **MUST-refuse-on-SpatialIndex-incompat** | `0008` §6.5 | Catastrophic silent-merge case at v0; non-conformance gate |
| **Feature-branching at scale** | `0008` §3.1 | Per-user/team Refs; coordinator merges into `refs/main` |
| **Monotonic ts (observability)** | `0008` §7.3 | SHOULD; never trusted for correctness |

---

## What's Next

The v0 spec is complete and the reference implementation is shipping. Current focus:

1. **Billion-scale validation**: imagenet-1k (1.3M records) ingest in progress 2026-05-18; SIFT1M / LAION-100M next.
2. **Production hardening**: observability spec, cross-SDK interop tests, RBAC / capability tokens (deferred to v0.1).
3. **Tombstone storage compaction** (per `spec/0020 §6`): operator pass that reclaims bytes for tombstoned anchors.
4. **Multi-parent merge formalization**: `design/0007-sharded-ingest.md` documents the algorithm; may be promoted to its own spec number.

See [`README.md`](README.md) for the project pitch, status table, and quickstart.

---

## Spec Status

| Field | Value |
| ----- | ----- |
| Spec version | `v0` + Phase-3 (`0010`/`0012`–`0014`) + Phase-4 (`0015`–`0019`) + Phase-5 (`0020`) |
| Document count | 21 (`0000`–`0020`) |
| Open Questions | 87 total — 45 resolved (v0 + Phase-3/4/5); 42 open or deferred to first-implementation pass |
| Reference implementation | shipping (`dreamdb-*` crates, `dreamdb-dataset-python`, browser SDK) |
| Conformance suite | partial (CBOR / address / time / spatial covered; cross-SDK interop deferred) |
| Test suite | 721 green across crates |
| Billion-scale benchmark | in progress (1.3M-record imagenet-1k ingest 2026-05-18) |