DreamDB

DreamDB v0 — Spec Index

A single navigable entry point for the DreamDB v0 specification. Maps every Open Question (OQ) to its owner and disposition and points at every spec document.


The 28 Spec Documents

Read the foundational documents first, then the relevant capability. Later owning sections refine earlier summaries; document number alone is not a blanket override. A conflict must be corrected or explicitly recorded, not silently resolved by order.

#DocumentWhat it pins down
0spec/0000-overview.mdVocabulary, four first principles, three-layer architecture (App → SDK → HTTP → Object Store)
1spec/0001-data-model.mdFive entities (Space, Timeline, Track, Item, Manifest); three Track kinds; Item ≠ Object
2spec/0002-content-addressing.mdBLAKE3, deterministic CBOR, address grammar, modality-tag grammar, paged Index Pages
3spec/0003-time-encoding.mdu64 ns ticks, integer-only arithmetic, 16-char hex, billion-scale sizing
4spec/0004-spatial-indexing.mddreamdb.lsh-cosine, ChaCha20 hyperplanes, multi-table & multi-probe levers
5spec/0005-backend-interface.mdHTTP contract, Manifest Supremacy, ETag normalization, transport-agnostic (HTTP/1.1+; HTTP/2/3 a SHOULD-negotiate perf upgrade)
6spec/0006-protocol-operations.mdEight verbs, per-session cache, GC algorithm, Stream prefetch, Ref freshness
7spec/0007-streaming-encapsulation.mdCMAF Fragments, Spatial Bucket layout, Index Page byte format, dynamic B-tree height
8spec/0008-versioning-collab.mdManifest DAG, branching, merging, MUST-refuse-on-incompat, monotonic-ts
9spec/0009-conformance.mdConformance roles, test categories, meta-OQ resolutions (CBOR tags, multihash)
10spec/0010-vector-compression.mdVectorCompressor framework + raw-f32/rabitq-cosine/pq-cosine; 200-byte v2 bucket header; re-rank contract — normative, v0.1 (qinco reserved, OQ-46)
11spec/0011-scalar-indexing.mdNative bitmap equality and ordered B-tree filtering, persistent path-copy appends, ScalarIndex Object — implemented reference paths
12spec/0012-federation.mdFederation Manifest, runtime backend resolution, cross-cluster scatter-gather and explicit partial results — implemented v0.2 read federation
13spec/0013-graph-indexing.mdVamana GraphIndex/GraphPages, compressed ADC graphs, routing GraphIndex, explicit raw Fresh Vamana append/consolidation — implemented reference paths; no background scheduler
14spec/0014-streaming-extensions.mdPath A item chunking (map-encoded ItemManifest and Schema write policy), Path B CMAF renditions (RenditionPlaylist) — Draft; published Path A encoding retained, portable conformance remains OQ-58
15spec/0015-hybrid-retrieval.mdTextIndex + MultiVectorIndex; HybridQuery verb; RRF default fusion; query planner cost model — draft, Phase 4
16spec/0016-streaming-freshness.mdv1 embedding and v2 scalar/text HotShards, explicit flush and Fresh Vamana maintenance; consumer staleness scheduling and index-health automation remain design work — partially implemented
17spec/0017-schema-evolution.mdMulti-version modalities; Reencode verb; compatible_with planner hint — draft, Phase 4
18spec/0018-multi-tenant.mdPer-Space quotas; deployment-local verified authorization boundary; TenantUsageBatch; fair-share scheduling — Draft; portable token profile and reference gateway remain OQ-78/OQ-92
19spec/0019-encryption.mdLineage-v3 encryption policy; convergent + randomized DENC envelopes; KMS recipient slots; authenticated range reads — normative format, no reference writer/reader
20spec/0020-tombstones.mdPersistent paged TombstoneIndex plus legacy flat lists; per-Space dreamdb.tombstones registry; range-pruned suppression; deletion levels + compliance posture — normative, v0.2
21spec/0021-compaction.mdBucket/tombstone compaction; lineage + anchor-conflict refusal; read-online property — normative, v0.1 (sharded compaction optional)
22spec/0022-fragment-packs.mdMany-Items-per-Object packing (pack_items); pack_offset byte-range reads — normative, v0.1
23spec/0023-performance-profiles.mdPerformance profiles (workload + environment + targets); the conformance-vs-performance boundary — companion to 0009
24spec/0024-embedding-spec-identity.mdEmbedding-spec identity: which fields make two vectors comparable, and the measured-parity bar for calling two implementations one spec — companion to 0007/0010
25spec/0025-typed-array-items.mdDense raw/NPY, ragged-array and sparse-CSR Items; validated Dataset/Python/WASM write, read and physical audit — implemented reference paths; structured Arrow projection unsupported

| 26 | spec/0026-entity-keys.md | Optional typed logical keys, stable entity identities and random revision tokens; mandatory lineage-v4; atomic Sample/tombstone/index publication; conservative divergent-key merge refusal — normative v1 | | 27 | spec/0027-progressive-geometry.md | Item-internal point/splat cells, packed payloads, paged hierarchy and discrete mesh LODs; authenticated incremental range reads — normative v1 |


Status and counting rules

FieldValue
Document count28 (0000–0027)

This reconciliation is updated through main 213d948 (2026-09-10); it is not a live CI or release-status feed.

  • Draft describes document maturity, not the absence of implemented code. Existing normative sections retain their obligations; a proposal does not become a supported capability merely because another section has shipped.
  • Normative / Accepted means the stated contract is adopted within its scope. It does not imply every SDK implements it. For example, 0019 has a normative encryption format but no reference Dataset writer/reader.
  • Implemented must name a capability and boundary. Shared Rust code exposed through Python/WASM is not independent-implementation agreement.
  • Resolved OQ means the owning document gives the decision. Implementation, portable conformance, performance and deployment can remain separate work. A proposed answer or a test-category name alone is not closure.
  • Performance examples are informative under 0023. Unbound test counts and historical benchmark-progress snapshots are not maintained here.

16 documents retain a Draft header: 0000–0009, 0014–0018, and 0023. This revision does not promote them wholesale. The other 12 are: 0010, 0011, 0013, 0019–0022, and 0026–0027 (Normative within their stated scope), 0012 and 0024 (Implemented), and 0025 (Accepted). Read the owning status paragraph for deferred subfeatures, notably QINCo, encryption runtime, compressed Fresh graphs and consumer freshness scheduling.

Open Questions: complete ledger

There are 104 distinct OQ identifiers: every numeric identifier OQ-1 through OQ-102 (none unused), plus OQ-PP1 and OQ-PP2 in 0023.

  • Numeric questions: 77 resolved, 25 open.
  • Named performance questions: 0 resolved, 2 open.
  • Combined: 77 resolved, 27 open.

The six compatibility dispositions in the review record are recorded separately, without allocating new OQ identifiers. Their remaining feature/conformance limits do not silently add to or subtract from these totals.

Open: owner and functional area

“Open” includes deferred decisions and unfinished capability-specific conformance. The owner below is where a resolution must land, even when an earlier document originally raised the question.

OQOwning specFunctional area / remaining question
OQ-180003Time: coarser resolutions beyond canonical nanoseconds
OQ-310005Storage: Azure block-upload boundary; S3 multipart is already addressed
OQ-370007Media: additional codec-specific contracts
OQ-420009Conformance: public report registry
OQ-430009Conformance: suite release/governance cadence
OQ-460010 / 0009Compression: QINCo determinism and portable vectors before promotion
OQ-500012Federation: child retention-root auditing
OQ-520012Federation: runtime mirror discovery
OQ-530013Graph storage: per-page node-offset index
OQ-570014Chunked Items: MIME metadata ownership
OQ-580014 / 0009Chunked Items: portable read/range/GC conformance for the map wire format
OQ-600014Chunked Items: numeric size bound, not a CBOR integer-width saving
OQ-640015Hybrid search: learned-retrieval default (SPLADE/ColBERT)
OQ-650015 / 0009Hybrid search: portable planning/fusion semantics; benchmarks separate
OQ-670016Freshness: proposed hot-tier durability exception; current commit obligations remain
OQ-690016Freshness: learned sparse/multi-vector drift metrics
OQ-700016 / 0009Graph updates: portable streaming-Vamana evidence
OQ-710017Evolution: explicit similarity-space transforms
OQ-740017 / 0009Reencode: portable failure/resume vectors
OQ-750018Multi-tenancy: usage-history authenticity/non-equivocation
OQ-760018Multi-tenancy: multi-region quota aggregation
OQ-770018 / 0009Multi-tenancy: fair-share scheduling conformance
OQ-780018Multi-tenancy: portable token profile; deployment-local authorization boundary defined
OQ-900020 / 0019 (raised in 0000)Deletion: selective erasure versus retained/shared history
OQ-920018 (formerly ledger-only)Trusted mediation gateway and its conformance boundary
OQ-PP10023Performance: shared version-pinned corpora/query sets
OQ-PP20023Performance: standard environment-class identifiers

Resolved: decision and owning contract

Each identifier appears once in this disposition table. A group shares an owning contract, not necessarily a delivery milestone.

OQsOwning contractDecision
OQ-410008 §4.4 / 0002 §10Create-only typed snapshot tags, pinned reads and GC roots; ordinary Ref mutation refused, not backend WORM (#358)
OQ-610014 §3.3 / 0006 §4.4.2Explicit media Stream selection, committed defaults, lazy verified Fragments and pinned snapshots (#360)
OQ-340006 §4.1.2Optional pull-based Ref Watch, independent cursors and pinned reads; no backend push or durable replay (#357)
OQ-720017 §3.5Enforced per-pass batch admission and resumable pauses, not encoder preemption or memory/tenant quotas (#356)
OQ-1, OQ-190003 §2–§4Genesis-relative u64 nanoseconds; no negative anchors
OQ-2, OQ-90004 §5LSH-cosine spatial-key derivation
OQ-30005 §5Content consistency and linearizable Ref CAS
OQ-4, OQ-100007 §5CMAF encapsulation; default Fragment duration
OQ-50008 §8Manifest distribution through Refs or immutable hashes
OQ-60002 §5Modality grammar and namespacing
OQ-70008 §6.3 / 0007 §8.2Constant conflict rule
OQ-80007 §8Per-modality time-batch sizing
OQ-11, OQ-170009 §3.2Local foreign-CBOR tag 65521; no time tag in ordinary Object schemas. Global registration is not established
OQ-120009 §3.1DreamDB's fixed 0x1e + 32 bytes digest encoding
OQ-130007 §6.5Spatial-first path layout, not LIST-based query visibility
OQ-14, OQ-15, OQ-20, OQ-210007 §7Index-page sizing/paging, time deltas; no byte-size delta
OQ-16, OQ-23, OQ-240007 §6Record layouts, inline/reference modes and payload byte cap
OQ-220005 §9Filesystem adapter exists; remote HTTP and local semantic evidence differ
OQ-960021 §7.1Snapshot-bound read-only fragmentation and exact/unknown debt
OQ-680016 §8.4Explicit operator maintenance plans; automatic scheduling deferred
OQ-98, OQ-990022 §6 / §10Workload-driven sizing and explicit bounded Fragment repacking
OQ-250004 §5.6.1 / 0010 §11.1Independently bound IVF + global-vector PQ; no residual-codec claim
OQ-260004 §5.8Deterministic L2 projection, signed bins and raw distance query semantics (#321)
OQ-27, OQ-30, OQ-33, OQ-36, OQ-390009 §5–§8Conformance categories specified; concrete reports still identify what actually ran
OQ-280006 §6.4–§6.5Adaptive widening/preload tactics implementation-defined
OQ-290005 §6HTTP/1.1+ semantics; HTTP/2/3 performance recommendation
OQ-320005 OQ-32Browser PresignedBackend pattern, not a multi-tenant gateway
OQ-350008 §4.2Explicit Publish parents
OQ-38, OQ-470010 §8Compressed Bucket layout; compressor at 86..119, reservation 119..200
OQ-440010 §5 / 0006 §4.3Query-local partition pool and explicit inherit/approximate/exact policy
OQ-400012 §7Verified ordinary Object copy plus backend-local retention roots
OQ-450010 §8.3Compressed reference records inline codes beside exact locators
OQ-480012 §5Absolute availability threshold, separate from score/recall policy
OQ-490012 §7.1Canonical snapshot-copy plans, verified resume and retained destination publication (#359); reference-mode retention repaired by #365; explicit capability limits remain
OQ-510013 §6 / 0012Exact-bound federation routing GraphIndex and eligible-shard traversal (#319)
OQ-540013 OQ-54 / 0016 §4Raw Fresh Vamana append, id-preserving consolidation and page lineage; no background scheduler (#318)
OQ-550013 §4 / §7Pinned 10,000-node build/search artifacts
OQ-560013 §4.4.1Compressed graph ADC candidates and direct exact-source pool reranking (#320)
OQ-590014 §2.3Reuse Fragment chunk-address slot; map wire shape separately specified in §2.2
OQ-620015 §11Positions off by default, opt-in
OQ-630015 §6.2Operator-tunable cost model with SDK defaults
OQ-660006 §2.2.1HybridQuery extends Query, not a new core verb
OQ-730006 §2.2Reencode is a distinct optional extension verb
OQ-79–OQ-870019 §2–§7 / 0009 §8.6.5Encryption wire-policy decisions and mandatory vectors; runtime writer/reader absent
OQ-880026 / 0001 §5.4Optional typed entity keys, stable ids and atomic revision operations; v1 size and merge limits remain (#322)
OQ-890020 §3.6–§5.4Persistent paged Timeline-scoped tombstone index
OQ-910002 §6.5.4Bao outboards and authenticated ranges; whole-read fallback for historical Objects
OQ-930002 §13 / 0015 §3.7Indexes are auxiliary registry-bound Objects, not Tracks; field-qualified bindings and contextual decode
OQ-94, OQ-950021 §5–§6 / §9Per-rewritten-Bucket exact sidecars; per-shard JSON output shape
OQ-970022 §10Schema-pinned pack_items
OQ-100, OQ-1010025 §15Array family discriminant and byte-exact semantic labels
OQ-1020017 §7 / 0025 §13Frozen-basis backfill claims. Ongoing omission is separately resolved for v1 as OutsideDomain by 0017 §7.11

Historical numbering

Keep these old references interpretable; do not reuse their numbers.

Former duplicateCurrent identifier
0021 OQ-44 / OQ-45 / OQ-46OQ-94 / OQ-95 / OQ-96
0022 OQ-47 / OQ-48 / OQ-49OQ-97 / OQ-98 / OQ-99

The original 0010 OQ-44–47 and 0012 OQ-48–49 retain their numbers. OQ-PP1/PP2 retain their published named identifiers rather than being silently renumbered into the numeric series.

Cross-cutting reading rules

RuleAuthorityConsequence
Snapshot visibility0005 §5.3.1Query the chosen Manifest's indexes; LIST does not authorize Items
Item versus Object0001 §3.1 / 0002 §6An Item may require an Object address plus locator; repacking can change that address
Binding identity0002 §7.2Type identity, active binding, retained lineage and GC reachability are different facts
Time versus entity0001 §5.4Equal anchors do not establish entity identity; scalar lookup does not enforce uniqueness
Critical extensions0002 §3.1.0Payload, visibility or closure changes cannot be silently ignored
Integrity of ranges0002 §6.5.4Whole-root verification is not obtained by hashing an arbitrary slice
Protocol versus performance0023 §7Workload targets do not redefine format conformance
Evidence scope0009A listed category is not an executed result; an implementation's roundtrip is not independent interop

See the reconciliation record for corrected contradictions and compatibility dispositions with their remaining limits. See README.md for usage; current release/CI evidence belongs with its exact commit and report, not a floating “all tests green” count in this ledger.