DreamDB

Release Notes

Python 0.0.12 / npm 0.5.4 — 2026-09-10

Install dreamdb==0.0.12 or @dreamlake/dreamdb@0.5.4. These package versions are independent of the documentation site's version. The source baseline is a119479, including PR #366 and PR #367; release PR #368 binds the package versions.

Fixed: live reference-vector retention

#365 repairs GC deleting VectorStorage referenced only inside reference-mode Bucket records. Inline and paged indexes now share authenticated traversal, including header index/compressor and entry-side exact dependencies. Invalid or unreadable Bucket metadata aborts before sweep. The direct regression first reproduced missing vectors after real GC, then checked that copied snapshots survive GC and return identical query results while unrelated garbage is deleted.

Upgrade warning: upgrade or fence every collector processing these roots; upgrading readers alone is insufficient. GC requires an external maintenance fence and retained reader roots. Header authentication can add reads, and old Objects without Bao outboards may require whole-object verification. Reference bodies are materialized one at a time, not under a universal memory bound. Already-deleted Objects and #89's historical metadata damage are not repaired.

Query controls

Python and JavaScript expose query-local rerank pool size and inherit/approximate/exact policy (#342). Explicit exactness refuses unavailable exact sources; no silent fallback or global exact-recall promise. Graph queries reject explicit pool controls/non-inherit modes. See examples.

Rust operator capabilities

This source interval also adds read-only fragmentation observations (#343), explicit maintenance plans (#344), bounded Fragment repacking (#345), Reencode batch admission/resume (#356), pull-based Ref Watch (#357), create-only snapshot tags (#358), media Stream selection (#360), and verified closure-copy plans (#359). These do not add corresponding Python/JavaScript wrappers.

Copy plans require operator-held source roots and destination GC/root-mutation fences. They are not a background replication service, retain no data by themselves, buffer whole Objects serially, and explicitly refuse unsupported profiles. Tags are not backend WORM; Watch is not a durable event log; Reencode admission does not preempt an encoder. IVF/global-PQ composition evidence (#346) does not claim residual PQ.

Packaging and evidence boundaries

The versioned npm tarball passed the existing clean-install smoke, CommonJS version/export and JS boundary checks. Product-fix CI passed all 11 executed jobs. Package-CI and registry provenance belong to the core release record, not inferred from a documentation build. Python retains CPython 3.8 ABI-stable wheels for macOS arm64 and manylinux2014 x86_64/aarch64; no Windows/Intel Mac wheel or source distribution. npm retains its bundler/web/Node feature split.

Python 0.0.11 / npm 0.5.3 — 2026-09-10

Both packages are published. These are package versions, not a new protocol or documentation-site version. Release PR #348 pins the merged source. Install dreamdb==0.0.11 from PyPI or @dreamlake/dreamdb@0.5.3 from npm.

Implementation

Progressive geometry, optional entity keys, structured ragged/CSR arrays, magnitude-preserving L2, embedding identity enforcement, compressed/Fresh/routed graph capabilities, typed HotShards, scalar path-copy updates and executed hybrid plans are included. See feature examples and limits.

Storage work since the previous Python release also includes capacity-aware bucket compaction and configurable payload caps, bounded per-cell merge work, persistent scalar B-trees, paged Timeline-scoped tombstones, backfill coverage, canonical multi-vector records, federation, rendition playlists and reference-mode compression. Encryption-envelope work remains a specification, not SDK encryption.

Changed

Behavior change: embedding compaction rejects a zero threshold. Capacity is a record-payload cap, not a total process-memory bound. New stored forms require capable consumers and collectors; the patch version does not make old readers safe for every new feature. Preserve entity revision tokens and evaluate retained history before rollback. Exact reranking is not global exact recall, and cost-model latency budgets are not hard deadlines.

Release acceptance

The npm tarball passed existing installed-package smoke, CommonJS and SDK boundary checks. The PyPI publishing workflow completed successfully and published exactly three wheels: macOS arm64, Linux x86_64 and Linux aarch64 (CPython 3.8 ABI-stable). The installed macOS wheel also passed the existing geometry, entity-key and typed-array boundary tests. These checks do not claim universal cross-platform performance or general support for every normative capability.

Historical documentation snapshot — 2026-09-10

At that snapshot, the site reflected core main 65c8e05: 28 specs, 66 resolved / 38 open OQs, and 16 Draft headers. Newly documented implementations include optional entity keys, progressive geometry, structured ragged/CSR, L2, compressed/routed/Fresh graphs, typed HotShards, embedding identity checks, scalar path-copy updates and hybrid planning. See the feature guide and limits.

That documentation publication was not an SDK package release. At the time, Python 0.0.10 and npm 0.5.2 were the published versions and these examples required a source build. This paragraph records that historical state, not current registry availability.

Per-version release notes for the DreamDB protocol and its reference implementation. Pre-1.0 — minor bumps may include breaking changes; each release flags those under Changed.

SDK entries name both independently versioned packages explicitly; older entries below record protocol/reference-implementation staging rather than SDK releases. For current availability and the meaning of each version number, see Versions.

0.1.0 — 2026-05-22 (Initial staging)

Specification

  • 23 protocol spec documents (0000–0022) covering data model, content addressing, time encoding, spatial indexing, backend interface, protocol operations, streaming encapsulation, versioning, conformance, vector compression, scalar indexing, federation, graph indexing, streaming extensions, hybrid retrieval, streaming freshness, schema evolution, multi-tenant operation, encryption, tombstones, compaction, and fragment packs.
  • 11 Chinese translations of specs 0000–0009.

Implementation

  • Rust reference implementation: core, protocol, connector, CLI, dataset, conformance, bench crates.
  • Python SDK (dreamdb-dataset-python): Dataset, iter_stream, delete, merge, history, list_refs, count.
  • AWS S3 / SigV4 production path via dreamdb-connector-http.
  • MinIO dev backend preserved for local development.

Design Documents

  • 8 design documents covering dataset platform architecture, known flaws retrospective, scope boundaries, TODO roadmap, ML training tutorial, 10B-scale blockers, sharded ingest, and fragment packs.