DreamDB

API Reference

Every public export of @dreamlake/dreamdb. Group headings match the export comments in src/index.ts.

Schema

Declarative field builder. Methods are chainable.

ts
import { Schema } from "@dreamlake/dreamdb";

const schema = new Schema()
  .addImage("image", { mime: "jpeg" })
  .addEmbedding("embedding", {
    dim: 512,
    algorithm: "dreamdb.ivf-cosine",
    rerank: true,
  })
  .addScalarCategorical("label");
MethodDescription
addImage(name, opts?)Image field. opts.mime defaults to "jpeg".
addVideo(name, opts?)Video field. opts.mime defaults to "mp4".
addAudio(name, opts?)Audio field. opts.mime defaults to "wav".
addEmbedding(name, opts)Vector field. Requires dim. Options: algorithm, lshBits, compressor, spatialIndex, rerank.
addScalarCategorical(name)Categorical string (labels, splits).
addScalarBool(name)Boolean.
addScalarInt(name)64-bit integer.
addScalarFloat(name)64-bit float.
addScalarString(name)Free-form string.
addScalarTimestamp(name)Nanosecond timestamp.

Every field accepts an optional required (default true).

Dataset

The high-level write + query handle.

Static factories

ts
Dataset.create(name: string, schema: Schema, backend: Backend): Promise<Dataset>
Dataset.open(name: string, opts: { backend: Backend }): Promise<Dataset>
Dataset.openRef(refName: string, backend: Backend): Promise<Dataset>

Append

ts
ds.appendMany(records: Record<string, any>[]): Promise<number>

Records are plain objects keyed by field name. Embedding values are number[] or Float32Array. Reserved key _anchor (bigint) supplies a caller-controlled time anchor — within one call, either every record carries it or none does (mixing is rejected).

Vector queries

ts
ds.iterVector(opts: {
  field: string;
  query: number[] | Float32Array;
  topK: number;
  batchSize?: number;                              // default 64
  whereEq?: Record<string, string | number | boolean>;
}): Promise<Batch[]>

When whereEq is provided, the filtered fields are auto-populated on each result batch.

Streaming iteration

ts
ds.iterStream(opts: { batchSize?: number }): AsyncIterable<Batch>

Bounded memory; one batch at a time. Use for training, export, analytics.

Snapshots

ts
ds.snapshot(label: string): Promise<SnapshotInfo>
Dataset.openRef(label, backend): Promise<Dataset>   // re-open the snapshot

Branches + merge

ts
ds.branch(name: string): Promise<Dataset>
ds.mergeMany(branchNames: string[], opts?: { strategy?: "fast-forward" | "union-tracks" }): Promise<Base32Hash>

Delete (tombstones)

ts
ds.delete(anchors: bigint[], opts?: { reason?: string }): Promise<Base32Hash>
ds.tombstoneSet(): Promise<Set<bigint>>

Compaction

ts
ds.compact(): Promise<{
  manifest: Base32Hash;
  cellsExamined: number;
  cellsCompacted: number;
  fragmentsCollapsed: number;
}>

Read-online + idempotent — safe to run in production.

Inspection

ts
ds.currentManifest(): Base32Hash
ds.history(maxDepth?: number): Promise<HistoryEntry[]>
ds.listRefs(): Promise<string[]>
ds.count(): Promise<number>          // visible records (excludes tombstoned)

Space

Read-side view of a single Manifest. The browser-friendly path — no write methods, no mutable state, just resolved Tracks + sample iteration.

ts
Space.fromUri(uri: string, backend?: Backend): Promise<Space>

space.tracks(): ResolvedTrack[]
space.samples(): Promise<{ samples: Sample[]; anchorOrder: bigint[]; /* + lookups */ }>
space.history(maxDepth?: number): Promise<HistoryEntry[]>

A URI is the absolute HTTP URL of either refs/<name> or manifests/<hash>. The bucket name is inferred from the URL path.

Backend

Pluggable storage layer. Two built-in implementations:

ts
new S3Backend(baseUrl: string, opts?: {
  region?: string;
  accessKeyId?: string;
  secretAccessKey?: string;
  fetch?: typeof fetch;
})

new MemoryBackend()                  // in-process; for tests

Both implement the Backend interface (get, put, head, list, delete). Roll your own (e.g. Cloudflare KV, IndexedDB) by implementing it.

CBOR

ts
decodeCbor(bytes: Uint8Array): CborValue
encodeCbor(value: CborValue): Uint8Array

DreamDB's wire format. Handles the subset of canonical CBOR the protocol emits.

Multihash + URI helpers

ts
bytesToBase32(bytes: Uint8Array): string
base32ToBytes(s: string): Uint8Array
parseSpaceUri(uri: string): ParsedSpaceUri
modalityForField(field: FieldDef): string
scalarModalityFragment(valueType: ScalarValueType): string
const MULTIHASH_BYTES = 33

Query primitives (lower-level)

Exposed for code that needs to drop below the high-level Dataset.iterVector API.

ts
normalizeL2(v: Float32Array | number[]): Float32Array
normalizeL2F32(v: Float32Array): Float32Array
dotF32(a: Float32Array, b: Float32Array): number
dotL2Normalized(a: Float32Array, b: Float32Array): number

// Bucket / spatial index
peekBucketHeader(bytes): { recordCount, recordSize, spatialIndexHash, ... }
decodeBucketRecords(bytes, recordSize, codeFormat): BucketRecord[]
decodeBucketAnchorsOnly(bytes): bigint[]

// LSH cosine
deriveHyperplanesLshCosine(seed, dim, bits): Float32Array
projectionsLshCosine(qNorm, hyperplanes): Float32Array
multiProbeKeys(projection, bits, maxHamming): string[]
bitsFromProjections(p: Float32Array): Uint8Array
bitsToBase2(bits: Uint8Array, nbits: number): string

// Vector compressors
buildPqCosineDecoder(metadataBytes): { decodeRaw, adcScore }
buildRabitqCosineDecoder(metadataBytes): { rotateQuery, adcScore, decodeRaw }
deriveRotationMatrix(seed: Uint8Array, dim: number): Float32Array

// ChaCha20 keystream (used for deterministic seed material)
chacha20Stream(key32, nonce12, counter, length): Uint8Array

All preserve byte-for-byte conformance with the Rust reference implementation — don't refactor the bitwise ops.

Types

Re-exported from ./types:

MultihashBytes, Base32Hash, TimeAnchor, FieldKind, ScalarValueType, FieldDef, ImageFieldDef, AudioFieldDef, VideoFieldDef, EmbeddingFieldDef, ScalarFieldDef, ManifestTrack, Manifest, RegistryEntry, ResolvedTrack, VectorQueryOptions, VectorQueryResult, QueryStats, HistoryEntry, Batch, Backend, BackendGetOptions, BackendPutOptions, Sample, SampleValue, DatasetInfo, SnapshotInfo, WhereClause, SpatialDispatcher, VectorCompressorDecoder, BucketRecord, CborValue, plus SpaceOptions, ParsedSpaceUri, ChaCha20Stream.

Constants: SCALAR_ENCODING_MAP, ZERO_SPATIAL_KEY.