DreamDB

Quickstart

End-to-end: create a dataset, append a few records with CLIP-style embeddings, query the top-K, then read one back.

Read a live public dataset

The fastest way to see the SDK working — no ingest required. There's a 10K ImageNet dataset with CLIP-512 embeddings at a public-read S3 ref:

ts
import { Space } from "@dreamlake/dreamdb";

const space = await Space.fromUri(
  "https://vortex-test-20260518.s3.us-east-1.amazonaws.com/refs/imagenet-10k-clip",
);

console.log(space.tracks().map(t => `${t.key} (${t.modality})`));
// → ["image (image.jpeg)", "embedding (embedding.f32.dim=512.bucketed)", "label (...)"]

const { samples } = await space.samples();
console.log(`${samples.length} samples, anchor[0] = ${samples[0].anchor}`);

No credentials needed — the bucket policy grants anonymous read.

Create your own dataset

ts
import { Schema, Dataset, S3Backend } from "@dreamlake/dreamdb";

const backend = new S3Backend("http://localhost:9000/demo");

// 1. Declare the shape.
const schema = new Schema()
  .addImage("image", { mime: "jpeg" })
  .addEmbedding("embedding", { dim: 512, algorithm: "dreamdb.ivf-cosine", rerank: true })
  .addScalarCategorical("label");

// 2. Create the dataset.
const ds = await Dataset.create("my-dataset", schema, backend);

// 3. Append records.
await ds.appendMany([
  { image: jpegBytesA, embedding: vecA, label: "cat" },
  { image: jpegBytesB, embedding: vecB, label: "dog" },
]);

// 4. Query.
for (const batch of await ds.iterVector({ field: "embedding", query: vecQ, topK: 5 })) {
  console.log(batch._time_anchors, batch.label);
}

That's it — 1 backend, 1 schema, 1 dataset, 1 query.

Re-open later

ts
const ds = await Dataset.open("my-dataset", { backend });
// Schema is recovered from the Manifest. No need to re-declare.

Time-travel

Every write produces an immutable Manifest. Walk the DAG:

ts
const history = await ds.history(50);
for (const entry of history) {
  console.log(entry.manifest, new Date(Number(entry.ts) / 1e6), entry.writer);
}

Open any prior Manifest as a read-only Space:

ts
const past = await Space.fromUri(
  `${backend.baseUrl}/manifests/${history[10].manifest}`,
);

Where to go next