DreamDB

Spec 0027 — Progressive Geometry Items

Status: Normative v1 format for #316, implementing ADR 0004. Geometry is internal to one outer Timeline Item, never an embedding SpatialBucket.

1. Coordinates and records

The namespaced modalities ai.dreamlake.spatial.geometry.point, ai.dreamlake.spatial.geometry.splat and ai.dreamlake.spatial.geometry.mesh have Track kind event and Object kind geometry-item, declared in the modality registry. Unknown Object kinds must refuse before reads, transformations or GC traversal. Schema family and the descriptor family must match. Existing kinds/bytes are unchanged.

Coordinates are a right-handed XYZ u16 integer grid. A descriptor carries origin_f64le and step_f64le, each bstr(24), three finite binary64 values; steps must be positive. Physical position on axis i is origin[i]+q[i]*step[i]. Both grid endpoints must remain finite under that binary64 evaluation. units is a nonempty UTF-8 label of at most 256 bytes, compared literally. The writer API accepts already quantized coordinates, not floating-point positions with an implicit rounding policy. Producers converting physical positions use binary64 subtraction then division, round-to-nearest-ties-even, and reject results outside [0,65535]; no silent clamp. Quantization is lossy and the physical mapping is not a claim to reconstruct original coordinates.

Point records are exactly 10 bytes: x/y/z u16le at 0/2/4, then RGBA u8 at 6..10. Splat records are exactly 46 bytes: x/y/z u16le, six f32le covariance entries xx,xy,xz,yy,yz,zz at 6..30, then linear RGBA f32le at 30..46. All floats are finite. RGBA is in [0,1]. Covariance is positive definite: xx>0, xxyy-xyxy>0 and the symmetric determinant>0, evaluated after binary64 widening with scalar operations in the printed order, no FMA or reassociation. Covariance units are physical units squared. No encoder/retraining or renderer is specified; these are already prepared scene records.

Morton48 interleaves all 16 coordinate bits: bit 3i is x[i], 3i+1 is y[i], 3i+2 is z[i]. A descriptor's depth is in [0,16]. A cell key is the high 3*depth Morton bits (depth=0 has only key 0). Its integer AABB is obtained by deinterleaving that prefix and filling the remaining coordinate bits; bounds are inclusive. Spatial selection uses integer AABB intersection, inclusive on both ends, not floating-point camera arithmetic.

Inside each cell records are sorted by reversal of exactly the 48 Morton bits, ascending, then by all record bytes lexicographically. Byte-identical records may repeat; their relative position cannot alter bytes. The identifier is morton_rev48. Prefix detail means a deterministic progressively longer sample of this ordering, not a certified screen-space or geometric-error bound. No standard importance order is defined. Every cell is nonempty and at most 1 MiB of records. Oversize cells refuse; the producer may choose a finer depth or smaller scene Item. Coincident records are not silently discarded.

2. GeometryData and packing

geometry-data/<multihash> holds raw immutable payload bytes. For point/splat data the canonical writer concatenates complete cells, by increasing cell key, into packs of at most 4 MiB, starting a new pack before a cell would overflow. No padding or header bytes are inserted. Each cell entry is exactly [key:u64, count:u64, object:bstr(33), offset:u64, size:u64]; size=count*stride with checked arithmetic. References sharing an Object must be disjoint and object extents are checked against authenticated length before exposing payload. Neither a tail nor an index can manufacture bytes beyond EOF.

The format also permits a cell to occupy its own Object; this changes addresses, not decoded records. The reference choice is packed cells. The protocol builder measured 512 cells × 128 point records: 655,360 payload bytes, 1 packed data Object instead of 512 cell Objects, 3 hierarchy pages, and zero padding bytes. All records passed membership/order validation after packing. Accordingly the outboard count also falls from 512 to 1. This is an Object-count/layout measurement, not a latency or compression benchmark (geometry_format test).

Every new GeometryData Object publishes its standard Bao outboard under 0002 §6.5.4 before the Manifest. Ranged readers verify first and expose bytes only afterwards. No geometry-specific unauthenticated fallback exists.

3. CLOSED descriptors and hierarchy

geometry-item/<hash> is canonical CBOR, at most 1 MiB, exactly these keys: v (uint 1), family (point/splat/mesh), origin_f64le, step_f64le, units, depth (uint), index (map). The dedicated GeometryTrack is a CLOSED map of exactly timeline (hash), modality (text), coverage ([lo,hi]) and items (ordered unique [anchor, descriptor_hash, descriptor_size] triples). It is at most 1 MiB; v1 refuses outer Track overflow rather than inventing a page format. Nonempty coverage is [first_anchor,last_anchor+1), empty coverage is [0,1); anchors must be less than u64::MAX. The descriptor size excludes its transitive closure. An outer anchor selects one scene, never one point.

For point/splat, index is one of these CLOSED forms:

  • {form:"cells-v1", cells:[entry,...]}: strictly increasing unique keys, at most 256 entries (an empty scene has this form with []).
  • {form:"paged-cells-v1", root:bstr(33)}: a GeometryPage tree.

geometry-page/<hash> is canonical CBOR at most 1 MiB. A page is exactly {form:"leaf-v1",cells:[entry,...]} (1..256 cells), or {form:"branch-v1",children:[[first_key,last_key,page_hash],...]} (1..256 children). Child key ranges are inclusive, ordered and disjoint; first<=last. The fetched child's actual extrema must equal its declared range. Tree depth is at most 8; cycles, repeated pages and inconsistent kinds/counts refuse. The canonical builder groups 256 consecutive entries per leaf and then 256 consecutive nodes per level, until one root remains; it never emits a single-entry inline form for an index that exceeds 256 entries. Metadata may be fetched in full to select cells; v1 makes no sublinear hierarchy-query claim.

All referenced pages and data are content-address verified. A record read must also verify the selected records' positions belong to the declared cell, record shape and within-prefix ordering. A complete physical audit additionally checks every cell and all extents; a partial read does not claim to have audited unseen records. Unknown forms, fields, duplicate keys, unsupported versions and invalid types are errors, not empty geometry.

4. Mesh levels

For mesh, depth is 0 and index is exactly {form:"mesh-lods-v1",levels:[...]}. There are 1..256 levels, coarsest to finest. Each level is exactly [error_grid:u64, vertex_count:u32, triangle_count:u32, object:bstr(33), size:u64]. Errors are nonincreasing and expressed in integer grid steps; the producer asserts this error, the codec verifies ordering but does not prove the claimed approximation error against a source surface. Both counts are positive. Data is vertex_count XYZ u16le triples followed by triangle_count triples of u32le vertex indices. Size=vertices6+triangles12, checked, at most 1 GiB. Every triangle index is below vertex_count. Each level is independently decodable, immutable and range-authenticated before exposure. It is read as a whole level; arbitrary mesh prefixes are never claimed to be valid meshes. There is no mesh simplifier, progressive-mesh stream or cluster DAG.

5. Reader sessions and publication

The Dataset/SDK opens an Item as a pinned geometry reader. It exposes cell metadata, integer-AABB selection and cell prefix extension by an output-byte budget. Only whole records fit; count is min(total,floor(budget/stride)). A request for a shorter/equal prefix returns no new bytes; extension returns only the newly exposed suffix. A session caches authenticated 1 KiB blocks per Object, including range-alignment overfetch. An extension does not re-request a cached payload block. Metadata/proof reads are not payload refetches. Budget is exposed record bytes, not a promise that network bytes equal the budget: Bao alignment/proofs can add overhead. The session retains its cache until closed; v1 does not promise bounded memory for an indefinitely growing session.

Point/splat publication validates and orders all cells, constructs data packs, pages, descriptor and the replacement outer Track, then publishes one child Manifest/Ref CAS. An anchor already in this field refuses rather than silently replacing a scene. Failure leaves the Dataset handle unchanged; unreferenced Objects after a racing CAS are ordinary GC candidates. This is an in-memory reference builder, not an out-of-core ingest claim. Geometry writes on keyed Datasets refuse until the scene transaction can join the entity-key contract.

GC traverses every retained root's Track, descriptor, hierarchy, data/mesh levels and Bao outboards. Unsupported or corrupt geometry aborts before sweep. Current embedding/scalar compaction carries other geometry Tracks unchanged; explicit geometry compaction (or a Dataset with no compactable fields) refuses without publishing. Any future cell rewrite must preserve/re-establish cell membership, layout and order or refuse. Python and WASM use the same codec and session semantics; camera/refinement policy stays in the caller. No query server or Backend API change is required.