Ursa

Database / data-access layer for Constellation’s research stack.

Ursa is Constellation’s data-access layer for multimodal recordings: EEG, video, eye tracking, biometrics, questionnaires, keyboard/mouse/screen captures, and more. It answers two questions for everything else in the stack — what recordings exist? and give me the bytes — so that Virgo and Orion never touch raw object storage directly.

The data model is intentionally minimal: Participant → Recording (recording_hash) → Modality + Events + flexible metadata. No sessions, no trials, no stimuli.

Concretely, Ursa is three things stacked together:

• a Lance metadata catalog — a small set of tables describing every participant, recording, modality, and event;

• an R2 object store holding the actual modality bytes, addressed by absolute URI from each catalog row; and

• format-aware readers that turn those bytes into Python objects — Zarr for dense streams, Lance for sparse/variable events, MP4 for video.

How a read flows through the layers:

Researcher / Virgo / Orion
          │  query()                          (filter catalog rows)
          ▼
   ursa.DataInterface ──────────────────►  Lance catalog
          │   materialize()           ◄─────  participants · recordings ·
          │   stream()                          modalities · events · …
          │   download()
          ▼
   Format-aware readers ──read by storage_uri──►  R2 object store
   (Zarr · Lance · MP4)                            raw + processed bytes

The current implementation is a catalog and byte-access layer: the Lance catalog plus the resolved-input read verbs ursa.DataInterface.query(), .materialize(), .stream(), and .download() over raw and processed (Zarr/Lance/MP4) modalities, including a flat windowed iterator. Vector search, lifecycle GC, and Polaris cache sync are roadmap work.

Contents

• Setup
  • Prerequisites
  • Install Ursa
  • Profile selection
  • Sanity check
  • First query
  • Troubleshooting
• Quickstart
  • Install (commands 1–2)
  • Browse the catalog (commands 3–4)
  • Select and materialize (commands 5–6)
  • Verify on a real recording (commands 7–8)
  • Next steps
• Architecture
  • Layered overview
  • Current state versus roadmap
  • Storage tiers and R2 layout — src/ursa/layout.py
  • Lance catalog schemas — src/ursa/catalog/schemas.py
  • Store abstraction — src/ursa/store/
  • DataInterface read verbs — src/ursa/data_interface.py:216
  • Register / write side — src/ursa/register/ and DataInterface.register_*
  • Temporal classes and the temporaldata extension — src/ursa/temporal.py
  • Active-catalog pointer — src/ursa/catalog/_pointer.py
  • Time model — src/ursa/time.py
  • Cross-repo contracts
  • Lifecycle (Planned)
• Concepts
  • The data model
  • Two stores: cold raw and hot processed
  • The catalog
  • The recording_hash
  • The active-catalog pointer
  • Nanosecond time
• Tutorials
  • Downloading session bytes to disk
  • Manually ingesting SSD-copied sessions
  • Streaming and time slicing
  • Runnable end-to-end notebook
  • Planned
• API Reference
  • API Reference
  • ursa
  • ursa.data_interface
  • ursa._query_types
  • ursa._temporaldata_compat
  • ursa.backends
  • ursa.backends._zarr
  • ursa.backends.lance
  • ursa.backends.video
  • ursa.catalog
  • ursa.catalog.catalog
  • ursa.catalog.exceptions
  • ursa.catalog.schemas
  • ursa.filters
  • ursa.layout
  • ursa.participant
  • ursa.raw
  • ursa.recovery
  • ursa.recovery.first_epoch
  • ursa.recovery.timing
  • ursa.register
  • ursa.register.orchestrator
  • ursa.register.payload
  • ursa.status
  • ursa.store
  • ursa.store.base
  • ursa.store.config
  • ursa.store.factory
  • ursa.store.uri
  • ursa.temporal
  • ursa.time
• Admin runbooks
  • Catalog rebuild — rebuild the production catalog from R2 raw data
  • Catalog maintenance — data-only backfills & fixes
  • Offline ingest — manual NAS-side ingest when a rig has been offline
  • When to use this section

Where this fits

Ursa is one of three packages in Constellation’s research stack:

• Ursa (this site) — database / data layer

• Virgo — DAG-based preprocessing

• Orion — research / training / benchmarking

Full architecture: Research Stack Architecture (Notion).

Status

Current shipped scope is raw catalog + raw byte access only. Implementation is tracked in the Linear Ursa project.