Manifest Sources & Auth¶

Nova can load manifest.json from local files or remote storage. Configure via:

• DBT_MANIFEST_PATH (local path)
• DBT_NOVA_MANIFEST_URI (remote URI, optional)

Supported URI schemes:

• file:// — local file
• http:// / https:// — remote URL
• dbfs:// — Databricks DBFS
• s3:// — Amazon S3
• gs:// — Google Cloud Storage

For compatibility, legacy dbfs:/... manifests are also accepted and normalized to dbfs://....

Provider Notes¶

Manifest sources are resolved via a provider registry keyed by URI scheme. Adding new storage backends means registering a new provider for its scheme.

Auth Matrix (Quick Reference)¶

Local File (default)¶

No auth required.

export DBT_MANIFEST_PATH=/path/to/manifest.json

HTTP / HTTPS¶

Nova does not send custom auth headers. Use a public URL or a presigned URL. https:// is allowed by default. http:// is blocked unless explicitly enabled.

export DBT_NOVA_MANIFEST_URI=https://example.com/manifest.json

To allow insecure http:// manifests:

export DBT_NOVA_MANIFEST_ALLOW_HTTP=true

Databricks DBFS (dbfs://)¶

Requires Databricks API credentials:

export DBT_NOVA_MANIFEST_URI=dbfs:///mnt/analytics/manifest.json
export DATABRICKS_HOST=https://<workspace>
export DATABRICKS_ACCESS_TOKEN=...

Legacy form also works:

export DBT_NOVA_MANIFEST_URI=dbfs:/mnt/analytics/manifest.json

Amazon S3 (s3://)¶

Default mode is HTTPS (public or presigned URLs).

export DBT_NOVA_MANIFEST_URI=s3://my-bucket/path/manifest.json

Optional endpoint override:

export DBT_NOVA_S3_ENDPOINT=s3.amazonaws.com

S3 SDK mode (credentialed)¶

export DBT_NOVA_S3_MODE=sdk
export AWS_REGION=eu-west-1
export AWS_ACCESS_KEY_ID=...
export AWS_SECRET_ACCESS_KEY=...

Nova uses the AWS default credential chain, so AWS_PROFILE and session tokens also work if configured.

Google Cloud Storage (gs://)¶

Default mode is HTTPS (public or presigned URLs).

export DBT_NOVA_MANIFEST_URI=gs://my-bucket/path/manifest.json

Optional endpoint override:

export DBT_NOVA_GCS_ENDPOINT=storage.googleapis.com

GCS SDK mode (credentialed)¶

export DBT_NOVA_GCS_MODE=sdk
export GOOGLE_APPLICATION_CREDENTIALS=/path/to/service-account.json

Application Default Credentials (ADC) are also supported (e.g. gcloud auth application-default login).

The same GOOGLE_APPLICATION_CREDENTIALS / ADC flow is also supported by the BigQuery SQL provider (DBT_NOVA_SQL_PROVIDER=bigquery).

Caching & Refresh¶

Remote manifests are cached under:

<storage_root>/manifests/

Configure:

• DBT_NOVA_MANIFEST_CACHE_DIR — override cache location
• DBT_NOVA_MANIFEST_MAX_BYTES — cap remote manifest size (0 = unlimited)
• DBT_NOVA_MANIFEST_REFRESH_SECS — refresh interval (0 = never refresh)

Reloading without restart¶

If you need to switch manifest sources at runtime (e.g., engineer local → analyst DBFS), use the reload_manifest tool:

{"name":"reload_manifest","arguments":{"manifest_uri":"dbfs:///mnt/analytics/manifest.json","refresh_secs":300}}

reload_manifest rebuilds indexes in the background and swaps atomically when ready.

Versioned Indexes & Atomic Swaps¶

Nova keeps versioned index directories per manifest content hash:

<storage_root>/instances/<instance_id>/versions/<hash>/

The active version is tracked in:

<storage_root>/instances/<instance_id>/manifest.current.json

When DBT_NOVA_MANIFEST_REFRESH_SECS is enabled, Nova:

1. Resolves the manifest source (local or cached remote).
2. Computes the content hash.
3. Builds new indexes in a new version directory in the background.
4. Atomically swaps the active version once ready (no downtime).
5. Keeps the previous version available for in-flight requests.

Multi‑repo / Multi‑manifest Workflow¶

Nova isolates indexes by manifest path or URI. If you work across multiple repos or multiple manifests:

• Use distinct DBT_MANIFEST_PATH / DBT_NOVA_MANIFEST_URI values per process.
• Override DBT_NOVA_STORAGE_INSTANCE_ID if you need explicit isolation.
• For remote manifests, the URI itself becomes the instance seed, so different URIs produce different instance directories automatically.