Skip to content

Storage Providers

Supported providers

ASTX.Storage.providers();
// ['gcs', 's3', 'dbfs']

Capability matrix

Provider list head read write delete exists copy move signed_url multipart_write
gcs Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes
s3 Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes
dbfs Yes Yes Yes Yes Yes Yes Yes Yes No Yes

Use runtime checks:

ASTX.Storage.capabilities('gcs');

GCS auth modes

  • oauth: uses request.auth.oauthToken or Apps Script ScriptApp.getOAuthToken().
  • service_account: uses request.auth.serviceAccountJson or GCS_SERVICE_ACCOUNT_JSON.
  • auto (default): tries OAuth first, then service account.

S3 auth

Requires:

  • accessKeyId (S3_ACCESS_KEY_ID)
  • secretAccessKey (S3_SECRET_ACCESS_KEY)
  • region (S3_REGION)

Optional:

  • sessionToken (S3_SESSION_TOKEN)
  • endpoint (S3_ENDPOINT) for S3-compatible APIs

Requests are signed with SigV4.

Signed URLs and multipart uploads both use SigV4 primitives.

DBFS auth

Requires:

  • host (DATABRICKS_HOST)
  • token (DATABRICKS_TOKEN)

Uses Databricks DBFS REST API (/api/2.0/dbfs/*).

DBFS does not support pre-signed URL generation in this release (signed_url returns AstStorageCapabilityError).

URI examples

  • gcs://my-bucket/folder/file.json
  • s3://analytics-bucket/exports/run-001.csv
  • dbfs:/mnt/project/data/events.parquet