Text-to-speech (TTS)¶

Aurum turns UTF-8 text → mono WAV on-device by default (no API key).

aurum tts "Hello from aurum" --output-file /tmp/a.wav
aurum tts --input-file prompt.txt -O /tmp/a.wav --emit-json
aurum tts models
aurum tts voices

TTS is a subcommand only — it does not change STT defaults (aurum <file>, aurum models, aurum cleanup).

Quality evidence (JOE-2217)¶

• Objective pack + evaluator: aurum_core::eval (tts_production_pack, score_tts_pcm)
• Fixture JSON: evals/observatory/tts_eval_pack.v1.json (≥60 utterances)
• Listening protocol: docs/operations/tts-listening-protocol.md
• Default decision (retain Kitten): evals/observatory/tts-default-decision.md
• Evidence version: 0.0.22-tts-listening-v1

Support-tier promotion requires objective pass and ≥3 blinded listeners (medians: intelligibility ≥4, pronunciation/joins ≥3.5). Changing the default is a separate product decision, not an automatic score flip.

Result contract¶

Local and remote TTS share one in-memory result: mono i16 PCM, accurate sample_rate_hz, and duration_ms derived from final PCM length. Honesty JSON (--emit-json / TtsMetaDto) never includes PCM and labels execution as backend_kind: "local" or "remote" (JOE-1937). Schema version stays at 1.

Remote providers convert wire formats (PCM / WAV / MP3) through a bounded pipeline — see Remote audio formats.

Remote TTS (opt-in)¶

Defaults remain local. Remote requires deliberate --provider and the matching API key. When only --provider is set, model and voice use that provider’s reviewed defaults (not Kitten / Luna).

# OpenRouter (default model: hexgrad/kokoro-82m)
export OPENROUTER_API_KEY=sk-or-...
aurum tts "Hello" --provider openrouter -O /tmp/or.wav --emit-json

# OpenAI
export OPENAI_API_KEY=sk-...
aurum tts "Hello" --provider openai --model tts-1 --voice alloy -O /tmp/oai.wav

# ElevenLabs (voice_id required; no local alias remap)
export ELEVENLABS_API_KEY=...
aurum tts "Hello" --provider elevenlabs \
  --model eleven_multilingual_v2 --voice 21m00Tcm4TlvDq8ikWAM -O /tmp/el.wav

# xAI (experimental; voices eve|ara|leo|rex|sal)
export XAI_API_KEY=...
aurum tts "Hello" --provider xai --model xai-tts --voice eve -O /tmp/x.wav

--emit-json reports backend_kind: "remote" and the selected provider. OpenRouter TTS is experimental until protected live smoke is retained; see provider matrix.

Quick start¶

1. Build the CLI (cargo build -p aurum-stt --release or install from source).
2. First synthesis downloads the pinned English voice pack (~26 MB) into the Aurum cache (…/aurum/tts/).
3. Subsequent runs are offline when the pack is cached.

aurum tts "Hello from aurum" --output-file /tmp/hello.wav --emit-json
file /tmp/hello.wav   # RIFF WAVE, mono PCM

Agent / command-provider template¶

aurum tts --input-file {input_path} --voice {voice} --language {language} \
  -o wav --output-file {output_path} --emit-json

Voice demos (local only — not in git)¶

Regenerate a short WAV per catalogue voice under target/tts-demos/ (gitignored via /target):

cargo build -p aurum-stt --release
./scripts/generate_tts_demos.sh
./scripts/generate_tts_demos.sh --play          # macOS: afplay each clip
./scripts/generate_tts_demos.sh --out ~/Downloads/aurum-tts-voices
TEXT='Custom line.' ./scripts/generate_tts_demos.sh

Do not commit the generated WAVs; they are rebuildable artifacts of the pinned model pack.

Opt-in Kokoro-82M (JOE-1618)¶

Higher-quality opt-in model (not the default). First use downloads ~120 MB of pinned int8 ONNX + voices (Apache-2.0). Kitten remains the default for size/latency.

aurum tts "Hello from Kokoro" --model kokoro-82m-int8 --voice Heart -O /tmp/k.wav
aurum tts models   # lists both kitten-nano-int8 and kokoro-82m-int8
aurum tts voices   # Heart, Nicole, … are model-scoped to kokoro-82m-int8

Adapter id: kokoro-onnx-v0. See ADR-001.

Integration test¶

cargo test -p aurum-core --test tts_synth -- --ignored --nocapture
# optional Kokoro smoke with pre-seeded cache:
AURUM_KOKORO_INTEGRATION=1 AURUM_TTS_CACHE=~/.cache/aurum \
  cargo test -p aurum-core --lib kokoro_real_synth -- --ignored

The full synth test is #[ignore] by default (may download the voice pack). Empty-text validation always runs in cargo test.

CLI¶

Nested catalogue commands (do not collide with STT aurum models):

aurum tts models
aurum tts voices

Exit codes¶

Same taxonomy as STT: 0 ok, 2 user, 3 environment, 4 provider, 1 internal.

I/O rules¶

• Shared secure output transaction (exclusive same-directory temp → flush/sync → atomic publish).
• Refuse overwrite of an existing non-empty file without --force (NoClobber); --force uses Replace.
• Destination symbolic links are rejected by default.
• Empty / whitespace-only text → exit 2.
• Max characters default 5000 (config [tts].max_chars); longer input is a user error (complete-or-error — no silent truncation).
• Long text within max_chars is chunked by model phoneme capacity (sentence, then word boundaries) with a short silence between chunks; an unsplittable unit that exceeds the model limit fails with a precise error.
• Sample rate is always the adapter native rate (24 kHz for KittenTTS). Arbitrary rate overrides are rejected; real resampling is not implemented.

Honesty JSON (--emit-json)¶

Stdout is only JSON metadata (no audio bytes). Model/voice IDs are canonical catalogue IDs actually used:

{
  "backend_kind": "local",
  "provider": "local",
  "model": "kitten-nano-int8",
  "voice": "Luna",
  "language": "en",
  "output_path": "/abs/path/out.wav",
  "format": "wav",
  "sample_rate_hz": 24000,
  "channels": 1,
  "duration_ms": 1234,
  "text_chars": 42,
  "text_truncated": false,
  "chunk_count": 1,
  "synthesized_chars": 42
}

Configuration¶

[tts]
provider = "local"
model = "kitten-nano-int8"
voice = "Luna"
language = "en"
max_chars = 5000
timeout_ms = 120000

Environment overrides (no secrets required):

Offline / cache¶

• Pack cache: <aurum-cache>/tts/<model-id>/ (onnx + voices.npz + config.json).
• Downloads use a cross-process advisory lock and pinned SHA-256 (fail-closed on mismatch).
• Warm cache + --local-only (or offline network): synthesis uses only local files.
• Cold cache offline: fails closed with a user/provider error (no silent stub audio).

Voices¶

Default pack ships eight English aliases: Bella, Jasper, Luna (default), Bruno, Rosie, Hugo, Kiki, Leo.

License matrix¶

Not linked into the default binary: eSpeak NG (GPL), Piper phonemize (espeak-backed), ffmpeg (not used for TTS WAV).

Pinned default pack¶

Source: https://huggingface.co/KittenML/kitten-tts-nano-0.8-int8

Engine notes¶

• Why KittenTTS (not Piper default): Piper-class pipelines commonly need eSpeak NG for phonemes (GPL). KittenTTS ONNX + MIT G2P keeps the MIT binary story intact while still delivering real neural speech.
• Sample rate: 24 kHz mono PCM 16-bit WAV (in-process via hound). Duration is derived from final PCM length after chunk concat and trailing-silence validation.
• Trailing silence is trimmed with a bounded energy detector (never empties short valid clips). Fixed sample chopping is not used.
• Voices are model-scoped: a voice only works with its catalogue model pack.
• Peak/loudness guard on PCM before write (no NaN).
• Cargo feature: aurum-core feature tts (default on for the CLI).

Binary size / dependencies¶

Enabling TTS (the CLI default) pulls in ONNX Runtime via the ort crate (download-binaries). That increases build artifacts and release binary size versus an STT-only build. Library consumers who only need STT can disable it:

aurum-core = { version = "0.0.20", default-features = false }
# or: default-features = false, features = []  — STT only

The CLI package keeps tts on so aurum tts works out of the box.

Timeouts¶

--timeout / [tts].timeout_ms bounds how long Aurum waits for a synthesis worker (spawn_blocking + tokio::time::timeout).

• On expiry the CLI returns a provider error and sets a cancel flag when one was provided.
• This is best-effort: ONNX Runtime work already running inside the blocking thread is not hard-killed; CPU may continue briefly until the worker returns.
• Do not assume wall-clock cancel frees the core the instant the timeout fires (same class of limitation as many embed runtimes).

Library hosts that need deterministic teardown should drop the LocalTtsProvider (and call LocalTtsProvider::clear_sessions when done) after cancelling in-flight work at the application layer.

Limits (current 0.0.x)¶

• Local default voice pack is English-focused (additional languages out of scope for Kitten default).
• Output container is WAV only (remote wire formats are normalized to mono PCM then written as WAV).
• No streaming synthesis, mic playback, or voice cloning.
• Remote TTS is opt-in (OpenRouter / OpenAI / ElevenLabs / xAI); see Providers.
• C ABI exposes local TTS jobs only — remote is not on the FFI.

Model platform & safe BYOM (JOE-1576)¶

Aurum does not support “load any ONNX file.” Every pack selects a known adapter that defines artifacts, tensors, voices, languages, limits, and output policy.

Adapters¶

aurum tts adapters

Trust modes¶

Pack layout¶

A model pack is a directory containing aurum-tts-manifest.json plus the adapter-required artifacts (not a bare .onnx path):

aurum tts inspect /path/to/pack
aurum tts verify /path/to/pack
aurum tts add /path/to/pack --adapter fake-sine-v1 --model-id my-tone --trust verified
# dry-run by default; pass --write-manifest to materialize the JSON

Local pack override¶

aurum tts "Hello" -O /tmp/a.wav --pack-dir /path/to/pack
# or config: [tts] pack_dir = "..." / allow_unverified = true

Local packs use a separate cache identity from built-ins and never silently override catalogue model IDs.

Custom catalogue¶

[[tts.custom_models]]
id = "my-tone"
adapter = "fake-sine-v1"
pack_dir = "/path/to/pack"
trust = "verified"
license = "CC0"

Custom IDs cannot collide with shipped built-ins. Custom models are never the default unless you set [tts].model yourself.

Provenance in honesty JSON¶

--emit-json may include adapter, trust, and provenance (builtin | local_pack | custom).