Audio rendering¶

TriageSim can turn a finished dialogue into multi-speaker speech using XTTS-v2, giving nurse and patient distinct cloned voices. This closes the loop from structured EHR to spoken data.

Audio is entirely optional — the simulation and metrics work without it.

Installation¶

pip install "triagesim[audio]"

This pulls in torch and TTS, which are large. Install the extra only if you need speech output.

Import path¶

triagesim/audio/__init__.py is empty, so SpeechRenderer must be imported from the module itself:

from triagesim.audio.renderer import SpeechRenderer

SpeechRenderer¶

renderer = SpeechRenderer()

Construction downloads the model on first use and loads it into memory, so build one renderer and reuse it. Instantiating a renderer per utterance will reload the model every time.

Device selection¶

The device is chosen automatically at construction, in order of preference:

1. cuda — if torch.cuda.is_available()
2. mps — Apple Silicon; also sets torch.set_float32_matmul_precision("high")
3. cpu — the fallback

renderer = SpeechRenderer()
print(renderer.device)  # 'cuda', 'mps', or 'cpu'

GPU acceleration is only enabled for CUDA. On MPS the model runs but is not passed the gpu=True flag, and on CPU expect synthesis to be several times slower than real time.

render¶

path = renderer.render(
    text="Can you tell me what happened this morning?",
    speaker_wav="voices/nurse.wav",
    out_path="audio/turn_000_nurse.wav",
    language="en",
)

All arguments are keyword-only.

Returns the output Path. Parent directories are created automatically, so you do not need to mkdir first.

Reference clips should be clean speech of roughly 6–20 seconds. Background noise in the reference is reproduced in the clone.

Rendering a full dialogue¶

Walk the artifact's history, giving each speaker its own reference voice:

from pathlib import Path

from triagesim.audio.renderer import SpeechRenderer

VOICES = {
    "nurse": "voices/nurse.wav",
    "patient": "voices/patient.wav",
}

renderer = SpeechRenderer()
out_dir = Path("audio") / artifact["run_id"]

rendered = []
for i, entry in enumerate(artifact["history"]):
    actor = entry["actor"]
    if actor not in VOICES:
        continue  # skip system events: released vitals, triage_end

    rendered.append(
        renderer.render(
            text=entry["utterance"],
            speaker_wav=VOICES[actor],
            out_path=out_dir / f"{i:03d}_{actor}.wav",
            language="en",
        )
    )

print(f"rendered {len(rendered)} utterances to {out_dir}")

Skipping entries whose actor is not in VOICES filters out system events — released vitals and the triage_end marker have no utterance key and would raise KeyError.

Numbering files by their index in history preserves turn order, which matters when concatenating them into a single conversation.

Varying voices across runs¶

Persona attributes such as gender and ethnicity are a natural key into a library of reference clips:

def voice_for(persona) -> str:
    return f"voices/{persona.gender}_{persona.ethnicity}.wav".lower().replace(" ", "_")

This keeps the acoustic identity consistent with the persona that generated the text, which is the point of persona-conditioned generation in the first place.

Preparing text for synthesis¶

triagesim.utils.text_control provides enforce_response_budget(text, response_length), which trims an utterance to a target length band. It is a text-shaping helper rather than an audio one, but it is useful before synthesis when you need utterance durations to stay within a budget.

Licensing and consent¶