Run artifacts¶
TriageRunner.run() returns a single dictionary containing everything that
happened. It is plain JSON-serialisable data — no live objects — so it can be
written to disk, shipped elsewhere, and analysed long after the run finished.
Top-level keys¶
| Key | Type | Contents |
|---|---|---|
run_id | str | UUID4 identifying the run. Also the namespace used in the state store. |
ground_truth | dict | The vignette the run was built from, echoed back unchanged. |
state | dict | Final simulation state: ground_truth, turn, done. |
history | list[dict] | The transcript: nurse and patient utterances plus system events. |
trace | list[dict] | Per-action nurse cognition record. |
belief | dict | Flattened final belief state. |
red_flags | list[str] | Red flags accepted by the environment, in logging order. |
artifact = runner.run()
print(artifact.keys())
# dict_keys(['run_id', 'ground_truth', 'state', 'history', 'trace', 'belief', 'red_flags'])
state¶
{
"ground_truth": { "chiefcomplaint": "Syncope", "acuity": 2, "pain": 7, "vitals": {} },
"turn": 12,
"done": true
}
turn counts patient turns. Comparing it against max_turns tells you
whether the run ended naturally or was cut short: if turn is well below
max_turns and done is true, the nurse chose to end — or the step safety
cap tripped.
history¶
The transcript, in order. Three entry shapes appear, distinguished by actor.
Nurse lines carry the triage level believed at the moment of speaking.
Rendering it as a readable transcript:
for h in artifact["history"]:
if h["actor"] == "nurse":
print(f"Nurse: {h['utterance']}")
elif h["actor"] == "patient":
print(f"Patient: {h['utterance']}")
elif h.get("event") == "vital":
print(f"[Vital] {h['name']} = {h['value']}")
elif h.get("event") == "triage_end":
print("[Triage ended]")
trace¶
One entry per nurse action — including micro-turns that produced no speech. This is the cognition record, and it is where most analysis happens.
{
"turn": 1,
"actor": "nurse",
"action": {
"type": "check_vital",
"vital": "heartrate"
},
"triage": 3,
"confidence": "low",
"explanation": "Syncope with no witnessed head strike. Need heart rate to assess for arrhythmia before assigning acuity.",
"red_flags_proposed": ["syncope"]
}
| Field | Meaning |
|---|---|
turn | Patient-turn index the action occurred within. |
actor | Always "nurse". |
action | The executable action, as a dict. type is one of utterance, check_vital, log_red_flag, end; remaining keys depend on the type. |
triage | Triage level at this step. |
confidence | low, medium, or high. |
explanation | The nurse's stated reasoning at this step. |
red_flags_proposed | Flags the model proposed — before environment deduplication. |
Proposed versus accepted flags
red_flags_proposed is what the model claimed; the top-level red_flags
list is what the environment actually accepted after normalising case and
whitespace, dropping repeats, and capping at five new flags per turn. The
two will usually differ, and the gap is itself informative.
Because triage, confidence and explanation are recorded at every step,
the trace shows the decision trajectory rather than just its endpoint:
for step in artifact["trace"]:
action = step["action"]
print(f"turn {step['turn']:>2} {action['type']:<13} "
f"triage={step['triage']} ({step['confidence']})")
turn 0 utterance triage=3 (low)
turn 1 check_vital triage=3 (low)
turn 1 log_red_flag triage=2 (medium)
turn 1 utterance triage=2 (medium)
turn 2 end triage=2 (high)
belief¶
The final belief state, flattened from the provenance graph into a consumer-friendly view. Each slot maps to a list of belief items, because the graph deliberately preserves competing values rather than resolving them.
{
"chief_complaint": [
{"value": "syncope", "source": "patient", "turn": 0, "certainty": "explicit"}
],
"associated_symptoms": [
{"value": "dizziness", "source": "patient", "turn": 1, "certainty": "explicit"},
{"value": "palpitations", "source": "patient", "turn": 2, "certainty": "inferred"}
],
"vitals_known": ["heartrate", "o2sat"],
"red_flags_logged": ["syncope", "tachycardia"]
}
| Item field | Meaning |
|---|---|
value | The extracted value. |
source | patient, nurse, system_vital, or system_other. |
turn | Turn the value was acquired on. |
certainty | explicit, inferred, or suspected. |
Two keys are always present and are not lists of belief items:
vitals_known— sorted list of released vital namesred_flags_logged— sorted list of accepted red flags
Slot names are whatever the belief detectors produced, with two renames applied
on the way out: associated_symptom becomes associated_symptoms, and
red_flag becomes red_flags. Treat every other slot key as optional and use
.get().
The observation view is lossy
belief drops the evidence graph — the quotes supporting each value and the
slot-to-evidence edges. That provenance exists in the environment's
BeliefGraph but is not included in the artifact.
Saving and reloading¶
import json
from pathlib import Path
out = Path("runs") / f"{artifact['run_id']}.json"
out.parent.mkdir(parents=True, exist_ok=True)
out.write_text(json.dumps(artifact, indent=2))
Everything is JSON-native, so no custom encoder is needed. Reloading:
A saved artifact is all the metrics require — you can score a batch of runs offline without touching a model:
import json
from pathlib import Path
from triagesim.utils import compute_all_metrics
for path in sorted(Path("runs").glob("*.json")):
artifact = json.loads(path.read_text())
metrics = compute_all_metrics(
trace=artifact["trace"],
belief=artifact["belief"],
ground_truth=artifact["ground_truth"],
)
print(path.stem, metrics["triage"]["correct"])
Storing ground_truth inside the artifact is what makes this work: a run file
is self-describing and needs no external key to be scored.
Next: Evaluation metrics.