Examples
Worked examples with code, one for each reader the record serves: the teams running agents, the evaluators and auditors who check them, and government programs.
Each example follows one situation from start to finish: what gets written, the code that writes it, who checks the record, and what the check does and does not show. The names, numbers, and entries are made up. Every call in them is the SDK's own, as published in @pacspace-io/sdk 0.12.0.
For frontier labs and teams running agents
- An agent in an evaluation reaches past its test: an agent calls a host its test does not allow, a monitor blocks the call and pages a person, and an outside evaluator checks the run.
- What an agent did, and who approved it: Agent-7 issues a refund under its limit on its own, and hands one above its limit to a person who approves before it acts.
- A model release: which build passed its tests, which build went live, and which build a deployment says it runs.
For evaluators and auditors
- An evaluator checks the record: a link and a code, the check in the browser, and the history file checked again with the open-source checker, without the lab and without PacSpace.
For government
- A claims agent and the person who decides: an agent recommends a decision on a benefits claim, an examiner approves or overturns it, and the program office and its inspector general each check the record.
For any record
- Correcting an entry: an entry that recorded the wrong test count, corrected by a new entry, with the original still there as it was written.