An evaluator checks the record

Evaluator E-2 tested agent-7 for a lab, and the lab's record of evaluation run 4417 is part of what E-2 will rely on in its report. (The record itself is An agent in an evaluation.) The lab's board committee wants the same record, with less of it shown. This page follows the record from the lab to each of them, and says what each check shows.

What the evaluator asks for#

E-2 asks for four things in its engagement letter, before it relies on the record:

• A link to the record, with its code sent by a separate channel. E-2 needs nothing else to check it.
• Which fields E-2 will see: the seals alone, chosen fields, or every field.
• The record's history file, to check again on E-2's own computer.
• Which kinds of entries the lab writes, so E-2 knows what the record covers before relying on it.

The lab shares the record#

The lab shares the run's record from the dashboard's share card and chooses Every field for each entry. The API has the share call too, and a link made from code shows each entry's seal and none of its fields:

curl -X POST https://app.pacspace.io/api/v1/records/machine-action-record/eval-run-4417/share \
  -H "x-api-key: $PACSPACE_API_KEY" \
  -H "Content-Type: application/json" \
  -d '{ "sharedWith": "Evaluator E-2", "expiresInDays": 30 }'

The answer carries the link in url and the code in accessCode. The lab sends the link by email and the code by phone. sharedWith is the name an acknowledgment is recorded under.

A record has one link at a time: sharing it again returns the same link and code, and a link that reveals fields replaces the one before it. So the board committee gets a file instead of a link. The lab builds it with records.disclose, revealing each entry's title and nothing else, and sends it:

import { writeFile } from 'node:fs/promises';
import { PacSpace } from '@pacspace-io/sdk';

const pac = new PacSpace({ apiKey: process.env.PACSPACE_API_KEY! });

const copy = await pac.records.disclose({
  record: 'eval-run-4417',
  entries: [1, 2, 3, 4, 5, 6, 7].map((entry) => ({ entry, reveal: ['/title'] })),   // entries count from 1
  source: { kind: 'mirror' },   // the file is checked against what was committed before it is returned
});
await writeFile('eval-run-4417.board.json', copy.bytes);

The committee checks that file with the same checker E-2 uses below. The checker confirms every seal, in order, and each title against its entry's seal; with --json, its report also counts the fields kept back.

E-2's browser checks it#

E-2 opens the link. The page shows the lab's name and asks for the code, and shows nothing of the record until the code is entered. Then the check runs in E-2's browser, against the seals as they were committed, before E-2 reads a word:

"This record has 7 entries. All 7 seals are here, in order, and each is in what was committed."

Under it: "Every entry shows fields the sharer chose to reveal. Each shown field was checked against its entry's seal." E-2 opens entries 3 and 4 and reads the call the agent made and the monitor's block. To see that the check is real, E-2 presses "Change one character": the page changes one character of E-2's copy, in the browser only, and the next check catches it. "Restore" puts the copy back.

The lab also sent the run's transcript with its blinding file, transcript.jsonl.pacspace.json. E-2 drops both on the page. The browser fingerprints the transcript there, uploads neither file, and says whether it is the file entry 7 names.

E-2 checks the history file#

For its report, E-2 wants a check that does not depend on the link staying open, or on the lab, or on PacSpace. E-2 asks for the record's history file, and the lab downloads it:

curl -H "x-api-key: $PACSPACE_API_KEY" -H "Accept: application/octet-stream" \
  -o history.json \
  https://app.pacspace.io/api/v1/records/machine-action-record/eval-run-4417/history

The history file carries every field of every entry. The lab sends it to E-2, who may read every field; for a reader who may not, the file records.disclose builds is the one to send.

E-2 runs the open-source checker on its own computer. It reads what was committed from the proof layer, the infrastructure no party controls where each seal is committed, and never calls PacSpace or the lab:

npx @pacspace-io/check history.json
npx @pacspace-io/check history.json --held-file transcript.jsonl --blinding transcript.jsonl.pacspace.json

Among the lines the second command prints:

This record has 7 entries. All 7 are here, and all 7 match what was committed.
Compared with the proof layer for this record: it matches.
The file you hold matches entry 7.
The check shows the record is unchanged since it was committed. It does not show that what was recorded was true.

It exits 0 when the check passed as asked, 1 when it did not or the file E-2 holds is not in the record, 2 when a file or an argument could not be used, and 3 when the proof layer could not be reached. --json prints the same report as a document E-2's own tools can read. E-2 keeps history.json with its working papers and can run the check again in a year, after the link has expired.

What the check shows, and what it does not#

The check shows:

• Every entry in the file is the entry that was committed, and all of the record's entries are here, in order.
• What was committed agrees with the proof layer, which the checker reads without PacSpace.
• The transcript E-2 holds is the file entry 7 names.
• Once committed, no one can change this record, PacSpace included. A change to any copy shows at the next check.

The check does not show:

• Whether what the lab wrote happened as written. The check shows the record is unchanged since it was committed. It does not show that what was recorded was true.
• Anything the lab did not write. The lab decides what to write, the same limit every log has, which is why E-2 asks which kinds of entries it writes. A gap in the record is as telling as a change.
• What a field kept back says. The board committee's check counts those fields and shows none of them.

E-2 records that it checked, if it chooses#

Recording a check is optional, and the record checks the same without it. If E-2 records it from the page, the record carries a line under the name the link was shared with: "Acknowledged through the link issued to Evaluator E-2, 2 October. Entry 7 was checked by its seal and it matched." The lab, and anyone who opens that link, sees it beneath the record.

When the evaluator's side is software#

When E-2's own systems do the checking, the lab issues a grant instead of a link, and E-2's software reads the history with it and runs the checker or records.check. See Share a record.