Tasks · desk · Spreadsheet
One staff directory from HR and IT
staff-directory-merge · timeout 1200 s · source
The ask
Sent verbatim as the prompt. The agent works in a copy of the workspace below and sees nothing else.
Colleen needs the staff directory for the new website, put together from the HR export and IT's Google user list. Her note has the rules - save it as directory.csv.
Workspace
- google_workspace_users.csv
- hr_employee_export_2026-09-10.csv
- note_from_colleen.txt
Expected deliverables
- directory.csv
Checks
The task passes only when every required check passes. Reference answers, tolerances, and the planted traps are in the repository, not here.
| Check | Type | Required | Failed (frozen) / 6 |
|---|---|---|---|
| requested columns | csv_columns | yes | 0 |
| people in the directory | csv_set_equal | yes | 0 |
| row count | csv_row_count | yes | 0 |
| department per person | csv_values_match | yes | 0 |
| name as people go by | csv_values_match | yes | 0 |
| title and extension | csv_values_match | yes | 0 |
Recorded results, complete desk comparison
pass pass, frozen scorer only fail timed out
| System | Repetitions | Frozen / 3 | Raw / 3 | Median minutes | Mean cost (USD) |
|---|---|---|---|---|---|
| Proto / DeepSeek V4.1 Flash | 3 | 3 | 1.7 | 0.0235 | |
| Codex / GPT-5.6 sol | 3 | 3 | 1.8 | 0.2436 |
Per-attempt check verdicts, raw and frozen verdicts, usage, receipt and artifact hashes for this task are in results/latest/attempts.jsonl under "task": "staff-directory-merge".