Tasks · desk · Spreadsheet
What is still coming on our purchase orders
open-purchase-orders · timeout 1200 s · source
The ask
Sent verbatim as the prompt. The agent works in a copy of the workspace below and sees nothing else.
Leila needs to know what is still coming on our open purchase orders. Check the PO report against the dock log and save open_po_lines.csv; her email has the vendor changes and how she wants it.
Workspace
- dock_receiving_log.xlsx
- email_thread_purchasing.txt
- item_master.csv
- open_po_report_2026-09-10.csv
Expected deliverables
- open_po_lines.csv
Checks
The task passes only when every required check passes. Reference answers, tolerances, and the planted traps are in the repository, not here.
| Check | Type | Required | Failed (frozen) / 6 |
|---|---|---|---|
| requested columns | csv_columns | yes | 0 |
| lines still open | csv_set_equal | yes | 0 |
| row count | csv_row_count | yes | 0 |
| open quantity per line | csv_values_match | yes | 0 |
| over-receipt flag | csv_values_match | yes | 0 |
Recorded results, complete desk comparison
pass pass, frozen scorer only fail timed out
| System | Repetitions | Frozen / 3 | Raw / 3 | Median minutes | Mean cost (USD) |
|---|---|---|---|---|---|
| Proto / DeepSeek V4.1 Flash | 3 | 3 | 2.1 | 0.0265 | |
| Codex / GPT-5.6 sol | 3 | 3 | 2.0 | 0.3237 |
Per-attempt check verdicts, raw and frozen verdicts, usage, receipt and artifact hashes for this task are in results/latest/attempts.jsonl under "task": "open-purchase-orders".