Tasks · desk · Extraction
Key August supplier bills into the AP import file
supplier-invoices-to-csv · timeout 1800 s · source
The ask
Sent verbatim as the prompt. The agent works in a copy of the workspace below and sees nothing else.
Can you get all of August's supplier bills into the accounting import file for me? The documents are in the folder and Ines's note says how the template is filled in. Save it as invoice_lines.csv.
Workspace
- ap_import_template.csv
- note_from_ines.txt
- supplier_documents_2026-08/Kaffa_Rechnung_RE-2026-0465.pdf
- supplier_documents_2026-08/PacificPackaging_CM-20539.pdf
- supplier_documents_2026-08/PacificPackaging_PPS-55983.pdf
- supplier_documents_2026-08/PacificPackaging_statement_2026-08-31.pdf
- supplier_documents_2026-08/SummitOffice-SOS-INV-39024.pdf
- supplier_documents_2026-08/bluestem_BD776964.pdf
- supplier_documents_2026-08/cascade_cup_lid_CCL-26-1146.pdf
- supplier_documents_2026-08/hollowell_invoice_HE-2026-0814.pdf
- supplier_documents_2026-08/scan_greenline_20260823.pdf
- vendor_list.csv
Expected deliverables
- invoice_lines.csv
Checks
The task passes only when every required check passes. Reference answers, tolerances, and the planted traps are in the repository, not here.
| Check | Type | Required | Failed (frozen) / 6 |
|---|---|---|---|
| template columns | csv_columns | yes | 0 |
| one row per line | csv_set_equal | yes | 4 |
| row count | csv_row_count | yes | 4 |
| document identity | csv_values_match | yes | 0 |
| dates and currency | csv_values_match | yes | 0 |
| descriptions | csv_values_match | yes | 0 |
| quantities and prices | csv_values_match | yes | 0 |
| line amounts and discounts | csv_values_match | yes | 0 |
| document totals | csv_values_match | yes | 0 |
Recorded results, complete desk comparison
pass pass, frozen scorer only fail timed out
| System | Repetitions | Frozen / 3 | Raw / 3 | Median minutes | Mean cost (USD) |
|---|---|---|---|---|---|
| Proto / DeepSeek V4.1 Flash | 2 | 2 | 1.7 | 0.0249 | |
| Codex / GPT-5.6 sol | 0 | 0 | 2.0 | 0.2746 |
Per-attempt check verdicts, raw and frozen verdicts, usage, receipt and artifact hashes for this task are in results/latest/attempts.jsonl under "task": "supplier-invoices-to-csv".