Tasks · desk · Bookkeeping
Match bank payments to open invoices, with card fees, short wires, and a wrong reference
payments-match-v2 · timeout 1200 s · source
The ask
Sent verbatim as the prompt. The agent works in a copy of the workspace below and sees nothing else.
Match the payments in this morning's bank export to our open invoices. Give me unpaid.csv with the invoices still open (invoice_id, customer, amount_outstanding), matches.csv showing what got applied where (invoice_id, line_id, amount_applied), and unapplied.csv for any credit that is not a payment on an open invoice (line_id, amount, reason). My note in the folder has the house rules.
Workspace
- bank_export.csv
- note.txt
- open_invoices.csv
Expected deliverables
- unapplied.csv
- unpaid.csv
This task also has a task-specific grader (check.py).
Checks
The task passes only when every required check passes. Reference answers, tolerances, and the planted traps are in the repository, not here.
| Check | Type | Required | Failed (frozen) / 6 |
|---|---|---|---|
| unpaid.csv columns | csv_columns | yes | 0 |
| unpaid invoice ids | csv_set_equal | yes | 0 |
| amount outstanding per open invoice (partial remainder required) | csv_values_match | yes | 0 |
| matches.csv applies payments correctly | custom | yes | 0 |
| unapplied.csv columns | csv_columns | yes | 0 |
| unapplied line ids (refund, interest, closed-invoice payment; nothing else) | csv_set_equal | yes | 0 |
Recorded results, complete desk comparison
pass pass, frozen scorer only fail timed out
| System | Repetitions | Frozen / 3 | Raw / 3 | Median minutes | Mean cost (USD) |
|---|---|---|---|---|---|
| Proto / DeepSeek V4.1 Flash | 3 | 3 | 2.2 | 0.0268 | |
| Codex / GPT-5.6 sol | 3 | 3 | 2.4 | 0.2407 |
Per-attempt check verdicts, raw and frozen verdicts, usage, receipt and artifact hashes for this task are in results/latest/attempts.jsonl under "task": "payments-match-v2".