Tasks · desk · Bookkeeping
Categorize card transactions per policy, with thread overrides
expense-categorize · timeout 1200 s · source
The ask
Sent verbatim as the prompt. The agent works in a copy of the workspace below and sees nothing else.
Categorize these card transactions per our policy, the thread has a couple of changes. Output categorized.csv with a category column added, and put REVIEW on anything you can't place.
Workspace
- expense_policy.md
- thread.txt
- transactions.csv
Expected deliverables
- categorized.csv
Checks
The task passes only when every required check passes. Reference answers, tolerances, and the planted traps are in the repository, not here.
| Check | Type | Required | Failed (frozen) / 6 |
|---|---|---|---|
| one output row per transaction | csv_row_count | yes | 0 |
| source columns kept and category added (txn_id, date, time, description, amount, card_last4, category) | csv_columns | yes | 0 |
| category per txn_id (thread-override rows, REVIEW rows, and policy-settled furniture and supplies merchants must be right) | csv_values_match | yes | 1 |
Recorded results, complete desk comparison
pass pass, frozen scorer only fail timed out
| System | Repetitions | Frozen / 3 | Raw / 3 | Median minutes | Mean cost (USD) |
|---|---|---|---|---|---|
| Proto / DeepSeek V4.1 Flash | 3 | 3 | 3.3 | 0.0422 | |
| Codex / GPT-5.6 sol | 2 | 2 | 2.1 | 0.3173 |
Per-attempt check verdicts, raw and frozen verdicts, usage, receipt and artifact hashes for this task are in results/latest/attempts.jsonl under "task": "expense-categorize".