Monthly revenue report by region from a messy sales export
monthly-report · timeout 1200 s · source
The ask
Sent verbatim as the prompt. The agent works in a copy of the workspace below and sees nothing else.
Build me a monthly revenue report by region from this export - regions down the side, months across - as an Excel file with live formulas (report.xlsx), plus a short memo.md with the three things I should know.
Workspace
- sales_export.csv
Expected deliverables
- memo.md
- report.xlsx
Checks
The task passes only when every required check passes. Reference answers, tolerances, and the planted traps are in the repository, not here.
| Check | Type | Required | Failed (frozen) / 6 |
|---|---|---|---|
| report.xlsx exists | file_exists | yes | 0 |
| report uses live formulas | xlsx_has_formulas | yes | 0 |
| no formula evaluates to an error | xlsx_no_errors | yes | 1 |
| North half-year total (five months, no invented March) | xlsx_value_present | yes | 1 |
| South half-year total | xlsx_value_present | yes | 1 |
| East half-year total | xlsx_value_present | yes | 1 |
| West half-year total | xlsx_value_present | yes | 1 |
| South February (contains a duplicate and a refund) | xlsx_value_present | yes | 1 |
| West June (contains a duplicate and a refund) | xlsx_value_present | yes | 1 |
| one sentence says North has no March data (a gap, not zero sales) | text_sentence_matches | yes | 0 |
| memo mentions duplicates or refunds | text_contains_any | yes | 0 |
Recorded results, complete desk comparison
pass pass, frozen scorer only fail timed out
| System | Repetitions | Frozen / 3 | Raw / 3 | Median minutes | Mean cost (USD) |
|---|---|---|---|---|---|
| Proto / DeepSeek V4.1 Flash | 3 | 3 | 2.5 | 0.0396 | |
| Codex / GPT-5.6 sol | 2 | 2 | 5.0 | 0.5271 |
Per-attempt check verdicts, raw and frozen verdicts, usage, receipt and artifact hashes for this task are in results/latest/attempts.jsonl under "task": "monthly-report".