Tasks · desk · Spreadsheet
How the August promo codes did
promo-code-analysis · timeout 1200 s · source
The ask
Sent verbatim as the prompt. The agent works in a copy of the workspace below and sees nothing else.
Hannah wants to know how each promo code did in August, including any codes that shouldn't have gone through. Use the order export and marketing's code list and save promo_summary.csv; her email has the details.
Workspace
- email_from_hannah.txt
- orders_2026-08.csv
- promo_codes_2026.xlsx
Expected deliverables
- promo_summary.csv
Checks
The task passes only when every required check passes. Reference answers, tolerances, and the planted traps are in the repository, not here.
| Check | Type | Required | Failed (frozen) / 6 |
|---|---|---|---|
| requested columns | csv_columns | yes | 0 |
| codes used | csv_set_equal | yes | 1 |
| row count | csv_row_count | yes | 1 |
| orders per code | csv_values_match | yes | 1 |
| revenue per code | csv_values_match | yes | 1 |
| status per code | csv_values_match | yes | 1 |
Recorded results, complete desk comparison
pass pass, frozen scorer only fail timed out
| System | Repetitions | Frozen / 3 | Raw / 3 | Median minutes | Mean cost (USD) |
|---|---|---|---|---|---|
| Proto / DeepSeek V4.1 Flash | 2 | 2 | 1.2 | 0.0174 | |
| Codex / GPT-5.6 sol | 3 | 3 | 1.8 | 0.3713 |
Per-attempt check verdicts, raw and frozen verdicts, usage, receipt and artifact hashes for this task are in results/latest/attempts.jsonl under "task": "promo-code-analysis".