Customer cohort retention table for the first half
retention-cohorts · timeout 1200 s · source
The ask
Sent verbatim as the prompt. The agent works in a copy of the workspace below and sees nothing else.
Build me a cohort retention table for January to June this year from the shop order exports, cohort months down the side written like 2026-01. My notes in the folder explain what I mean. Save it as cohorts.xlsx with the counts as formulas.
Workspace
- cohort_notes.txt
- orders_2025_jul-dec.csv
- orders_export_2026-01-01_2026-06-30.csv
Expected deliverables
- cohorts.xlsx
Checks
The task passes only when every required check passes. Reference answers, tolerances, and the planted traps are in the repository, not here.
| Check | Type | Required | Failed (frozen) / 6 |
|---|---|---|---|
| cohorts.xlsx exists | file_exists | yes | 0 |
| live formulas | xlsx_has_formulas | yes | 0 |
| no error cells | xlsx_no_errors | yes | 2 |
| January new customers | xlsx_value_present | yes | 2 |
| February new customers | xlsx_value_present | yes | 2 |
| January cohort month 1 | xlsx_value_present | yes | 2 |
| March cohort month 2 | xlsx_value_present | yes | 2 |
| total new customers | xlsx_value_present | yes | 0 |
Recorded results, complete desk comparison
pass pass, frozen scorer only fail timed out
| System | Repetitions | Frozen / 3 | Raw / 3 | Median minutes | Mean cost (USD) |
|---|---|---|---|---|---|
| Proto / DeepSeek V4.1 Flash | 3 | 3 | 4.9 | 0.0597 | |
| Codex / GPT-5.6 sol | 1 | 0 | 4.7 | 0.5101 |
Per-attempt check verdicts, raw and frozen verdicts, usage, receipt and artifact hashes for this task are in results/latest/attempts.jsonl under "task": "retention-cohorts".