Tasks · desk · Reformatting
Summer signups into one Mailchimp audience import
mailchimp-audience-import · timeout 1200 s · source
The ask
Sent verbatim as the prompt. The agent works in a copy of the workspace below and sees nothing else.
Put everyone who signed up with us this summer into one Mailchimp import file using their template. Ines's note has the rules. Save it as mailchimp_import.csv.
Workspace
- farmers_market_signups_aug.csv
- mailchimp_import_template.csv
- mailchimp_unsubscribed_2026-09-01.csv
- note_from_ines.txt
- website_popup_form_export.csv
- workshop_registrations_summer.xlsx
Expected deliverables
- mailchimp_import.csv
This task also has a task-specific grader (check.py).
Checks
The task passes only when every required check passes. Reference answers, tolerances, and the planted traps are in the repository, not here.
| Check | Type | Required | Failed (frozen) / 6 |
|---|---|---|---|
| Mailchimp template columns in order | csv_columns | yes | 0 |
| one row per subscriber | csv_set_equal | yes | 0 |
| row count | csv_row_count | yes | 0 |
| tags per subscriber | custom | yes | 0 |
| birthday merge field | csv_values_match | yes | 0 |
| name merge fields | csv_values_match | yes | 0 |
Recorded results, complete desk comparison
pass pass, frozen scorer only fail timed out
| System | Repetitions | Frozen / 3 | Raw / 3 | Median minutes | Mean cost (USD) |
|---|---|---|---|---|---|
| Proto / DeepSeek V4.1 Flash | 3 | 3 | 1.7 | 0.0228 | |
| Codex / GPT-5.6 sol | 3 | 3 | 2.2 | 0.2911 |
Per-attempt check verdicts, raw and frozen verdicts, usage, receipt and artifact hashes for this task are in results/latest/attempts.jsonl under "task": "mailchimp-audience-import".