Prompt Efficiency Challenge, Season 1

How it works

1. Solve the five tasks below with any language model and any prompts you like.
2. Record every prompt, every output, the token counts and the total cost at list price.
3. Publish your entry as a public gist, document or repository and submit the link below.

Scoring (100 points)

• Accuracy, 60 points. 12 points per task, checked against the answer key and rubric published after the deadline.
• Efficiency, 30 points. Among entries with at least 48 accuracy points, the lowest total cost scores 30 and others score in proportion.
• Reproducibility, 10 points. Exact model names, settings and prompts that a judge can rerun.

Prizes

• First place: $300
• Second place: $150
• Third place: $50

Winners are featured on LLModel.com and in The LLM Brief. Sponsors may add prizes; any additions will be announced on this page.

The five tasks

Task 1: extract an invoice

Return a JSON object with the fields vendor, invoice_number, issue_date (ISO 8601), subtotal, tax, total and currency (ISO 4217).

INVOICE A-20931 | Larkspur Paper Co.
Issued: 3rd of September 2026
12 boxes A4 paper @ 18.50 = 222.00
Delivery 15.00
Subtotal 237.00
GST 5% 11.85
Amount due: CAD 248.85

Task 2: triage support tickets

Label each message with exactly one of: Billing, Bug, Feature request, Account access, Other.

1. I was charged twice for October, please refund one.
2. The export button does nothing when I click it in Safari.
3. Could you add a dark mode to the mobile app?
4. I can't log in since I changed phones; the 2FA codes don't work.
5. Do you have an office in Berlin?
6. My invoice shows the wrong company name. Can you fix it?
7. After the update, charts show yesterday's data instead of today's.
8. It would be great to schedule reports weekly instead of daily.

Task 3: summarize a notice in 40 words or fewer

Starting November 3, 2026, the Team plan changes in three ways. First, the monthly
price rises from $12 to $14 per seat, though annual subscribers keep their current
rate until renewal. Second, every Team workspace gets 100 GB of shared storage, up
from 25 GB, with no action needed. Third, single sign-on, previously an add-on at $3
per seat, is now included at no extra cost. Workspaces on the Free and Business plans
aren't affected. Existing invoices won't change, and admins will see the new price on
their first bill after November 3.

Task 4: fix the function

Return a corrected version that passes all four tests.

def median(values):
    """Return the median of a non-empty list of numbers."""
    s = sorted(values)
    mid = len(s) // 2
    return s[mid]

assert median([3, 1, 2]) == 2
assert median([4, 1, 3, 2]) == 2.5
assert median([5]) == 5
assert median([-1, -3, 10, 2]) == 0.5

Task 5: reason over a price table

1. A team of 12 needs 50,000 API calls a month. Which plan is cheapest, and what does it cost per month?
2. What does the Scale plan cost per month for 35 seats?
3. Ignoring API limits, at how many seats does Scale first become cheaper than Growth?