# support-ticket-triage fixture

Origin: 130 synthetic support tickets written for this evaluation (100 scored
test, 30 dev). No real customer data and no public dataset text. The prose in
the template bank was authored by the Pareto coding agent in this repository;
the multilingual tickets use short template sentences in French, Spanish,
German, Japanese and Portuguese. No baseline service was called.

License: CC BY 4.0 (text) for fixture.jsonl and labels.jsonl; the generator and
recipe code are MIT. Full text: LICENSE.txt.

Labeling method: labels are generator-intended labels. Category, priority and
team were fixed by deterministic rules before the ticket text was written
(labels precede prose; the generator cannot see the text it will produce), then
spot-checked by the authoring agent against the rubric frozen in eval.json. They are not independent
human labels. Two human labelers and a published kappa remain an open item; the
page states plainly that labels were set by the generator and spot-checked by
us. The template-seed subset (the first ten test tickets, strata
seed_template=true; agent-authored, not hand-written by a person) is scored separately as a secondary cut.

Strata in the scored test split (n=100): 20 ambiguous, 10 long threads (16+ 
earlier messages), 5 non-English (fr/es/de/ja), 5 with no useful detail, and
realistic class skew (30 billing, 25 access, 25 bug, 15 feature, 5 unknown;
12 P1, 44 P2, 44 P3). The dev split (n=30) has the same label rules, separate
templates, and no scored ticket is reused there.

Generator: deterministic Python templates authored by a Pareto coding agent.
No separate dataset-generation API calls were made. Pareto's internal selected
models are not exposed here, so exclusion of OpenAI model families cannot be
established. This is a generator-truth pilot, not an independent model-family
benchmark. The authoring agent spot-checked labels, not a human reviewer.

Known limitations: synthetic tickets are cleaner than production queues;
"unknown" is defined as no useful detail rather than an open category; the
non-English tickets are short translated templates, not fluent prose.