# Synthetic research-paper extraction fixture

Origin: original deterministic generator, scripts/evals/research-paper-data-extraction/generate_fixture.py, authored with Pareto on 2026-10-27. No separate model generated the text. Values and quotes are fixed before prose assembly. No external abstracts, PMC text, patient data, or copyrighted articles were copied.

License: fixture.jsonl and labels.jsonl are dedicated to CC0 1.0 by their contributor, Unbiased. https://creativecommons.org/publicdomain/zero/1.0/ . All executable code is MIT; see LICENSE.txt. No upstream license or attribution obligations apply to this original synthetic set.

Scope: 20 test excerpts and 4 separate dev excerpts, 15 fields each. One fictional question: exercise added to usual care for adults with essential hypertension. These are short synthetic RCT-style excerpts, not real RCTs, PMC articles, scans, full papers, or medical evidence. SYN identifiers are fictional and are not PMCIDs. Clinical realism and completeness have not been validated.

Labels set by the generator, spot-checked by us through an automated field-to-source check. No human annotators or adjudication yet. Human clinical spot-check remains required. Generator-intended labels replace the card's proposed two annotators and PMC OA collection under the synthetic-data instruction. No human agreement number is claimed.

Rubric: normalize case and whitespace only; exact string values otherwise, including signed changes and numeric strings. Funding omitted from the source is null with an empty quote. Every other field requires a verbatim nonempty source substring. Quote match checks literal presence, not entailment. A quote can match the text yet support the wrong field; value accuracy is scored separately. Pilot enrollment, completion counts, diastolic outcomes and later visits are distractors.

Headline: mean field accuracy over equally sized papers. The per-item score is correct fields / 15, aggregated by median across repeats, then averaged across papers. This uses paper-level resampling instead of the card's field resampling to avoid treating correlated fields in the same excerpt as independent. Publish run mean, sample sd and range. Secondary: quote-match fraction, review-load fraction (null or unmatched quote), first-call schema validity. Invalid/missing fields count against accuracy; extra calls are rejected by the runner. Review flags are local checks, not Pareto's internal disagreement.

Cross-arm agreement, errors captured by disagreement and comparative review load are deferred until approved baseline evidence exists. No agreement claim is made. Repeated Pareto answers are correlated; output variation is not proof of statistical independence. This tiny templated dataset cannot estimate clinical systematic-review accuracy or long-input performance.

Freeze policy: prompt frozen before any inference; no tuning on dev or test. Dev repeats check the executable path and output variation only. Any later prompt change requires a new freeze and fresh runs. The scored raw request never includes gold labels or reference quotes. GPT-6.1 Sol was not used for generation, tuning or scoring.