from ODM to SDTM / ADaM and define.xml
NJSUG & PhilaSUG Joint In-Person Meeting 2026
2026-10-09
The views and opinions expressed in this presentation are those of the individual presenter. They do not represent those of their affiliated organizations or institutions.
From an FDA warning letter.
“you must review the AI-generated documents to ensure they were accurate and actually compliant…”
| Prompt tier | Python | R | opensas |
|---|---|---|---|
| Full (every rule stated) | 99.3% (133/134) | 98.5% (132/134) | 86.6% (116/134) |
| Conventions (labels and value rules) | 35.8% (48/134) | 33.6% (45/134) | 34.3% (46/134) |
| Brief (columns only) | 18.7% (25/134) | 17.9% (24/134) | 15.7% (21/134) |
Agent: opencode 1.18.33. Model: opencode-go/muse-spark-1.3-contributor, 1 attempt, Oct 12 2026
Shared by all prompts: derive ADVS from ADVS_RAW using CDISC ADaM standards; one record per subject, visit, and parameter.
Columns, in order: STUDYID, USUBJID, PARAMCD, PARAM, AVAL, VISIT, DTYPE
Files: read from /app/input; write /app/output/advs.csv.
| Rule in the prompt | Full · passes | Conventions · passes | Brief · fails |
|---|---|---|---|
| MAP identifiers | PARAMCD = "MAP"PARAM = "Mean Arterial Pressure (mmHg)" |
Same MAP identifiers | Not stated |
| Records to add | Keep all collected records. Add one MAP per subject/visit only when both SBP and DBP results exist and are non-missing. | Not stated | Not stated |
AVAL |
Keep collected values. MAP = (2 × DBP + SBP) / 3; do not round. |
MAP is not rounded; formula not stated. |
Not stated |
DTYPE |
Added MAP: CALCULATION.Collected records: blank. |
CALCULATION or blank;assignment not stated. |
Not stated |
Brief omits both the MAP formula and the rule for adding a record.
SBP = systolic blood pressure; DBP = diastolic blood pressure.
https://elong0527.github.io/yamaa/benchmark/adam-advs-map.html
Scroll inside the frame to explore the spec, inputs, and golden output.
A yamaa specification has exactly one execution. Where it would have two, yamaa fails instead of choosing.
output_dataset = derive(input_datasets, spec)
The design of yamaa is a contract between people and AI agents.
Inheritance reduces duplication while keeping every resolved study specification self-contained and reviewable. (example)
| Component | Purpose | Links |
|---|---|---|
| Schema | Declares the vocabulary of the specification. | Repository / Introduction |
| Rules | Defines the meaning of each specification. | Repository / Rules |
| Engine | Runs specifications in Python and R according to the rules. | Python / R |
| Benchmark | Provides runnable specification examples. | Repository / Dashboards |
Learn with AI: https://arena.ai/
Note: this is an iterative process while the study is ongoing. Team members and AI collaborate centrally on the version controlled specification.
https://github.com/RConsortium/submissions-pilot7-synthetic-data