SLDS 2026, New York
2026-11-03
The views and opinions expressed in this presentation are those of the individual presenter. They do not represent those of their affiliated organizations or institutions.
From an FDA warning letter.
“you must review the AI-generated documents to ensure they were accurate and actually compliant…”
| Prompt tier | Python | R | opensas |
|---|---|---|---|
| Full | 99.3% (133/134) | 98.5% (132/134) | 86.6% (116/134) |
| Conventions | 35.8% (48/134) | 33.6% (45/134) | 34.3% (46/134) |
| Brief | 18.7% (25/134) | 17.9% (24/134) | 15.7% (21/134) |
Same 134 tasks, same agent. Only the prompt changed.
opensas: https://github.com/kirha-ai/opensas
Agent: opencode 1.18.33. Model: opencode-go/muse-spark-1.3-contributor, 1 attempt, Oct 12 2026
Shared Prompt:
ADVS from ADVS_RAW using CDISC ADaM standards; one record per subject, visit, and parameter.STUDYID, USUBJID, PARAMCD, PARAM, AVAL, VISIT, DTYPE/app/input; write /app/output/advs.csv.| Rule in the prompt | Full - passes | Conventions - passes | Brief - fails |
|---|---|---|---|
| MAP identifiers | PARAMCD = "MAP"PARAM = "Mean Arterial Pressure (mmHg)" |
Same MAP identifiers | Not stated |
| Records to add | Keep all collected records. Add one MAP per subject/visit only when both SBP and DBP results exist and are non-missing. | Not stated | Not stated |
AVAL |
Keep collected values. MAP = (2 x DBP + SBP) / 3; do not round. |
MAP is not rounded; formula not stated. |
Not stated |
DTYPE |
Added MAP: CALCULATION.Collected records: blank. |
CALCULATION or blank;assignment not stated. |
Not stated |
Brief omits both the MAP formula and the rule for adding a record.
SBP = systolic blood pressure; DBP = diastolic blood pressure.
https://elong0527.github.io/yamaa/benchmark/adam-advs-map.html
Human and AI review the specification during planning; the engine executes the agreed rules.
A yamaa specification has exactly one execution. Where it would have two, yamaa fails instead of choosing.
output_dataset = derive(input_datasets, spec)
The design of yamaa is a contract between human and AI agents.
| Component | Purpose | Links |
|---|---|---|
| Schema | Declares the vocabulary of the specification. | Repository / Introduction |
| Rules | Defines the meaning of each specification. | Repository / Rules |
| Engine | Runs specifications in Python, R and CLI consistently following the rules. | Python / R / CLI |
| Benchmark | Provides runnable specification examples. | Repository / Dashboards |
Inheritance reduces duplication while keeping every resolved study specification self-contained and reviewable. (example)
Scroll inside the frame to explore the spec, inputs, and golden output.
Note: this is an iterative process while the study is ongoing. Team members and AI collaborate centrally on the version controlled specification at the planning stage.