yamaa: agentic workflow for CDISC data standardization

from ODM to SDTM / ADaM and define.xml
NJSUG & PhilaSUG Joint In-Person Meeting 2026

Yilong Zhang

2026-10-09

Disclaimer

The views and opinions expressed in this presentation are those of the individual presenter. They do not represent those of their affiliated organizations or institutions.

Contributors

  • Peng Zhang CIMS Global
  • Bingjun Wang Genentech
  • Ming-chun Chen Summit Therapeutics
  • Yilong Zhang BBSW AI Committee

Motivation

  • Two AI agents use the same specification to derive ADaM data.
  • Both report completion, but their source code and outputs differ.
  • A reviewer makes non-trivial efforts to resolve the difference before signing off.

From an FDA warning letter.

“you must review the AI-generated documents to ensure they were accurate and actually compliant…”

One root cause

  • The user prompt leaves business rules implicit, so agents must infer them.
  • Business rules can differ from language defaults, e.g.:
    • R and Python round ties to even.
    • SAS compares missing numeric values as lower than non-missing values.

Evidence: prompt completeness decides success

Prompt tier Python R opensas
Full (every rule stated) 99.3% (133/134) 98.5% (132/134) 86.6% (116/134)
Conventions (labels and value rules) 35.8% (48/134) 33.6% (45/134) 34.3% (46/134)
Brief (columns only) 18.7% (25/134) 17.9% (24/134) 15.7% (21/134)
  • Same 134 tasks, same agent. Only the prompt changed.

Agent: opencode 1.18.33. Model: opencode-go/muse-spark-1.3-contributor, 1 attempt, Oct 12 2026

Mean arterial pressure: three prompts

Shared by all prompts: derive ADVS from ADVS_RAW using CDISC ADaM standards; one record per subject, visit, and parameter.
Columns, in order: STUDYID, USUBJID, PARAMCD, PARAM, AVAL, VISIT, DTYPE
Files: read from /app/input; write /app/output/advs.csv.

Rule in the prompt Full · passes Conventions · passes Brief · fails
MAP identifiers PARAMCD = "MAP"
PARAM = "Mean Arterial Pressure (mmHg)"
Same MAP identifiers Not stated
Records to add Keep all collected records. Add one MAP per subject/visit only when both SBP and DBP results exist and are non-missing. Not stated Not stated
AVAL Keep collected values.
MAP = (2 × DBP + SBP) / 3;
do not round.
MAP is not rounded;
formula not stated.
Not stated
DTYPE Added MAP: CALCULATION.
Collected records: blank.
CALCULATION or blank;
assignment not stated.
Not stated

Brief omits both the MAP formula and the rule for adding a record.
SBP = systolic blood pressure; DBP = diastolic blood pressure.
https://elong0527.github.io/yamaa/benchmark/adam-advs-map.html

Goal

  • Reduce human review effort.
  • Leverage organization templates.
  • Capture business rules that AI agents can apply.
  • Human and AI agents align at the planning stage.

yamaa: YAML specification example

schema_version: "1.0"
domain: ADSL
keys: [STUDYID, USUBJID]
input:
  SOURCE: input/adsl.csv

columns:
  - name: BMI
    type: float
    label: Body Mass Index (kg/m2)
    derivation:
      compute:
        expr: "WEIGHTKG / POWER(NULLIF(HEIGHTCM, 0) / 100, 2)"

verifications:
  - unique: [STUDYID, USUBJID]

yamaa: benchmarks

~ 300
runnable benchmarks
2
engines (Python, R)

Scroll inside the frame to explore the spec, inputs, and golden output.

yamaa: full studies

  • Build publicly available ground truth.
  • CDISC pilots (SDTM -> ADaM)
    • cdiscpilot01 and RConsortium submission pilots 3, 5, and 6.
  • OpenClinica real sample data (ODM -> SDTM -> ADaM)
    • Real sample data for testing the full data journey.
  • Synthetic studies (ODM -> SDTM -> ADaM)
    • Simulate ODM XML from real clinical studies.

yamaa: core principle

A yamaa specification has exactly one execution. Where it would have two, yamaa fails instead of choosing.

output_dataset = derive(input_datasets, spec)

The design of yamaa is a contract between people and AI agents.

  • spec: schema following closed vocabulary.
  • derive: business rules with derivation semantics.

yamaa: inheritance

Inheritance reduces duplication while keeping every resolved study specification self-contained and reviewable. (example)

yamaa: four components

Component Purpose Links
Schema Declares the vocabulary of the specification. Repository / Introduction
Rules Defines the meaning of each specification. Repository / Rules
Engine Runs specifications in Python and R according to the rules. Python / R
Benchmark Provides runnable specification examples. Repository / Dashboards

Recommendations: ODM XML

  • CDISC ODM is an XML standard for clinical research data.
  • EDC systems can export study data in ODM XML.
  • Teams familiar with define.xml will recognize the ODM structure.

Learn with AI: https://arena.ai/

Recommendations: Input and Output

  • Plain text is the input.
  • Annotated CRF is not an input but an output.
  • Excel specification is not an input but an output.
  • Communication costs more than tokens.

Summary of the workflow

  • Extract: enable regular EDC extraction in ODM format for each study to create a versioned source.
  • Draft: ask AI agent to draft the study spec from the organization or program template, adding only study differences.
  • Review: human and AI review study derivations and verifications in yamaa where fixes are cheap.
  • Finalize: approve and tag the resolved spec in version control with full lineage.
  • Automate: run the same spec on schedule in R or Python to produce identical SDTM and ADaM plus define.xml.

Note: this is an iterative process while the study is ongoing. Team members and AI collaborate centrally on the version controlled specification.

How to contribute

  • RConsortium / BBSW
  • Public benchmarks
    • Review a benchmark: inspect the specification and golden output
    • Add comments: question assumptions, clarity, and conformance
    • File issues: record anything ambiguous or incorrect
    • Open pull requests: contribute fixes and new benchmarks

Benchmark to deployment

  • Benchmarks help assess an AI agent before adoption.
  • Deployment also requires onboarding the agent to a specific organization environment. Collaborating with keiji.ai to test synthetic clinical trial data.

https://github.com/RConsortium/submissions-pilot7-synthetic-data

Reference