yamaa: agentic workflow for CDISC data standardization

SLDS 2026, New York

Yilong Zhang

2026-11-03

Disclaimer

The views and opinions expressed in this presentation are those of the individual presenter. They do not represent those of their affiliated organizations or institutions.

Contributors

  • Peng Zhang CIMS Global
  • Bingjun Wang Genentech
  • Ming-chun Chen Summit Therapeutics
  • Jingwei Xiong Merck & Co., Inc
  • Yilong Zhang BBSW AI Committee

Motivation

  • Two AI agents use the same specification to derive analysis datasets in clinical trials. example
  • Both report completion, but their source code and outputs may differ.
  • A reviewer makes non-trivial efforts to resolve the difference before signing off.

From an FDA warning letter.

“you must review the AI-generated documents to ensure they were accurate and actually compliant…”

One root cause

  • The user prompt leaves business rules implicit, so agents must infer them.
  • Business rules can differ from language defaults, e.g.:
    • R and Python round ties to even.
    • SAS compares missing numeric values as lower than non-missing values.

Evidence: prompt completeness decides success

Prompt tier Python R opensas
Full 99.3% (133/134) 98.5% (132/134) 86.6% (116/134)
Conventions 35.8% (48/134) 33.6% (45/134) 34.3% (46/134)
Brief 18.7% (25/134) 17.9% (24/134) 15.7% (21/134)

Same 134 tasks, same agent. Only the prompt changed.
opensas: https://github.com/kirha-ai/opensas
Agent: opencode 1.18.33. Model: opencode-go/muse-spark-1.3-contributor, 1 attempt, Oct 12 2026

Mean arterial pressure: three prompts

Shared Prompt:

  • derive ADVS from ADVS_RAW using CDISC ADaM standards; one record per subject, visit, and parameter.
  • STUDYID, USUBJID, PARAMCD, PARAM, AVAL, VISIT, DTYPE
  • read from /app/input; write /app/output/advs.csv.
Rule in the prompt Full - passes Conventions - passes Brief - fails
MAP identifiers PARAMCD = "MAP"
PARAM = "Mean Arterial Pressure (mmHg)"
Same MAP identifiers Not stated
Records to add Keep all collected records. Add one MAP per subject/visit only when both SBP and DBP results exist and are non-missing. Not stated Not stated
AVAL Keep collected values.
MAP = (2 x DBP + SBP) / 3;
do not round.
MAP is not rounded;
formula not stated.
Not stated
DTYPE Added MAP: CALCULATION.
Collected records: blank.
CALCULATION or blank;
assignment not stated.
Not stated

Brief omits both the MAP formula and the rule for adding a record.
SBP = systolic blood pressure; DBP = diastolic blood pressure.
https://elong0527.github.io/yamaa/benchmark/adam-advs-map.html

Goal

  • Reduce human review effort.
  • Leverage organization templates.
  • Capture business rules that AI agents can apply.
  • Move human and AI agents collaboration at the planning stage.

yamaa: collaboration at the planning stage

Human and AI collaborate on a yamaa specification during planning.

Human and AI review the specification during planning; the engine executes the agreed rules.

yamaa: YAML specification example

schema_version: "1.0"
domain: ADSL
keys: [STUDYID, USUBJID]
input:
  SOURCE: input/adsl.csv

columns:
  - name: BMI
    type: float
    label: Body Mass Index (kg/m2)
    derivation:
      compute:
        expr: "WEIGHTKG / POWER(NULLIF(HEIGHTCM, 0) / 100, 2)"

verifications:
  - unique: [STUDYID, USUBJID]

yamaa: core principle

A yamaa specification has exactly one execution. Where it would have two, yamaa fails instead of choosing.

output_dataset = derive(input_datasets, spec)

The design of yamaa is a contract between human and AI agents.

  • spec: schema following closed vocabulary.
  • derive: business rules with derivation semantics.

yamaa: four components

Component Purpose Links
Schema Declares the vocabulary of the specification. Repository / Introduction
Rules Defines the meaning of each specification. Repository / Rules
Engine Runs specifications in Python, R and CLI consistently following the rules. Python / R / CLI
Benchmark Provides runnable specification examples. Repository / Dashboards

yamaa: inheritance

Inheritance reduces duplication while keeping every resolved study specification self-contained and reviewable. (example)

yamaa: benchmarks

~ 600
runnable benchmarks
3
execution interfaces (Python, R, CLI)

Scroll inside the frame to explore the spec, inputs, and golden output.

RConsortium Pilot 7: full studies

  • Build publicly available agentic data standardization workflow.
  • CDISC pilots (SDTM -> ADaM)
    • cdiscpilot01 and RConsortium submission pilots 3 and 5.
  • OpenClinica real sample data (ODM -> SDTM -> ADaM)
    • Real sample data for testing the full data journey.
  • Synthetic studies (ODM -> SDTM -> ADaM)
    • Simulate ODM XML from real clinical studies.

yamaa reproduces R Submission Pilot 5

Real size: 254 subjects
End-to-end time, 1 core
yamaa1.62 s
R (admiral)26.45 s
Peak memory
yamaa0.25 GB
R (admiral)0.60 GB
16×
faster than R
2.4×
less memory than R
Stress test: 10× subjects (2,540)
End-to-end time, 1 core
yamaa12.62 s
R (admiral)181.70 s
Peak memory
yamaa1.93 GB
R (admiral)4.13 GB
14×
faster than R
2.1×
less memory than R

Recommendations: Input and Output

  • Plain text is the input.
  • Annotated CRF is not an input but an output.
  • Excel specification is not an input but an output.
  • Communication costs more than tokens.

Summary of the workflow

  • Extract: enable regular EDC extraction in ODM format for each study to create a versioned source.
  • Draft: ask AI agent to draft the study spec from the organization or program template, adding only study differences.
  • Review: human and AI review study derivations and verifications in yamaa where fixes are cheap.
  • Finalize: approve and tag the resolved spec in version control with full lineage.
  • Automate: run the same spec on schedule in R or Python to produce identical SDTM and ADaM plus define.xml.

Note: this is an iterative process while the study is ongoing. Team members and AI collaborate centrally on the version controlled specification at the planning stage.

How to contribute

Benchmark to deployment

  • Benchmarks help assess an AI agent before adoption.
  • Deployment also requires onboarding the agent to a specific organization environment. Collaborating with keiji.ai to test synthetic clinical trial data.

Reference