HEOR Studio

The problem

Outcomes research on claims data runs on hand-written SAS that is slow to write, hard to review, and only as right as whoever wrote it. I wanted the model kept away from the code entirely: it drafts a spec a person reviews, and deterministic emitters write the code.

How it is built

The generated SQL itself runs in Postgres 16, in wasm, against a synthetic cohort whose answers I worked out by hand first as exact fractions (3 cases over 2,425 person-days, for example). Mutation tests corrupt the output on purpose to prove the checks can fail. A byte-identity snapshot over 2,721 emitted files catches a shared change that moves a number nobody touched.

1. ProtocolA study protocol goes in.
2. Reviewable specThe model drafts a JSON study spec. It never writes code.
3. Human sign-offgenerate_code won’t run without an explicit sign-off.
4. Deterministic emittersSAS and SQL twins from the approved spec.Fixed here
5. Run the SQLThe emitted SQL runs in Postgres 16 against hand-derived answers.

What it refuses

Some things it won’t do, and it says why before generating anything. Overall survival is refused because MarketScan’s only native death signal is an in-hospital discharge status, masked from data year 2016, so the curve would quietly mean something else. Greedy nearest-neighbour matching is refused because a different row order gives a different estimate.

The human gate

Its generate_code tool won’t run without an explicit sign-off and every code list marked verified, so a host model can’t talk its way to an artifact.

Scope

20 analysis modules plus attrition and Table 1. Measured against 48 published MarketScan studies: 29 are partially expressible today, 19 are not, and none is fully expressible. The test data is synthetic, not real claims.

What broke, and the fix

Other things that broke

• Thirteen mutations across nine waves were weaker than they looked, because a replace without the global flag corrupted only the first occurrence. No emitted number was wrong, but each check had less behind it than the report said.
• A coverage guard had been passing vacuously for three modules that shipped that way.
• I shipped a false claim about propensity weighting: that the weighted populations agree only if every cell holds both arms. A case with seven cells, four single-arm, and a gap of exactly zero proved the “only if” wrong, and I corrected it everywhere it appeared.

Run it here

The incidence arithmetic in real Postgres, in this tab. No HEOR Studio code runs here.