Centered vs Reparam — Side-by-Side Comparison Tables¶

Two tables comparing the published centered Aladynoulli numbers (Tables S6 and S8 in the Nature submission) against the reparam (nokappa-v3) re-runs on the same diseases / horizons.

Sources¶

Centered (published, paper image transcribed):

  • Table S6: 1-yr Baseline (single pi at enrollment), 1-yr Median (rolling, 10K), 10-yr (best of static/dynamic)
  • Table S8: 10-yr Dynamic Rolling Interpolation (10 offset-trained models stitched)

Reparam (nokappa-v3, feb18 LOO pipeline):

  • Static 1-yr: claudefile/results_feb18_complete/static_1yr_results_400k.csv
  • Rolling 1-yr median: claudefile/results_feb18_complete/rolling_1yr_results_10k.csv (median across offsets)
  • 10-yr (S6 best): tableS6_side_by_side.csv Reparam_10yr_best_auc (max of static_10yr and dynamic_10yr from FULL pi)
  • 10-yr S8 rolling: dynamic_rolling_10yr_results_10k_reparam.csv (this run, batch 0)

Output CSVs (already saved)¶

  • claudefile/results_feb18_complete/tableS6_1yr_centered_vs_reparam.csv
  • claudefile/results_feb18_complete/tableS6_S8_centered_vs_reparam.csv

Caveat to know going in¶

Reparam improves AUC for 24–25 of 28 diseases on every horizon, but three diseases regress: Multiple Sclerosis, Prostate Cancer, Bladder Cancer. The mechanism (per MEMORY.md) is the known signature-assignment instability of non-centered MAP for low-prevalence / small-cell signatures — psi correlation between centered and reparam is ~0.76, so a few signatures shuffle. The published paper uses the centered model; these comparison tables document the reparam refinement for supplementary / future use.

In [1]:
import pandas as pd
import numpy as np
from pathlib import Path

ROOT = Path('/Users/sarahurbut/aladynoulli2/claudefile/results_feb18_complete')

df_1yr = pd.read_csv(ROOT / 'tableS6_1yr_centered_vs_reparam.csv')
df_10yr = pd.read_csv(ROOT / 'tableS6_S8_centered_vs_reparam.csv')

print(f'1-yr comparison: {len(df_1yr)} diseases')
print(f'10-yr comparison: {len(df_10yr)} diseases')

1-yr comparison: 28 diseases
10-yr comparison: 28 diseases

Table 1 — 1-year predictions¶

1-yr (Baseline) = static prediction at enrollment, full 400K cohort.
1-yr (Median) = rolling 1-yr median across age offsets, 10K subset.

In [2]:
from IPython.display import display

def style_1yr(df):
    def color_delta(v):
        if pd.isna(v): return ''
        if v < -0.01: return 'color: #c8102e; font-weight: bold;'   # regression in red
        if v > 0.01:  return 'color: #2e7d32;'                       # improvement in green
        return ''
    fmt = {c: '{:.3f}' for c in df.columns if df[c].dtype.kind == 'f'}
    return df.style.format(fmt).map(color_delta, subset=['1yr_enroll_delta', '1yr_median_delta'])

display(style_1yr(df_1yr))

# Means / wins
print(f'\n1-yr Baseline:  centered mean = {df_1yr["1yr_enroll_cent"].mean():.3f}, '
      f'reparam mean = {df_1yr["1yr_enroll_rep"].mean():.3f}, '
      f'mean delta = {df_1yr["1yr_enroll_delta"].mean():+.3f}, '
      f'reparam wins {(df_1yr["1yr_enroll_delta"] > 0).sum()}/{len(df_1yr)}')
print(f'1-yr Median:    centered mean = {df_1yr["1yr_median_cent"].mean():.3f}, '
      f'reparam mean = {df_1yr["1yr_median_rep"].mean():.3f}, '
      f'mean delta = {df_1yr["1yr_median_delta"].mean():+.3f}, '
      f'reparam wins {(df_1yr["1yr_median_delta"] > 0).sum()}/{len(df_1yr)}')

print('\nDiseases where reparam regresses on 1-yr:')
regress_1yr = df_1yr[(df_1yr['1yr_enroll_delta'] < 0) | (df_1yr['1yr_median_delta'] < 0)].copy()
display(regress_1yr[['Disease', '1yr_enroll_cent', '1yr_enroll_rep', '1yr_enroll_delta',
                     '1yr_median_cent', '1yr_median_rep', '1yr_median_delta']]
        .style.format({c: '{:.3f}' for c in regress_1yr.columns if regress_1yr[c].dtype.kind == 'f'}))
  Disease 1yr_enroll_cent 1yr_enroll_rep 1yr_enroll_delta 1yr_median_cent 1yr_median_rep 1yr_median_delta
0 ASCVD 0.881 0.929 0.048 0.879 0.937 0.058
1 All_Cancers 0.753 0.794 0.041 0.757 0.777 0.020
2 Anemia 0.648 0.827 0.179 0.690 0.859 0.169
3 Anxiety 0.604 0.778 0.174 0.639 0.891 0.252
4 Asthma 0.690 0.892 0.202 0.702 0.893 0.191
5 Atrial_Fib 0.797 0.895 0.098 0.801 0.918 0.117
6 Bipolar_Disorder 0.758 0.841 0.083 0.758 0.870 0.112
7 Bladder_Cancer 0.825 0.772 -0.053 0.891 0.804 -0.087
8 Breast_Cancer 0.782 0.827 0.045 0.867 0.896 0.029
9 CKD 0.651 0.755 0.104 0.760 0.861 0.101
10 COPD 0.736 0.856 0.120 0.738 0.871 0.133
11 Colorectal_Cancer 0.825 0.899 0.074 0.848 0.921 0.073
12 Crohns_Disease 0.896 0.948 0.052 0.930 0.941 0.011
13 Depression 0.616 0.833 0.217 0.647 0.882 0.235
14 Diabetes 0.741 0.857 0.116 0.787 0.849 0.062
15 Heart_Failure 0.769 0.885 0.116 0.811 0.908 0.097
16 Lung_Cancer 0.699 0.766 0.067 0.784 0.850 0.066
17 Multiple_Sclerosis 0.840 0.767 -0.073 0.902 0.784 -0.118
18 Osteoporosis 0.756 0.866 0.110 0.767 0.846 0.079
19 Parkinsons 0.809 0.887 0.078 0.796 0.897 0.101
20 Pneumonia 0.634 0.761 0.127 0.750 0.889 0.139
21 Prostate_Cancer 0.831 0.720 -0.111 0.828 0.664 -0.164
22 Psoriasis 0.607 0.810 0.203 0.640 0.783 0.143
23 Rheumatoid_Arthritis 0.749 0.881 0.132 0.801 0.943 0.142
24 Secondary_Cancer 0.600 0.731 0.131 0.683 0.792 0.109
25 Stroke 0.653 0.732 0.079 0.674 0.743 0.069
26 Thyroid_Disorders 0.678 0.829 0.151 0.668 0.861 0.193
27 Ulcerative_Colitis 0.816 0.917 0.101 0.809 0.935 0.126 
1-yr Baseline:  centered mean = 0.737, reparam mean = 0.831, mean delta = +0.093, reparam wins 25/28
1-yr Median:    centered mean = 0.772, reparam mean = 0.859, mean delta = +0.088, reparam wins 25/28

Diseases where reparam regresses on 1-yr:
  Disease 1yr_enroll_cent 1yr_enroll_rep 1yr_enroll_delta 1yr_median_cent 1yr_median_rep 1yr_median_delta
7 Bladder_Cancer 0.825 0.772 -0.053 0.891 0.804 -0.087
17 Multiple_Sclerosis 0.840 0.767 -0.073 0.902 0.784 -0.118
21 Prostate_Cancer 0.831 0.720 -0.111 0.828 0.664 -0.164

Table 2 — 10-year predictions¶

S6 (10-yr) = best of static and dynamic 10-yr from the single enrollment-trained pi tensor, full 400K cohort.
S8 (10-yr Rolling) = year-by-year stitched predictions using 10 offset-trained models, 10K subset (offsets 0–9).

In [3]:
def style_10yr(df):
    def color_delta(v):
        if pd.isna(v): return ''
        if v < -0.01: return 'color: #c8102e; font-weight: bold;'
        if v > 0.01:  return 'color: #2e7d32;'
        return ''
    fmt = {c: '{:.3f}' for c in df.columns if df[c].dtype.kind == 'f'}
    return df.style.format(fmt).map(color_delta, subset=['S6_delta', 'S8_delta'])

display(style_10yr(df_10yr))

print(f'\nS6 (10-yr best):       centered mean = {df_10yr["S6_centered"].mean():.3f}, '
      f'reparam mean = {df_10yr["S6_reparam"].mean():.3f}, '
      f'mean delta = {df_10yr["S6_delta"].mean():+.3f}, '
      f'reparam wins {(df_10yr["S6_delta"] > 0).sum()}/{len(df_10yr)}')
print(f'S8 (10-yr rolling):    centered mean = {df_10yr["S8_centered"].mean():.3f}, '
      f'reparam mean = {df_10yr["S8_reparam"].mean():.3f}, '
      f'mean delta = {df_10yr["S8_delta"].mean():+.3f}, '
      f'reparam wins {(df_10yr["S8_delta"] > 0).sum()}/{len(df_10yr)}')

print('\nDiseases where reparam regresses on 10-yr:')
regress_10yr = df_10yr[(df_10yr['S6_delta'] < 0) | (df_10yr['S8_delta'] < 0)].copy()
display(regress_10yr[['Disease', 'S6_centered', 'S6_reparam', 'S6_delta',
                      'S8_centered', 'S8_reparam', 'S8_delta']]
        .style.format({c: '{:.3f}' for c in regress_10yr.columns if regress_10yr[c].dtype.kind == 'f'}))
  Disease S6_centered S6_reparam S6_delta S8_centered S8_reparam S8_delta
0 ASCVD 0.733 0.757 0.024 0.836 0.893 0.057
1 All_Cancers 0.674 0.716 0.042 0.735 0.766 0.031
2 Anemia 0.588 0.626 0.038 0.658 0.785 0.127
3 Anxiety 0.514 0.535 0.021 0.573 0.777 0.204
4 Asthma 0.529 0.546 0.017 0.612 0.767 0.155
5 Atrial_Fib 0.707 0.756 0.049 0.781 0.873 0.092
6 Bipolar_Disorder 0.492 0.535 0.043 0.624 0.697 0.073
7 Bladder_Cancer 0.708 0.742 0.034 0.850 0.790 -0.060
8 Breast_Cancer 0.554 0.590 0.036 0.767 0.776 0.009
9 CKD 0.708 0.738 0.030 0.737 0.806 0.069
10 COPD 0.658 0.697 0.039 0.715 0.816 0.101
11 Colorectal_Cancer 0.648 0.655 0.007 0.791 0.850 0.059
12 Crohns_Disease 0.580 0.598 0.018 0.737 0.836 0.099
13 Depression 0.484 0.538 0.054 0.546 0.765 0.219
14 Diabetes 0.651 0.731 0.080 0.725 0.790 0.065
15 Heart_Failure 0.701 0.758 0.057 0.779 0.857 0.078
16 Lung_Cancer 0.669 0.680 0.011 0.741 0.789 0.048
17 Multiple_Sclerosis 0.591 0.472 -0.119 0.690 0.518 -0.172
18 Osteoporosis 0.681 0.664 -0.017 0.707 0.786 0.079
19 Parkinsons 0.724 0.736 0.012 0.787 0.827 0.040
20 Pneumonia 0.644 0.679 0.035 0.740 0.838 0.098
21 Prostate_Cancer 0.687 0.700 0.013 0.786 0.670 -0.116
22 Psoriasis 0.546 0.597 0.051 0.508 0.650 0.142
23 Rheumatoid_Arthritis 0.608 0.634 0.026 0.707 0.832 0.125
24 Secondary_Cancer 0.610 0.622 0.012 0.664 0.745 0.081
25 Stroke 0.681 0.704 0.023 0.663 0.702 0.039
26 Thyroid_Disorders 0.594 0.559 -0.035 0.637 0.746 0.109
27 Ulcerative_Colitis 0.583 0.620 0.037 0.793 0.826 0.033 
S6 (10-yr best):       centered mean = 0.627, reparam mean = 0.649, mean delta = +0.023, reparam wins 25/28
S8 (10-yr rolling):    centered mean = 0.710, reparam mean = 0.778, mean delta = +0.067, reparam wins 25/28

Diseases where reparam regresses on 10-yr:
  Disease S6_centered S6_reparam S6_delta S8_centered S8_reparam S8_delta
7 Bladder_Cancer 0.708 0.742 0.034 0.850 0.790 -0.060
17 Multiple_Sclerosis 0.591 0.472 -0.119 0.690 0.518 -0.172
18 Osteoporosis 0.681 0.664 -0.017 0.707 0.786 0.079
21 Prostate_Cancer 0.687 0.700 0.013 0.786 0.670 -0.116
26 Thyroid_Disorders 0.594 0.559 -0.035 0.637 0.746 0.109

Summary¶

Horizon Centered mean Reparam mean Δ Wins (reparam)
1-yr Baseline (400K) 0.737 0.831 +0.093 25/28
1-yr Median (rolling, 10K) 0.772 0.859 +0.088 25/28
10-yr S6 best (400K) 0.627 0.649 +0.023 24/28
10-yr S8 rolling (10K) 0.710 0.778 +0.067 25/28

Three consistent regressors across all horizons:

  • Multiple Sclerosis — large drop on every horizon (−0.07 to −0.17). Lowest-prevalence signature in the panel; psi-assignment instability under reparam.
  • Prostate Cancer — drops at both 1-yr horizons and S8 (−0.11 to −0.16). 10-yr S6 essentially unchanged (+0.01).
  • Bladder Cancer — drops at both 1-yr horizons and S8 (−0.05 to −0.09). 10-yr S6 actually improves.

Conclusion. Reparam materially improves prediction for the headline diseases (ASCVD, T2D, HF, AF, IBD, cancers in aggregate) but creates three documented regressions that a careful reader of the per-disease panel will notice. The published Nature paper retains the centered Aladynoulli model; these tables are kept for supplementary documentation / future work on the reparam refinement.