MK

arXiv preprint · 2026 · Interactive companion

Memorisation bias in medical AI

Medical AI models are routinely deployed on the same population from which their training data was sourced. This study shows that when a patient’s historical records are part of a model’s training data, the model’s predictions on that patient’s future records shift towards their earlier health state, an effect termed memorisation bias. The bias can persist for decades and can lead to missed diagnoses.

M. A. Knolle, M. J. Menten, L. Lux, M. Roschewitz, E. A. M. Stanley, G. Kaissis, D. Rueckert, B. Glocker · Read the paper · Code

Background

Methodology

The study measures memorisation bias by comparing models that did and did not see a patient’s earlier records. The walkthrough below explains each step, using Alice as the example.

  1. 1Split each patient’s records in time

    Each patient’s records are split temporally. Earlier records form the historical set, which can be used for training. Later records form the future set, which is never used for training or model selection. Alice has three historical ECGs with a normal healthy rhythm and a new one showing a heart attack.

  2. 2Train many models on random subsets

    The study trains 200 models, each on the historical records of a random half of the patients. Some models see Alice’s historical records and some do not. A blue corner marks the ones that do.

  3. 3Partition the models for each patient

    For each patient, the models fall into two groups: 100 trained on that patient’s historical records (IN) and 100 not trained on them (OUT). Apart from this one patient, both groups saw equally random data, so any systematic difference between them is due to the patient’s own records.

  4. 4Predict on the future record

    Both groups predict on Alice’s future ECG, which no model has seen. Here the IN models output a lower probability of infarct than the OUT models. The difference between the two average predictions, for the condition where it is largest, is the size of the shift. Alice’s numbers are illustrative.

  5. 5Test whether the difference is significant

    An energy-distance test compares the IN and OUT predictions as whole distributions. Shuffling the IN and OUT labels shows how large the distance gets by chance alone. Alice’s distance lies beyond every shuffle. The study uses 100 models per group.

  6. 6Correct for testing many records

    Each dataset means hundreds of thousands of tests, so some records would pass p < 0.05 by chance alone, about 5% of those without any shift. The study therefore applies the Benjamini–Hochberg correction. It sorts all p-values and compares each one with a threshold that rises with its rank, which keeps the expected share of false discoveries among the flagged records at 5%. The p-values shown here are illustrative.

  7. 7Significant shifts across all datasets

    After correction, a small share of future records in each of the four datasets shows a significant shift, from 0.6% to 4.7% depending on the dataset.

  8. 8Check against chance

    As an additional control, the models are split into two random groups that ignore which patients they were trained on. Repeating every test with these groups finds no significant records in any dataset, so the shifts are due to memorisation and not random predictive variation between models.

The four datasets

01 · Future predictions shift

Training on a patient’s historical records alters their future predictions

In all four datasets, a share of future records showed significantly different predictions depending on whether the patient’s historical records were in the training data. Most shifts are small, but some exceed 70 percentage points. On the historical records themselves, the ones used for training, the effect is far more common and more pronounced.

Future records with a significant shift

–

 

Historical training records with a significant shift

–

 

Randomly split models (control)

–

records with a significant shift

20
Share of records whose average predicted probability, for at least one condition, differs between IN and OUT models by at least the amount on the horizontal axis. The vertical scale is logarithmic. Drag the slider or click the chart to set a size.

Examples from the data

Single future records with a significant shift from the selected dataset. Only aggregate test results are shown; no patient data.

Model choice matters

MIMIC-IV-ED is tabular, so the study compared three model types, each tuned for the best validation performance. The share of future records affected differs by orders of magnitude.

02 · Persistence

Memorisation bias fades with time, but can persist for decades

The longer the gap between a patient’s most recent training record and their new record, the rarer and weaker the shift. It does not disappear: in HEEDB, a dataset of electrocardiograms collected from the 1980s to 2025, significant shifts appear on records acquired more than 35 years after the patient’s last training record.

Longest gap with a significant shift

–

 

Longest follow-up in the dataset

–

between last training record and a future record

Share of future records with a significant shift, grouped by the time between the patient’s most recent historical training record and the future record (the acquisition interval). Labels give the number of affected records.
Show as table

03 · Diagnostic accuracy

Data contributors returning with a new condition are at elevated risk of missed diagnoses

The study simulated deployment: fixed decision thresholds turn each model’s output into a yes/no call, and the calls of IN and OUT models are compared for patients with a significant shift. The effect is asymmetric. When a patient returns with a condition absent from all their training records, sensitivity drops, so the condition is missed more often. When their health is unchanged, both sensitivity and specificity are artificially inflated, which makes the model look better than it is for these patients.

 
Show as table

04 · Mitigation

Patient-level differential privacy prevents memorisation bias; record-level does not

Differential privacy (DP) adds calibrated noise during training to limit how much any single unit of data can affect the model. The unit matters. With record-level DP, the variant most used in practice, significant shifts on future records remain even at the strongest protection tested (ε = 1). With patient-level DP, which protects all of a patient’s records together, at most two future records are affected at any budget, at some cost in diagnostic performance.

Records with a significant shift

Diagnostic performance (test AUROC)

Smaller ε means stronger protection; “none” is the matching non-private model. The count scale is logarithmic, with zero shown on the bottom line. Performance is the mean ± s.d. of the macro-average AUROC across 200 models. These models are smaller than in the main experiments (1.9 million parameters), for computational reasons.
Show as table

About the data

All four datasets are credentialed clinical data (MIMIC-ECG, MIMIC-CXR and MIMIC-IV-ED from PhysioNet, and HEEDB). In line with their data use agreements, this page shows only aggregate statistics and test results: no ECG traces, images, record contents or identifiers. The shift of a record is the largest absolute difference, across conditions, between the average predicted probabilities of IN and OUT models; its direction is not shown.

Significance uses energy-distance tests with Benjamini–Hochberg correction (false discovery rate 5%). Diagnostic comparisons use exact permutation tests with Bonferroni correction across the 82 comparisons in Fig. 4 of the paper; for HEEDB, the ten conditions with the most new cases are shown. One HEEDB record with an implausible 85-year interval is excluded, as in the paper. Illustrations are adapted from Figs. 1 and 2 of the paper.