Dataset Size Calculator

Test Set Size

How many samples to measure the final AUC with a specific precision.

Expected AUC 0.75

How well you think your model will perform (0.5 = random, 1.0 = perfect).

Margin of Error (±) 0.05

The precision you want. E.g., 0.05 means you want the true AUC to be within ±0.05 of your result.

Confidence Level 95%

How certain you want to be that the true AUC falls within your margin of error.

Positive Class % 50%

What fraction of your samples are 'positive' (e.g., diseased or target class).

Total Needed -

Positive -

Negative -

Validation Set Size

How many samples needed to reliably tell if Model A is truly better than Model B.

Expected AUC 0.75

Expected performance of your best model (0.5 = random, 1.0 = perfect).

Min. Meaningful Diff (Δ) 0.05

The smallest AUC difference that would make you choose one model over the other. E.g., 0.02 means you only care if one model is at least 2% better.

Detection Confidence 80%

The probability of correctly choosing the better model if the true difference is at least your 'Min. Meaningful Difference'.

Positive Class % 50%

What fraction of your samples are 'positive' (e.g., diseased or target class).

Total Needed -

Positive -

Negative -

ℹ️ Note on formulas: Based on the Hanley-McNeil variance estimator for the Wilcoxon-Mann-Whitney statistic (the gold standard for AUC). The validation set assumes a moderate correlation (ρ ≈ 0.5) between models evaluated on the same dataset, which roughly halves the required sample size compared to independent tests.