The Enigma behind the Good–Turing formula

Abstract

Finding the total number of species in a population based on a finite sample is a difficult but practically important problem. In this snapshot, we will attempt to shed light on how during World War II, two cryptanalysts, Irving J. Good and Alan M. Turing, discovered one of the most widely applied formulas in statistics. The formula estimates the probability of missing some of the species in a sample drawn from a heterogeneous population. We will provide some intuition behind the formula, show its wide range of applications, and give a few technical details.