The principle observable quantity is the empirical error rate () of a hypothesis. What is the distribution of the empirical error rate for a fixed hypothesis? For each example, we know that the probability that the hypothesis will err is given by true error rate, . This can be modeled by a biased coin flip: heads if you are wrong and tails if you are right.
Let us call the bias of the coin . What then is the probability of observing heads out of coin flips? This is a very familiar distribution in statistics called the Binomial distribution. Let be the observed rate of heads.
(Binomial Distribution) The Binomial distribution is given by: Here we use ’choose’ notation defined by .