Math

Ninety-five times out of a hundred

Inspect the observations behind a confidence interval. Repeat the experiment and compare coverage and width of methods on the same samples.

By Mykhailo Makarov

Actual application image

What happens

Inspect the observations behind a confidence interval. Repeat the experiment and compare coverage and width of methods on the same samples. The nominal level describes long-run coverage under the method’s assumptions. It is not a probability for this realized interval or a guarantee of exactly 95 hits in 100. Finding a miss counts every intervening sample too.

Equations used in the module

Equations
mean: m ± t_{1−α/2, n−1} · s/√n
proportion: Wald p ± z√(p(1−p)/n);  Wilson [p + z²/2n ± z√(p(1−p)/n + z²/4n²)]/(1 + z²/n)
coverage = fraction of intervals that contain the truth
Student's t via the incomplete beta

Related experiments

Back to catalogue

Math

The p-value, and how to fool yourself with it

Two thousand experiments ask whether a mean is zero; the share that rejects at α = 0.05 is counted. With no effect it is 5 % - the type I rate - and the p-values are uniform. With an effect it is the power, which rises with the effect and with n: half a σ is found 32 % of the time with ten observations and 82 % with thirty-four. Look at the data every five observations and stop when p dips below 0.05, and the false positive rate is a quarter. Everything is counted; Student's curve comes from the code.

Math

Three answers people refuse

Monty Hall: switching wins 2/3 - and only 1/2 if the host did not know where the prize was, though he opened the same door. The two children: 'at least one is a boy' makes two boys 1/3, 'the older is a boy' 1/2, 'a boy born on a Tuesday' 13/27. Bertrand's chord: a random chord is longer than the triangle's side with probability 1/3, 1/2 or 1/4, depending on what 'random' means. Every answer is an exact fraction and a count over twenty thousand seeded trials, and the picture shows the procedure that produced it.

Math

Every distribution from the uniform

There are only a few ways to make a random number with a given distribution, and all of them start from a uniform. Push it through the inverse of the CDF (exponential, Cauchy, triangular). Take two and turn them into a pair of normals (Box-Muller). Throw darts at a box around the density and keep the ones under the curve (rejection - and the share kept is π/4 for the semicircle). Count exponential waits in a unit of time (Poisson) or heads in ten coins (binomial). Follow one numbered token through the machine, then hold the whole sample against its exact curve.