Ask the Wizard #227

There is no definitive point at which confidence is earned. It is a matter of degree. First, I would ask what is being tested for, and what the shooter estimates will happen. With any test there are two possible errors. A skilled shooter might fail, because of bad luck, or a random shooter might pass because of good luck. Of the two, I would prefer to avoid a false positive. I think a reasonable test would set the probability of a false negative at about 5%, and a false positive at about 1%.

For example, suppose the claimant says he can average one total of seven every seven throws of the dice. A random shooter would throw one seven every six throws, on average. By trial and error I find that a test meeting both these criteria would be to throw the dice 3,600 times, and require 547 or fewer sevens to pass, or one seven per 6.58 rolls.

A one in seven shooter should average 514.3 sevens, with a standard deviation of 21.00. Using the Gaussian approximation, the probability of such a skilled shooter throwing 548 or more sevens (a false negative) is 5.7%. A random shooter should average 600 sevens, with a standard deviation of 22.36. The probability of a random shooter passing the test (a false positive) is 0.94%. The graph below shows the possibe results for skilled and random shooters. If the results are to the left of the green line, then I would consider the shooter to have passed the test, and I would bet on him.

The practical dilemma is if we assume two throws per minute, it would take 30 hours to conduct the test. Perhaps I could be more liberal about the significance level, to cut down the time requirement, but the results would not be as convincing. I do think the time has come for a bigger test than the 500-roll Wong experiment.