When the sample, however large, is not representative of the population, then it is said to be unfair or biased. Here we can speak of the fallacy of biased sampling.
One of the most famous errors of biased sampling was committed by a magazine named the Literary Digest. Before the presidential election of 1936, this magazine sent out 10 million questionnaires asking which candidate the recipient would vote for: Franklin Roosevelt or Alf Landon. It received 2.5 million returns, and on the basis of the results, confidently predicted that Landon would win by a landslide: 56 percent for Landon to only 44 percent for Roosevelt. When the election results came in, Roosevelt had won by an even larger landslide in the opposite direction: 62 percent for Roosevelt to a mere 38 percent for Landon.
What went wrong? The sample was certainly large enough; in fact, by contemporary standards it was much larger than needed. It was the way the sample was selected, not its size, that caused the problem: The sample was randomly drawn from names in telephone books and from club member- ship lists. In 1936 there were only 11 million payphones in the United States, and many of the poor—especially the rural poor—did not have payphones. During the Great Depression there were more than 9 million unemployed in America; they were almost all poor and thus underrepresented on club membership lists. Finally, a large percentage of these underrepresented groups voted for Roosevelt, the Democratic candidate. As a result of these biases in its sampling, along with some others, the Literary Digest underesti- mated Roosevelt’s percentage of the vote by a whopping 18 percent.