The other day, my friend told me that her math teacher’s favorite thing to say is: “If you take the average of a bunch of people who estimate the length of a cow, the outcome will be better than that of an expert.” (This is false don’t believe this and walk away!) I think that it’s implicit that the people can’t see the cow. Being tired and considering that I have a 4 minute passing period at my high school, I believed it and moved on. I mean after all, if you take the average of a million peoples’ guesses, the outcome should be more or less right, right? Doesn’t the scientific method say that accuracy increases through repetition?
Wrong. (The statement about the cow, not the scientific method.) As the title implies, the average of a bunch of garbage is still garbage. What, do you expect all the guesses to be perfectly symmetric about the actual length of the cow? I think my best resolution to this paradox (I think it’s a paradox maybe you aren’t so surprised) is that while accuracy does increase with more people, the accuracy increases more and more slowly and there’s a limit to how accurate you can get. The problem is also pretty ill-defined, since after all, isn’t an expert defined to be someone who gets very very nearly the right answer? You could call the first hit on Google for cow-length the expert answer, and maybe that helps convince you. When you start asking questions like “how will the average of a bunch of amateur cow-guessers do” (pretend that they exist!) it gets fuzzy.
(ADDENDUM: Another satisfying explanation is accuracy vs. precision. People are easily swayed by many factors, and so while your results may be precise, they’re not accurate. If I polled another million people in the same way, I would get a very close answer, but that doesn’t say anything about cows, it says something about people. Thanks dad.)
By the way, the scientific method does work of course. It’s not like you can ever say–well, let me just do this experiment really, really, really well and that’ll suffice. No, you do your best in every experiment and taking the average definitely increases accuracy.
Ironically enough, this fact can be applied to the act of people guessing about this very problem. If you ask a bunch of people what the answer is, they’ll probably say yes; especially if they’ve been swayed by public opinion. If you ask an expert like Feynman, however, you can’t go wrong. (By the way, Feynman’s example of this was when he was reviewing textbooks for the California Board of Education. He found that people gave good reviews to a textbook that was blank, because people took the average of the reviews they saw and passed it on to other people. A few good experts would be better in this scenario rather than polling tons of teachers and administrators. :D)
If you still don’t believe me (if there’s one thing to take away here it’s to always question a popular opinion) feel free to post your best argument for why it should be better in the comments section. And even better, if you don’t believe me try doing the experiment yourself 😉