Continuing the theme of ethical statistics from the previous two posts, I would be remiss if I did not mention one of the best books on the subject: The Mismeasure of Man, by Stephen Jay Gould. In the book, Gould aptly discusses flaws with measures of intelligence that have been used to justify social hierarchies such as racism and eugenics on a supposedly "scientific" basis. He spends about half of the book on craniometry--the measure of skulls--but gets to IQ in chapter five. A brief summary of that chapter, and some further reading on IQ and economic success, will illuminate our ongoing discussion.
The book is packed with interesting anecdotes of how scientific research is actually done in practice, and how researchers can unconsciously fall prey to their own biases. One such account concerns a project by Catherine M. Cox to analyze the IQ of great historic figures using their biographies. The project was chock full of methodological problems, the foremost of which was the availability data. A second problem was bias in the arithmetic: Cox had her five scorers start at a base IQ of 100 and add points as they were motivated by accounts of the subjects' precocity. As a result, there were no scores below 100 and all of the scores basically reflected the amount of available information about the individual.
Cox "measured" IQ at two time points: childhood (A1) and as young adults (A2). The result?
Cox published disturbingly low A1 IQ figures for some formidable characters, including Cervantes and Copernicus, both at 105. Her dossiers show the reason: little or nothing is known about their childhood, providing no data for addition to the base figure of 100. (p. 185)
What does this have to do with politics? It seems that IQ is still popping up as an explanatory variable, even in cross-national research. (Gould adeptly describes the problem with using measures on individuals as the basis for between-group comparisons, but that is beyond the scope of the current post.) Take a look at the figure below.
Garrett Jones, in The New Palgrave Dictionary of Economics, discusses this project optimistically:
Can IQ be measured across countries, even in developing countries? And if so, do these tests have similar real-world reliability to IQ tests given within OECD countries?The answer to both questions is yes, with some modest grounds for caution....The psychologist Richard Lynn and the political scientist Tatu Vanhanen (henceforth LV) assembled two collections of IQ scores by scouring the academic and practitioner literatures for reported IQ in a total of 113 countries (2002, 2006). They included some IQ standardisation samples and some national tests of mathematical ability, but most of the studies they used were ‘opportunity samples’, studies of an ostensibly typical classroom or school in a particular country.
The "cautions" that Jones mentions have to do with the reliability of the figures, but he accepts them because the IQ scores are correlated with GDP. If ever there were a case of begging the question, this appears to be it. What we have is a case of reification ("thing-ification") of IQ: "because we measure it, it must be real; because it correlates with what we think it should, it must be right."
Unfortunately, there is very little evidence that IQ actual measures anything innate. In fact, measured IQ has been going up "substantially and consistently, all over the world" since the tests started being administered in the first half of the twentieth century. (The phenomenon is known as the Flynn effect.) Scientists "correct" for this by normalizing scores to 100, hiding from the casual observer the fact that a measured IQ of 100 nowadays would be equivalent to 130 a century ago. There's more to say on the subject but this post is long enough already, so I will leave you with some additional reading and highly recommend Gould's book to anyone who works in quantitative science of any kind.
"Rising Scores on Intelligence Tests." Ulric Nelsser, American Scientist.
"Old Fashioned Play Builds Serious Skills." Alix Spiegel, NPR. (Money quote: "In fact, good executive function is a better predictor of success in school than a child's IQ." The measure of 'good executive function'? A child's ability to stand still. via @newsyc20)