Statistical Thinking and the Birth of Modern Computing

John von Neumann and the IAS computer, 1945

John von Neumann and the IAS computer, 1945

What do fighter pilots, casinos, and streetlights all have in common? These three disparate topics are all the subject of statistical thinking that led to (and benefitted from) the development of modern computing. This process is described in Turing’s Cathedral by George Dyson, from which most of the quotes below are drawn. Dyson’s book focuses on Alan Turing far less than the title would suggest, in favor of John von Neumann’s work at the Institute for Advanced Studies (IAS). Von Neumann and the IAS computing team are well-known for building the foundation of the digital world, but before Turing’s Cathedral I was unaware of the deep connection with statistics.

Statistical thinking first pops up in the book with Julian Bigelow’s list of fourteen “Maxims for Ideal Prognosticators” for predicting aircraft flight paths on December 2, 1941. Here is a subset (p. 112):

7. Never estimate what may be accurately computed.

8. Never guess what may be estimated.

9. Never guess blindly.

This early focus on estimation will reappear in a moment, but for now let’s focus on the aircraft prediction problem. With the advent of radar it became possible for sorties at night or in weather with poor visibility. In a dark French sky or over a foggy Belgian city it could be tough to tell who was who until,

otherwise adversarial forces agreed on a system of coded signals identifying their aircraft as friend or foe. In contrast to the work of wartime cryptographers, whose job was to design codes that were as difficult to understand as possible, the goal of IFF [Identification Friend or Foe] was to develop codes that were as difficult to misunderstand as possible…. We owe the existence of high-speed digital computer to pilots who preferred to be shot down intentionally by their enemies rather than accidentally by their friends. (p. 116)

In statistics this is known as the distinction between Type I and Type II errors, which we have discussed before. Pilots flying near their own lines likely figured there was a greater probability that their own forces would make a mistake than that the enemy would detect them–and going down as a result of friendly fire is no one’s idea of fun. This emergence of a cooperative norm in the midst of combat is consistent with stories from other conflicts in which the idea of fairness is used to compensate for the rapid progress of weapons technology.

casino-monte-carlo-roulette-monaco-1Chapter 10 of the book (one of my two favorites along with Chapter 9, Cyclogenesis) is entitled Monte Carlo. Statistical practitioners today use this method to simulate statistical distributions that are analytically intractable. Dyson weaves the development of Monte Carlo in with a recounting how von Neumann and his second wife Klari fell in love in the city of the same name. A full description of this method is beyond the scope of this post, but here is a useful bit:

Monte Carlo originated as a form of emergency first aid, in answer to the question: What to do until the mathematician arrives? “The idea was to try out thousand of such possibilities and, at each stage, to select by chance, by means of a ‘random number’ with suitable probability, the fate or kind of event, to follow it in a line, so to speak, instead of considering all branches,” [Stan] Ulam explained. “After examining the possible histories of only a few thousand, one will have a good sample and an approximate answer to the problem.”

For a more comprehensive overview of this development in the context of Bayesian statistics, check out The Theory That Would Not Die.

The third and final piece of the puzzle for our post today is the well-known but not sufficiently appreciated distinction between correlation and causation. Philip Thompson, a meteorologist who joined the IAS group in 1946, learned this lesson at the age of 4 and counted it as the beginning of his “scientific education”:

[H]is father, a geneticist at the University of Illinois, sent him to post a letter in a mailbox down the street. “It was dark, and the streetlights were just turning on,” he remembers. “I tried to put the letter in the slot, and it wouldn’t go in. I noticed simultaneously that there was a streetlight that was flickering in a very peculiar, rather scary, way.” He ran home and announced that he had been unable to mail the letter “because the streetlight was making funny lights.”

Thompson’s father seized upon this teachable moment, walked his son back to the mailbox and “pointed out in no uncertain terms that because two unusual events occurred at the same time and at the same place it did not mean that there was any real connection between them.” Thus the four-year-old learned a lesson that many practicing scientists still have not. This is also the topic of Chapter 8 of How to Lie with Statistics and a recent graph shared by Cory Doctorow.

The fact that these three lessons on statistical thinking coincided with the advent of digital computing, along with a number of other anecdotes in the book, impressed upon me the deep connection between these two fields of thought. Most contemporary Bayesian work would be impossible without computers. It is also possible that digital computing would have come about much differently without an understanding of probability and the scientific method.

Micro-Institutions Everywhere: Species and Regime Types

geeks_evolveIn a two-for-one example of micro-institutions, Jay Ulfelder blogs this paragraph from a paper Ian Lustick:

One might naively imagine that Darwin’s theory of the “origin of species” to be “only” about animals and plants, not human affairs, and therefore presume its irrelevance for politics. But what are species? The reason Darwin’s classic is entitled Origin of Species and not Origin of the Species is because his argument contradicted the essentialist belief that a specific, finite, and unchanging set of categories of kinds had been primordially established. Instead, the theory contends, “species” are analytic categories invented by observers to correspond with stabilized patterns of exhibited characteristics. They are no different in ontological status than “varieties” within them, which are always candidates for being reclassified as species. These categories are, in essence, institutionalized ways of imagining the world. They are institutionalizations of difference that, although neither primordial nor permanent, exert influence on the futures the world can take—both the world of science and the world science seeks to understand. In other words, “species” are “institutions”: crystallized boundaries among “kinds”, constructed as boundaries that interrupt fields of vast and complex patterns of variation. These institutionalized distinctions then operate with consequences beyond the arbitrariness of their location and history to shape, via rules (constraints on interactions), prospects for future kinds of change.

Jay follows this up with an interesting analogy to political regime types–the “species” that political scientists study:

Political regime types are the species of comparative politics. They are “analytic categories invented by observers to correspond with stabilized patterns of exhibited characteristics.” In short, they are institutionalized ways of thinking about political institutions. The patterns they describe may be real, but they are not essential. They’re not the natural contours of the moon’s surface; they’re the faces we sometimes see in them.

I have no comment other than that I think Jay is right, and it reminds me of a Robert Sapolsky lecture on the dangers of categorical thinking. And yes, Sapolsky is a biologist. We’ll go right to the best part (19:40-22:05) but the whole lecture is worth watching:

What is the Future of Publishing?

Today’s journal publishing system is the best possible. If you limit yourself to 17th century technology, that is.

Quips like these were sprinkled throughout Jason Priem’s presentation on altmetrics at Duke on Monday. Altmetrics is short for “alternative metrics,” or ways of measuring the impact of a particular author or article rather than the canonical impact factor of journals (which, it turns out, was initially resisted; Thomas Kuhn FTW).

Priem is a doctoral candidate at UNC, and recently started a site called ImpactStory. According to the LSE blog:

ImpactStory is a relaunched version of total-impact. It’s a free, open-source webapp we’ve built (thanks to a generous grant by the Sloan Foundation and others) to help researchers tell these data-driven stories about their broader impacts. To use ImpactStory, start by pointing it to the scholarly products you’ve made: articles from Google Scholar Profiles, software on GitHub, presentations on SlideShare, and datasets on Dryad (and we’ve got more importers on the way).

Then we search over a dozen Web APIs to learn where your stuff is making an impact. Instead of the Wall Of Numbers, we categorize your impacts along two dimensions: audience (scholars or the public) and type of engagement with research (view, discuss, save, cite, and recommend).

Priem’s presentation was informative and engaging. He has clearly spent a good deal of time thinking about academic publishing, and about the scientific undertaking more generally. I particularly liked how he responded to some tough audience questions about potential for gaming the system by re-iterating that we do not want a “Don Draper among the test tubes,” but for better or worse the way that we communicate our ideas makes a difference in how they are received.

If you are interested in hearing more of Jason’s ideas, here is a video of a similar talk he gave at Purdue earlier this year. The altmetrics portion starts around the 25-minute mark.

My Ten Favorite Posts from the Past Year

As promised yesterday, here are my top ten favorite posts from the first year of YSPR. They are arranged chronologically.

Addiction in The English Opium Eater

Thoughts on Public Enemies

Iraq Casualties and Public Opinion, 2003

Lessons from Moneyball

Casinos as Institutions

Moneyball Roundup

Problems with Science Statistics Aren’t New

Traffic Circles and Safety

Wednesday Nerd Fun: The Game of 99

Micro-Institutions Everywhere: Crime Bosses

Do you have a favorite post? Is there something you would like to see on YSPR that you haven’t yet? Put that comments button to good use.

Profile of a Conflict Statistician

BALL IS 46, STOCKY, SHORT, and bearded, with glasses and reddish-brown hair, which he used to wear in a ponytail. His manner is mostly endearing geek. But he is also an evangelist, a true believer in the need to get history right, to tell the truth about suffering and death. Like all evangelists, he can be impatient with people who do not share his priorities; his difficulty suffering fools (a large group, apparently) does not always help his cause….

He first applied statistics to human rights in 1991 in El Salvador. The U.N. Commission on the Truth for El Salvador arose at an auspicious moment — the new practice of collecting comprehensive information about human rights abuses coincided with advances in computing that allowed people with ordinary personal computers to organize and use the data. Statisticians had long done work on human rights — people like William Seltzer, the former head of statistics for the United Nations, and Herb Spirer, a professor and mentor to almost everyone in the field today, had helped organizations choose the right unit of analysis, developed ways to rank countries on various indices, and figured out how to measure compliance with international treaties. But the problem of counting and classifying mass testimony was new.

Ball, working for a Salvadoran human rights group, had started producing statistical summaries of the data the group had collected. The truth commission took notice and ended up using Ball’s model. One of its analyses plotted killings by time and military unit. Killings could then be compared with a list of commanders, making it possible to identify the military officers responsible for the most brutality.

“El Salvador was a transformational moment, from cases, even lots of them, to mass evidence,” Ball says. “The Salvadoran commission was the first one to understand that many, many voices could speak together. After that, every commission had to do something along these lines.”

That’s an excerpt Foreign Policy’s “The Body Counter,” and it’s worth reading in full, especially if you enjoyed this post.

Visualization Basics: Japanese Multiplication

Data visualization became very popular in 2011, as evidenced by NYT pieces like this one and the release of Nathan Yau‘s book Visualize This. It seems to me that the upper limit of the amount of information a dataviz/infographic/pick-your-term can convey is bounded by three things: the creativity of the designer, technology available to him/her, and the perceptibility of the viewer. But is there an optimal point where simplicity of design and information conveyed are both maximized?

For one answer to this question, consider multiplication. In most (all?) American schools that I know of, multiplication is taught in terms of area (two terms) or volume (three terms). Harvard’s Stats 110 begins by teaching probability as area. This concept is simple enough, and is particularly handy because often what we care about in practical terms can be expressed as an area/volume: how much wallpaper do I need? how much water will fit in that bucket?

But in terms of just manipulating the numbers, the area/volume interpretation can be a bit clunky–it doesn’t really save any steps, and once you have more digits than you can hold in your head, most people will reach straight for a calculator. There’s nothing wrong with that, except that there are many applications of multiplication beyond area or volume (take total cost of large order as an example). The Japanese have a different method, as the video below shows.

Two characteristics readily recommend this method. First, it is a very basic visualization that, if practiced, seems like it could make multiplication of large numbers in your head simpler. Second, there is no strict interpretive paradigm imposed on the answer. The practical meaning of the answer need not be an area, or anything geometric at all. However, it is not clear how this would extend beyond three terms either.

It may be that readers more experienced than myself with linear algebra or geometry will sense intuitively how this works. If you have a straightforward explanation, please share it in the comments. (h/t @brainpicker)

Einstein and Reality

Source: flickr user mansionwb

David Duff had some thoughtful comments on my post about Stephen Jay Gould’s arguments in The Mismeasure of Man about whether IQ is a “real thing” or just the result of measurement. I will provide further illustration of what I meant in that post, and then share some thoughts from a biography of Einstein that speak to David’s last sentence. (“This is not reification, this is normal science.”)

To show that creating a numeric index does not necessitate an underlying reality, suppose I created a “health-per-wealth” index to determine whether someone ate sufficiently healthy for a member of their social class. To create my index, we count the number of different types of vegetables in someone’s refrigerator and divide that by the number of walls in their home. My own number right now is an embarrassingly low 0.16. Now, has this measurement told me anything about how healthy I am relative to other members of the population? Not really. Nor would adding more data on other individuals help to do so. The measure itself is flawed, relying on two (very) imperfect proxies for real underlying characteristics. Most people would agree that health and wealth are real concept, but we learn very little about them from shopping habits and architecture. Likewise, intelligence seems to me a real enough “thing,” but I am not convinced that IQ tests are an accurate measure, nor that any one-dimensional measure would suffice.

Speaking more directly to David’s point about whether reification inheres in normal science, I would concede that it does. This is the essence of inductive or causal reasoning: to take disparate facts and reason that there is some logic underlying them, and hopefully a relatively simple logic at that. But we cannot be convinced that it actually exists without a great deal of either experimentation or faith, or both. Walter Isaacson’s excellent Einstein has a helpful example from the debate between Einstein and Planck over quantum theory:

For Planck, a reluctant revolutionary, the quantum was a mathematical contrivance that explained how energy was emitted and absorbed when it interacted with matter. But he did not see that it related to a physical reality that was inherent in the nature of light and the electromagnetic field itself….

Einstein, on the other hand, considered the light quantum to be a feature of reality: a perplexing, pesky, mysterious, and sometimes maddening quirk of the cosmos. For him, these quanta of energy (which in 1926 were named photons) existed even when light was moving through a vacuum. (p. 99)

At stake here is exactly the same issue as in the IQ case–whether a theoretical concept (in this case, quanta) was a feature of reality or a mere incident of measurement. Modern physical theory has generally accepted the reality of quanta, but the acceptance was by no means automatic.

Isaacson’s book makes clear how the scientific process can be affected by personal politics. Later on, Einstein takes Planck’s side in a debate over relativity and the principle of least action.

Planck was pleased. “As long as the proponents of the principle of relativity constitute such a modest little band as is now the case,” he replied to Einstein, “it is doubly important that they agree among themselves.” (p. 141)

In another instance, Isaacson describes how increasing anti-Semitism spurred Einstein into being more conscience of his Jewish identity. (Some might ascribe this to a social form of Newton’s third law.) The biography is interesting throughout, and highly recommended.

Are Casualty Statistics Reliable?

The question posed in the title is obviously too broad to be addressed in a single post, but the short answer is “no.”* This has been an unfortunate awakening for me, since I got into the study of political violence for the simple reason that measurement seemed straightforward. “Public opinion polls are so fuzzy,” I naively thought, “but a dead body is a dead body.” I have become aware of several problems with this view, a few of which I will share in this post.

Was the death due to conflict? This one is more complex than it first seems. A bullet in the head is pretty directly attributable to conflict. But what about someone who dies from treatable illness because the road to the hospital was blocked by fighters? Health care is increasingly integrated into research about conflict. The boundary line between what is or is not conflict-related, however, remains blurry.

Who is responsible for the death? I encountered this issue in my recent work on Mexico. The dataset that I relied on was one of three that counted “drug-related” murders. Since I was arguing that a certain policy had increased violence, I went with the smallest numbers to try to prevent false positives. The fact that there were three different datasets that attributed different body counts to the same cause reveals that there is still work to be done in this area.

What is the counterfactual? The first two are questions of causality, whereas this one addresses policy implications. Would the person with the illness above still have died in the absence of conflict? Would violence have become much worse without X or Y happening? Definitive answers to these questions may never be possible, but trying to answer them is at the heart of scientific research on violence.

These problems become even more exaggerated when looking at historical conflicts and trying to put them in context. Readers may recognize that Steven Pinker faced just that challenge in his recent book, The Better Angels of Our Nature, which argues that violence has declined over time. I am sympathetic to the basic point of “things aren’t as bad as you think,” but it turns out that there are some problems with his method. Michael Flynn points out two major issues, the first being the quality of the casualty data and the second being Pinker’s efforts to treat them as percentages of contemporary world population. One egregious error is attributing a large portion of the decrease between two consecutive Chinese censuses to the An Lushan revolt of the eighth century.

I do not mean to pick on Pinker, since I have yet to read his book, but his errors do show that someone with the capacity to write an entire book on this subject and get lots of press can still make basic mistakes while raising very few critical reviews. Doing good science is hard, even with body counts.

Further reading: Statistics of Deadly Quarrels, review by Brian Hayes (via Michael Flynn)

________________

*Note: Short answers being what they are, this leaves a lot to be desired. I must say that modern militaries are quite good at maintaining records of their own casualties. Most of the problems I mention here pertain primarily to non-state fighters or civilian casualties.

“You Are Not So Smart”

That’s the title of a new book by David McRaney. Here’s part of an excerpt from The Atlantic:

You rarely watch films in a social vacuum with no input at all from critics and peers and advertisements. Your expectations are the horse, and your experience is the cart. You get this backwards all the time because you are not so smart.

I would call this something like “the socialization of priors,” meaning that beliefs are informed by social group before they are informed by the (non-social) world around us. This is a topic that I am just beginning to consider, so I am far from having strong beliefs on it. So feel free to socialize my beliefs in the comments.

In particular, how does socialization impact the scientific process? Does it have any bearing on the implications that Michael Nielsen discusses here of the “new era of networked science”?

Nature and Politics

In the last post I discussed how nature has come to be regarded as a synonym for good, and suggested that that has not always been the case. Indeed, I am indebted to William Cronon for making the same point much better.* Allow me to quote from him before I move on to the main point of this post:

But the most troubling cultural baggage that accompanies the celebration of wilderness has less to do with remote rain forests and peoples than with the ways we think about ourselves—we American environmentalists who quite rightly worry about the future of the earth and the threats we pose to the natural world. Idealizing a distant wilderness too often means not idealizing the environment in which we actually live, the landscape that for better or worse we call home….

Indeed, my principal objection to wilderness is that it may teach us to be dismissive or even contemptuous of such humble places and experiences. Without our quite realizing it, wilderness tends to privilege some parts of nature at the expense of others.

The trouble that Cronon mentions–figuring out where to focus our definition of nature–at first seems tangential to politics, until we remember that several of the first great modern political philosophers were greatly concerned with answering the question, “What is the state of nature?”

Frontispiece to Rousseau's "Discourses"

The answer that one gives to that question is extremely consequential to everything that follows in his argument about how to best structure a society.
For Thomas Hobbes, famously, life in the state of nature was “nasty, poor, brutish and short.” Thus, anyone powerful enough to protect men from such a miserable life and quick, violent death could be regarded as a legitimate ruler. Jean-Jacques Rousseau, on the other hand, regarded nature as a land of peace and plenty. His idea for society, then, was that it should be as unrestrictive (“natural”) as possible, with some accommodations made to induce social cooperation.**

In this day and age, we have the ability to learn more about how nature does things and design our shoes and supermarkets accordingly. Can the same be done for human nature and society? Indeed, psychology and many of the social sciences are already attempting to answer this question. But they are doing so in ways that we would consider outmoded in other areas: we are at the point of making clogs, not barefoot running shoes; we talk with you about the social equivalent of a hydroponic system, but not organic vegetables. Getting there will be the next great challenge for the social sciences, and in my view it is going to require a paradigm shift away from unsatisfactory models that rest on excessively artificial assumptions. Nevertheless our new approaches, whatever they may eventually become, will still be simplifications of reality. Let’s not confuse them with an overly simplistic definition of nature that is exclusively good or bad. We live in a complex world, and that is enough.

____________________________

*And to Eric Higgins, for encouraging me to explore Cronon’s work as part of our teaching/facilitating of the spring 2011 ENGL 1304 course at the University of Houston.

**Apologies to the great thinkers of ages past for doing such violence to their philosophies by summarizing them so briefly.