One theme of this book is that government agencies don’t do what you think they do. The USDA manages national forests and grasslands, for instance. The Department of Energy keeps the world’s nuclear weapons safe. And the Commerce Department has all of the best data.
Another thread in the book is how people make use of that data. One of those is DJ Patil, who learned to forecast complex systems by working with weather data and then moving into national defense:
The relevance of that ambition became a bit clearer after the terrorist attacks of September 11, 2001. “There was a sense that this was, among other things, a failure of data analysis,” he said. “If we had known how to distinguish signal from noise we’d have seen it and prevented it.‘ Hey, why are all these guys suddenly taking flight lessons?’” The assassins’ use of credit cards alone, properly analyzed, would have revealed they were up to no good. “The image of a good network is messy,” said DJ. “It’s really hard to fake messiness. It’s hard to fake being an American with a credit card.” (p. 154)
Later he went on to work at LinkedIn and helped coin the term “data scientist”:
The people in human resources complained to him that the company had too many data-related job titles. The company was about to go public, and they wanted to clean up the organization chart. To that end DJ sat down with his counterpart at Facebook, who was dealing with the same problem. What could they call all these data people? “Data scientist,” his Facebook friend suggested. “We weren’t trying to create a new field or anything, just trying to get HR off our backs,” said DJ. He replaced the job titles for some openings with “data scientist.” To his surprise, the number of applicants for the jobs skyrocketed. “Data scientists” were what people wanted to be. (p. 156)
Patil went on to become the U.S. Chief Data Scientist.
David Friedberg also found innovative ways to use data that the federal government provides. He founded WeatherBill, which offered crop insurance to citrus growers and packers (in addition to government insurance, which was only available to farmers). Then, he worked with Google to digitize decades of NOAA’s satellite images to better understand the citrus growing cycle and found out that the farmers had been cheating the insurers:
Plants absorb visible light and emit infrared light: you could calculate the biomass in a field by how much infrared light it emitted. Friedberg brokered a deal with Google, which had digitized the information and gave him access to it for free. “That’s when we discovered that farmers were lying about the dates they were planting,” said Friedberg. The federal crop insurance program, seeking to minimize the risk of freeze, stipulated the earliest date that a farmer was allowed to plant. But the earlier the seeds went into the ground, the richer the crop. To qualify for the insurance, farmers had been claiming to have planted their seeds later than they had. The lie had been captured for decades by satellite, but no one had been able to see the data. (p. 182)
WeatherBill was later renamed to Climate Corporation, which was the first unicorn in the ag tech space.
Lewis’s account of the lesser-known areas of the federal government is a compelling read for anyone interested in how we make the world more legible.