# Visualizing Political Unrest in Egypt, Syria, and Turkey

The lab of Michael D. Ward et al now has a blog. The inaugural post describes some of the lab’s ongoing projects that may come up in future entries including modeling of protests, insurgencies, and rebellions, event prediction (such as IED explosions), and machine learning techniques.

The second post compares two event data sets–GDELT and ICEWS–using recent political unrest in the Middle East as a focal point (more here):

We looked at protest events in Egypt and Turkey in 2011 and 2012 for both data sets, and we also looked at fighting in Syria over the same period…. What did we learn from these, limited comparisons?  First, we found out first hand what the GDELT community has been saying: the GDELT data are in BETA and currently have a lot of false positives. This is not optimal for a decision making aid such as ICEWS, in which drill-down to the specific events resulting in new predictions is a requirement. Second, no one has a good ground truth for event data — though we have some ideas on this and are working on a study to implement them. Third, geolocation is a boon. GDELT seems especially good a this, even with a lot of false positives.

The visualization, which I worked on as part of the lab, can be found here.  It relies on CartoDB to serve data from GDELT and ICEWS, with some preprocessing done using MySQL and R. The front-end is Javascript using a combination of d3 for timelines and Torque for maps.

GDELT (green) and ICEWS (blue) records of protests in Egypt and Turkey and conflict in Syria

If you have questions about the visualizations or the technology behind them, feel free to mention them here or on the lab blog.

# Visualizing the BART Labor Dispute

Labor disputes are complicated, and the BART situation is no different. Negotiations resumed this week after the cooling off period called for by the governor of California as a result of the July strikes.

To help get up to speed, check out the data visualizations made by the Bay Area d3 User Group in conjunction with the UC Berkeley VUDLab.  They have a round up of news articles, open data, and open source code, as well as links to all the authors’ Twitter profiles.

The infographics address several key questions relevant to the debate, including how much BART employees earn, who rides BART and where, and the cost of living for BART employees.

More here.

# Wednesday Nerd Fun: Where Things Come From

Sourcemap for a Laptop Computer, by Leo Bonanni

Where did your shoes come from? Your coffee? Your laptop? One of the beautiful things about the modern world is that you can hold a piece of technology in your hands–or wear it on your feet–without having to know the answer to this question.

As anyone who has read or heard “I, Pencil” knows, the genealogies of even the most banal products are immensely complicated. The punch line of that story is that pencils have too many components for one person to make or even fully understand, but through innovations like the price system they can be produced without central planning. In the author’s words, the moral is to “Leave all creative energies uninhibited.”

On today’s nerd fun site, Sourcemap, you can see cool visualizations of where all kinds of things come from. The maps for pencils are probably a little oversimplified, but some others are really neat, like the ones at the beginning of the post. My favorite are the food sourcemaps–check out Tropicana, Chicken of the Sea tuna, and Nutella. Now you knowl.

# Eliminate File Redundancy with Ruby

Say you have a file with many repeated, unnecessary lines that you want to remove. For safety’s sake, you would rather make an abbreviated copy of the file rather than replace it. Ruby makes this a cinch. You just iterate over the file, putting all lines the computer has already “seen” into a dictionary. If a line is not in the dictionary, it must be new, so write it to the output file. Here’s the code designed with .tex files in mind, but easily adaptable:

```puts 'Filename?'
filename = gets.chomp
input = File.open(filename+'.tex')
output = File.open(filename+'2.tex', 'w')
seen = {}
input.each do |line|
if (seen[line])
else
output.write(line)
seen[line] = true
end
end
input.close()
output.close()```

Where would this come in handy? Well, the .tex extension probably already gave you a clue that I am reducing redundancy in a $\LaTeX$ file. In particular, I have an R plot generated as a tikz graphic. The R plot includes a rug at the bottom (tick marks indicating data observations)–but the data set includes over 9,000 observations, so many of the lines are drawn right on top of each other. The $\LaTeX$ compiler got peeved at having to draw so many lines, so Ruby helped it out by eliminating the redundancy. One special tweak for using the script above to modify tikz graphics files is to change the line

`if (seen[line])`

to

`if (seen[line]) && !(line.include? 'node') &&  !(line.include? 'scope') && !(line.include? 'path') && !(line.include? 'define')`

if your plot has multiple panes (e.g. `par(mfrow=c(1,2))` in R) so that Ruby won’t ignore seemingly redundant lines that are actually specifying new panes. The modified line is a little long and messy, but it works, and that was the main goal here. The resulting $\LaTeX$ file compiles easily and more quickly than it did with all those redundant lines, thanks to Ruby.