For all the bragging on the winning side — and an explicit coveting of these methods on the losing side — there are many unanswered questions. What data, exactly, do campaigns have on voters? How exactly do they use it? What rights, if any, do voters have over this data, which may detail their online browsing habits, consumer purchases and social media footprints?
How did Mr. Obama win? The message and the candidate matter, of course; it’s easier to persuade voters if your policies are more popular and your candidate more appealing. But a modern winning campaign requires more. As Mr. Messina explained, his campaign made an “unparalleled” $100 million investment in technology, demanded “data on everything,” “measured everything” and ran 66,000 computer simulations every day. In contrast, Mitt Romney’s campaign’s data operations were lagging, buggy and nowhere as sophisticated. A senior Romney aide described the shock he experienced in seeing the Obama campaign turn out “voters they never even knew existed.” And that kind of ability matters: while Mr. Obama did win decisively, the size of his lead in four states that determined the outcome, Florida, Ohio, Virginia and Colorado, was about 400,000 votes — or about 1.2 percent of the eligible voters.
The confluence of marketing and politics goes back a long way. A blizzard of direct mail engineered by political consultants is credited with defeating President Harry S. Truman’s national health care proposal after World War II. The new methods, however, are not just better direct mail. Noxious TV ads and slick mailers are like machetes compared with the scalpels of social-science-based big-data. The crude methods may still work to soften the ground and drown out other voices, but in the end they are still very big sticks. Sometimes they kill the patient — just ask swing-state voters about the TV ads they were bombarded with.
The scalpels, on the other hand, can be precise and effective in a quiet, un-public way. They take persuasion into a private, invisible realm. Misleading TV ads can be countered and fact-checked. A misleading message sent in just the kind of e-mail you will open or ad you will click on remains hidden from challenge by the other campaign or the media. Or someone who visits evangelical Web sites might be carefully shielded from messages about gay rights, and someone who has hostile views toward environmentalism may receive messages stroking that sentiment even if the broader campaign woos the green vote elsewhere.
The Obama campaigns technologists were tense and tired. It was game day and everything was going wrong.
Josh Thayer, the lead engineer of Narwhal, had just been informed that theyd lost another one of the services powering their software. That was bad: Narwhal was the code name for the data platform that underpinned the campaign and let it track voters and volunteers. If it broke, so would everything else.
They were talking with people at Amazon Web Services, but all they knew was that they had packet loss. Earlier that day, they lost their databases, their East Coast servers, and their memcache clusters. Thayer was ready to kill Nick Hatch, a DevOps engineer who was the official bearer of bad news. Another of their vendors, StallionDB, was fixing databases, but needed to rebuild the replicas. It was going to take time, Hatch said. They didnt have time.Theyd been working 14-hour days, six or seven days a week, trying to reelect the president, and now everything had been broken at just the wrong time. It was like someone had written a Murphys Law algorithm and deployed it at scale.Theyd been working 14-hour days, six or seven days a week, trying to reelect the president, and now everything had been broken at just the wrong time.
And that was the point. “Game day” was October 21. The election was still 17 days away, and this was a live action role playing LARPing! exercise that the campaigns chief technology officer, Harper Reed, was inflicting on his team. “We worked through every possible disaster situation,” Reed said. “We did three actual all-day sessions of destroying everything we had built.”
In late spring, the backroom number crunchers who powered Barack Obama’s campaign to victory noticed that George Clooney had an almost gravitational tug on West Coast females ages 40 to 49. The women were far and away the single demographic group most likely to hand over cash, for a chance to dine in Hollywood with Clooney — and Obama.
So as they did with all the other data collected, stored and analyzed in the two-year drive for re-election, Obama’s top campaign aides decided to put this insight to use. They sought out an East Coast celebrity who had similar appeal among the same demographic, aiming to replicate the millions of dollars produced by the Clooney contest. “We were blessed with an overflowing menu of options, but we chose Sarah Jessica Parker,” explains a senior campaign adviser. And so the next Dinner with Barack contest was born: a chance to eat at Parker’s West Village brownstone.
For the general public, there was no way to know that the idea for the Parker contest had come from a data-mining discovery about some supporters: affection for contests, small dinners and celebrity. But from the beginning, campaign manager Jim Messina had promised a totally different, metric-driven kind of campaign in which politics was the goal but political instincts might not be the means. “We are going to measure every single thing in this campaign,” he said after taking the job. He hired an analytics department five times as large as that of the 2008 operation, with an official “chief scientist” for the Chicago headquarters named Rayid Ghani, who in a previous life crunched huge data sets to, among other things, maximize the efficiency of supermarket sales promotions.
At the same time, a new data-science team within Romney’s strategy department sifts through reports on Obama’s broadcast buys as assembled by the Campaign Media Advertising Group. Romney staff code each of Obama’s ads according to their distinctive characteristics: content, style, the candidate attributes and characteristics they’re trying to drive—even the gender of the narrator and at what point the legally mandated “I approve this message” tag is inserted—along with the perceived demographic audience and the markets in which it appeared.
Last fall, a tech-savvy startup burst onto the scene with a hiring spree: “We are looking for analytics engineers and scientists in our Chicago headquarters to work on text analytics, social network/media analysis, web personalization, computational advertising, and online experiments & testing.” Since then, Obama for America OFA quietly has added dozens of positions that never would have existed 10 years ago—titles like chief scientist, director of modeling, battleground states election analyst, and chief integration and innovation officer. Heres what they do: