U.S. Unemployment 2014: Tweet Analysis Paints More Accurate Employment Picture Than The US Government Release

An economics professor has found a way to estimate U.S. employment more quickly and accurately than the government can -- by using Twitter data. University of Michigan Professor Matthew Shapiro and colleagues searched the microblogging platform for tweets containing words and phrases people commonly use to talk about jobs and unemployment, like “lost my job.” Then Shapiro, Margaret Levenstein of the Survey Research Center, and Stanford University computer scientists Michael Cafarella and Dolan Antenucci tested the results to make sure the terms were referring to jobs and not something else -- like, in the case of “lost work,” a computer crashing.

“When we started,” Shapiro said in a press release, “we had no idea if we could track job loss with tweets, but over a two-year period, we’ve seen the social media index perform quite well.”

At several points, their social media index lined up with government data, like when it leveled off from 2011 to 2012, and showed fluctuations around Hurricane Sandy in late 2012 and the government shutdown in late 2013. When the social media index didn’t match government data, it turned out to be more accurate, such as when California's new computer system resulted in delays in processing unemployment claims that caused government data to underestimate employment, Shapiro said.

Many economic indicators, like the government’s estimated employment rate and UMich’s well-known consumer sentiment index, are based on mail and phone surveys. But these surveys are costly and becoming more difficult as fewer people have landline phones or respond to surveys at all. The data, when released, is often four to six weeks old. Mining Twitter, on the other hand, provides data in real time.

“In turning points or times of crisis, having something available instantaneously is critical for policymakers,” Shapiro said.