Unemployment rate accuracy impacted by rotation group bias, paper says| Sep 24, 2014
The statistical accuracy of the official unemployment rate is questionable, concludes a working paper recently published in the National Bureau of Economic Research by Alan Kreuger, Wilson School professor and former chairman of President Barack Obama’s Council of Economic Advisers.
He conducted the research with coauthors Alexandre Mas, professor of economics, and Xiaotong Niu,analyst at the Congressional Budget Office.
Methodology used by the U.S. Census Bureau and the U.S. Bureau of Labor Statistics to calculate the unofficial employment rate through information obtained from the Current Population Survey, also known as CPS, is the source of continually worsening inaccuracy in the statistical measure of the unemployment rate, the paper argues.
To calculate the official unemployment rate, 60,000 households are surveyed and broken into eight rotation groups designed to be representative of the overall population. The rotation groups are then surveyed for a period of four months, not surveyed for the following eight, and surveyed again for the last four months of the 16-month period.
Kreuger, Mas and Niu’s findings, however, indicate that survey nonresponse rates at later stages have increased since 1994 when the CPS moved from pen-and-paper questionnaires to computer-assisted phone interviews, resulting in a skewed final calculation when averages obtained from the different periods are weighted to calculate the official unemployment rate released to the public.
This phenomenon, referred to as rotation group bias, was first explored in 1975, and the authors of this paper have found that the effect of rotation group bias is worsening over time, having more than doubled since 1994 when CPS was redesigned.
“One of the takeaways from the paper is that we think the increase in bias is a result of the lower response rate to the survey," Mas said. "We think this should be an impetus to find ways to increase the response rate to the survey.”
Kreuger and Niu declined to comment.
“I think it is a specific and relevant finding," Massachusetts Institute of Technology professor David Autor said. "The way we measure the unemployment rate, which is a key indicator of the health of the U.S. economy, has become less accurate over time … because the actual measure of unemployment is affected by participation in the survey.”
However, Mas noted that the paper is not intended to imply that the unemployment rate no longer has significance.
“We think it’s a good measure, actually. We just think it can be improved,” he said.
Professor of Economics at Michigan State University Gary Solon GS ’83 also said the official unemployment rate has significance in determining the health of the economy regardless of statistical inaccuracy in the number itself because economists use the changes and patterns in the rate that economists use in their analysis, rather than the number itself. However, like Mas, Autor added that additional forms of calculating unemployment can be incorporated to make the statistic more accurate.
Nonetheless, the emphasis should remain on improving survey response statistics, because the rotation group bias is not necessarily unavoidable, Mas said. He noted that their study of Canadian surveys indicates no rotation group bias in Canadian employment statistics.
“People don’t fully appreciate that labor force statistics that come from these surveys are run by the government, and suffer from the same kind of budget shortfalls that other programs do," he said. "We need to maintain the integrity of these surveys. It’s a small price to pay to understand the state of the economy.”
The working paper, entitled “The Evolution of the Rotation Group Bias: Will the Real Unemployment Rate Please Stand Up?” was published in August in the National Bureau of Economic Research.