Big Government Is Getting In The Way Of Big Data

Big Government Is Getting In The Way Of Big Data

Big Government Is Getting In The Way Of Big Data

When the government wants to know how many people are unemployed, it calls people and asks them whether they’re working. When it wants to know how quickly prices are rising, it sends researchers to stores to check price tags. And when it wants to know how much consumers are spending, it mails forms to thousands of retailers asking about their sales.

“Big data” may have revolutionized industries from advertising to transportation, but many of our most vital economic statistics are still based on methods that are decidedly, well, small.

Now economists both inside and outside government are trying to change that. They are working to open up access to government records, make better use of private-sector data and use modern statistical techniques to link together different sources — steps, they believe, that could allow for economic statistics to become more accurate, more detailed and perhaps available more quickly. Ultimately, they hope to allow economists to tackle questions that aren’t answerable using currently available data sources — how government programs affect participants years or even decades down the line, for example. President Obama’s latest budget, released last month, dedicates an entire chapter to proposals to expand access to so-called administrative data, records collected as part of government programs rather than through surveys.

Read Also:
5 Steps to Get Big Analytics for a Small Price Tag

But it won’t be easy. Efforts to change the way the government collects statistics face legal, bureaucratic and practical hurdles and in some cases could run afoul of privacy advocates worried about how the government tracks its citizens. Despite bipartisan support for change, actual progress has been slow.

“It’s just taking a lot longer than anyone wanted,” said Hal Varian, chief economist for Google and an outspoken advocate for the expanded use of modern data-collection approaches.

One example of the shortcomings of the current system — and the potential for improvement — is the monthly jobs report. Every month, investors, economists and journalists (myself included) race to digest the Bureau of Labor Statistics’ count of new jobs. But the report is highly imperfect. Its numbers, based on a survey of businesses, are volatile and subject to revision. It provides little information about what kinds of jobs are being created or how much they pay. The only demographic information available is based on an entirely separate survey with even larger margins of error.

Read Also:
Visually Speaking: Patterns for Humane Data Visualization

The government has much more complete information on almost all those subjects. State unemployment systems collect detailed information on employment and wages. The Internal Revenue Service has extensive information about virtually every business in the country, and the Social Security Administration tracks nearly all workers. There’s even the National Directory of New Hires, a little-known database created to track child-support delinquents.

Statisticians at the Bureau of Labor Statistics don’t have access to any of those sources for the monthly jobs report, however. IRS data is off-limits under federal law. Other sources are unavailable either because the agencies that control them won’t share them or because the BLS doesn’t have the resources to turn them into a usable form.

The problem goes far beyond the jobs report. Most of the government’s most closely watched economic indicators are based on surveys, from monthly reports on construction, retail sales and inflation to annual reports on household income and consumer spending. In nearly every case, more complete data exists from either public or private sources, if only government agencies could get access to it.

Read Also:
How healthcare can play safe in the data lake

The access limitations are at least partly the legacy of the last time that the government tried to expand its collection and use of administrative data. In 1965, the Johnson administration proposed the creation of a national data center in part to track the performance of Johnson’s Great Society initiatives.

 



Chief Analytics Officer Europe

25
Apr
2017
Chief Analytics Officer Europe

15% off with code 7WDCAO17

Read Also:
Bringing #DevOps to #DataScience

Chief Analytics Officer Spring 2017

2
May
2017
Chief Analytics Officer Spring 2017

15% off with code MP15

Read Also:
5 Steps to Get Big Analytics for a Small Price Tag

Big Data and Analytics for Healthcare Philadelphia

17
May
2017
Big Data and Analytics for Healthcare Philadelphia

$200 off with code DATA200

Read Also:
Why Image Analytics Holds the Key to Better Big Data Analysis

SMX London

23
May
2017
SMX London

10% off with code 7WDATASMX

Read Also:
Data Analytics: How Your New Career is Changing Everything

Data Science Congress 2017

5
Jun
2017
Data Science Congress 2017

20% off with code 7wdata_DSC2017

Read Also:
3 Surprising Things That Big Data Reveals About HR

Leave a Reply

Your email address will not be published. Required fields are marked *