I’m sad not to be among the hundreds of survey researchers, data scientists, and other data collection professionals gathering this week in Austin, Texas, for the American Association for Public Opinion Research (AAPOR) annual conference. Granted, I’m a huge music fan and I love Austin. But more than that, I think this is an incredibly exciting time for data collection. 
Last year at this time, I was with the group in Florida as part of a Big Data Task Force that released this report. Although there were many important recommendations, the very first one is essential because so many of our colleagues fear that the era of Big Data is going to usher in the end of primary data collection. My fellow task force members disagree, which is why our primary recommendation is that surveys and Big Data must be viewed as complementary—not competing—data sources.
A lot of conversations in the research and policy community about Big Data bring me back to the first time I saw this. Yes, it’s been almost 35 years since “Video Killed the Radio Star” debuted on a brand-new network called MTV. I was among the many music lovers who were certain that this was the end of music as we knew it. Well, the obituary for radio, vinyl, and musicians who aren’t “video friendly” was premature. The same is true for data collection. It’s not going away, it’s just transforming. And transformation can be a very good thing. Just look at Austin’s South by Southwest festival. Over the past decade, SXSW has transformed from a sleepy (but amazing) music festival into one of the leading international gatherings of the digital and interactive communities. The music’s still there, it’s now complemented with a much richer landscape of creative content across different platforms.
Why isn’t data collection disappearing? The simple reason is causality. Policymakers and public program leaders now have access to more and larger data sets, along with the tools and technology to rapidly process and analyze that information. I have been among those advocating for policymakers to make better use of Big Data. But relying on Big Data alone leaves the decision-making picture incomplete—like a melody without the lyrics. Primary data collection is essential to understanding why people and populations think and behave the way they do. This has never been more apparent than in our current election cycle. Voting records are an administrative data set that tell us who people want for president, but the exit polls and other primary data collection help us better understand why people are choosing one candidate over another.
Although the need for primary data collection isn’t going away, the tools and methods are rapidly changing and will continue to do so. The most visible change in data collection will be the methods and approaches used in the future. Lady Gaga had it right—we are not going to reach her on her telephone. If current trends continue, we aren’t going to reach anyone else that way either. Although the big 60- or 90-minute survey and the call centers that support them may eventually become the province of purists like the modern-day record store, they will be replaced with new tools and technology that will be far more efficient and effective. It’s the music equivalent of the transition from vinyl to 8-tracks, to cassettes, to CDs, to mp3s. When it comes to primary data collection and new tools, we are somewhere in the cassette phase. Although I’m impressed by wearable technology, passive data collection, and the potential of the Internet of Everything, the field has yet to see its Steve Jobs moment. I don’t know precisely what that will look like, but I imagine the future of primary data collection will be cheaper, faster, and higher quality—and very bright indeed.
The goal of Evidence in Action is to spur discussion and conversation, so please let me and your colleagues know what you think. When you envision the future of data collection, what will change or stay the same?
