Taming the Data Torrent
Human beings demonstrate an unsuspected talent for digitization. Over the last two decades, we have applied our digitizing tools to virtually every man-made or natural phenomenon we’ve encountered, unleashing a torrent of digitized information that exceeds our ability to manage or analyze it. “The Sloan Digital Sky Survey is an important example,” said Donald Brown, the William Stansfield Calcott Professor of Engineering and Applied Science. In its first few weeks, the Sloan Survey amassed more data than had been collected in the history of astronomy.
But as Brown points out, this massive flood of digital information presents an unprecedented opportunity to understand and respond to our world. If we use it effectively, we can make dramatic advances in addressing every scientific and societal challenge we face.
Brown’s work illustrates the power of harnessing big data. He uses massive amounts of data in his predictive modeling—or forecasting of future events. For example, he’s developed analytical models that forecast the deployment of improvised explosive devices and shed light on insurgent decision making. His models provide insight into how insurgents might act differently as their technology improves, saving lives.
The expertise that Brown and his systems engineering colleagues have in data analytics is complemented by the pioneering efforts of U.Va.’s computer scientists to build the cyberinfrastructure for big data computation. And in fields like systems biology, astrochemistry, and environmental science, University researchers are playing a lead role in developing techniques to make the leap from big data to discovery.
In Brown’s view, the ability to use big data skillfully will be essential for future researchers. Accordingly, he has joined with colleagues around the University to propose a training program in big data for graduate students. “Ultimately, our nation’s ability to reap the benefits of big data depends on people who are fluent in analytics and can apply them across multiple applications using components of the cyberinfrastructure,” Brown said. “We would like them to come to U.Va. to learn those skills.”