big data toolsBig data. It’s massive. It comes in all types of formats. It’s dynamic, changing.

Government managers are looking to derive value from the mountains of data collected by their agencies to tackle a host of issues, including cybersecurity, fraud detection, crime prevention, medical research, weather modeling, intellectual property protection, operation efficiency and situational awareness.

A growing challenge is choosing the right technology to aid in collecting, processing, analyzing and storing massive amounts of data, especially since the pool of big data tools keeps expanding.

Unfortunately, there are no “must have” tool sets, since any initial big data deployment will be driven by an individual agency’s business requirements, according to the TechAmerica Foundation’s report, “Demystifying Big Data.”

Tools that ingest and extract data, index it, translate it and then clean it up for analysis and presentation are part of an expanding big data ecosystem, said Barbara Toohill, vice president and director of Mitre’s Homeland Security Systems Engineering and Development Institute.

To put these technologies to best use, agency managers need to understand the problem they are trying to solve and then determine what data is needed to solve that problem before investing heavily in tools, Toohill advised an audience of government and industry representatives at a recent FCW Executive Briefing on big data in Washington, D.C. “One of the challenges is that tools sound great in PowerPoint presentations, but are much more challenging when people start using them,” she said.

Still, there are core technologies that come in both open-source and proprietary solutions to support any agency’s big data portfolio. But where to start? Read more