nosql to hadoopHow will it fit in traditional enterprise IT environments?

Apache Hadoop is an open source technology that offers a radically cheaper alternative to the processing and storage of large amounts of unstructured data.

But for all of its potential benefits, Hadoop can be an uncomfortable fit within many IT environments. It challenges traditional approaches to data warehousing architecture, to the way in which IT projects are funded, and in some cases – can even threaten jobs.

On that basis, selling a Hadoop-based analytics project into a business is not as straightforward as a back-of-the-envelope cost calculation might suggest.

A different approach

Hadoop features characteristics that aren’t shared with most data processing solutions IT departments have supported to date:

  •     the core software is 100 percent free and available for just about any form of use,
  •     the core software is entirely open source and can be downloaded directly off the internet,
  •     Hadoop can perform extremely well using low cost network equipment and does not require expensive high end server hardware, a high performance SAN or NAS, or tape or disk-based archival, and
  •     a properly configured Hadoop cluster can provide data protection without requiring additional tools.

The fundamental selling point of the technology is that doesn’t require a large capital investment to get started.

Equally, however, some of these positive attributes have presented challenges for individuals pursuing big data projects inside large organisations.

Some CIOs are naturally cautious about adopting open source technologies without feeling confident adequate support structures are on the ground in Australia.

They are also likely to be cautious about any claim that Hadoop reduces dependence on RAID designs that have been a staple of data protection in the enterprise for decades.

And finally, Hadoop is radically different enough that some storage, server and data warehousing engineers will see a potential for the technology to obsolete their (vendor-specific) skills. As with any technology project, change management is crucial. We’ll explore each of these barriers on page two. By Brett Winterford read more