Has Hadoop Entered the Mainstream?
After all the hype around Apache Hadoop, we are starting to see some successful implementations that highlight its benefits. However, we do not yet see the kind of enterprise adoption typically associated with a technology that promises to manage high volumes, reduce implementation costs and, more importantly, monetize data. The question is why?
The increased volume, variety and velocity of “big data” have strained the conventional data processing technologies used for the data management life cycle. In response, many solutions have evolved to address these challenges. One key technology is Apache Hadoop, open source software used for reliable and scalable distributed computing.
In spite of the potential of Hadoop, however, most companies continue to approach it cautiously, evaluating its suitability as an enterprise solution, although the concepts of Hadoop MapReduce have been around from the early 2000s.
To assess Hadoop’s suitability for enterprise-wide adoption, it should be judged against the characteristics that help a technology (such as extract, transform and load) gain acceptance as a mainstream enterprise technology.
Characteristics Required for Enterprise-Wide Adoption
To become widely accepted and achieve an established presence within the enterprise, a technology should:
1. Be proven: A proven solution addresses use cases effectively and efficiently. For example, in a reporting scenario, a focused data mart with large numbers of standard and pre-built canned reports is the proven approach as opposed to trying to tax the data warehouse or operational data store.
2. Be based on a design paradigm: IT solution architects more readily embrace solutions based on designs that are an extension of an existing approach. For example, in the ETL world, the combination of row-based processing and rich user interface made understanding the solution and developing expertise in the technology relatively easy.
3. Be a solution enabler: Solution enablers are tools or frameworks that support a design paradigm. This drives a technology solution’s pace of evolution from a theoretical concept to mainstream production. A stable integrated development environment with tools to debug, monitor performance and manage workflows in addition to accelerating code development is a great enabler for a technology. These help the developers focus on application logic rather than non-functional challenges.
4. Offer good value: A technology is much more liable to be adopted and receive further investment if it performs well against these measures:
- Break-even analysis
- Return on investment
- Addition/reduction to total cost of ownership read more