Big Databig dataIn the Gartner model, businesses will first enter a stage where they will innovate and experiment with big data. This will be followed by a period of optimism and very high expectations from IT and business management.

Gartner’s next predicted phase for companies is a period of disillusionment. This is when companies are expected to take a “time out” from big data pursuits. Companies will begin to question whether they are getting all of the competitive-advantage information that they expected. It is also during this period that companies will regroup and revisit what they are trying to do with big data, asking whether the business is on the right path. As a consequence of this introspection, companies will make adjustments to their methodologies and expectations and then launch into a new path toward big data productivity where they will begin to see their results of their labor.

The Gartner model is particularly interesting because it downwardly revises the estimated arrival of business productivity from big data. The 2012 Gartner big data hype cycle projected that companies would achieve productivity with their big data within a two to five year timeframe. In today’s restated hype cycle model, this attainment of productivity is estimated to be five to ten years away.

Corporate big data project leaders don’t want to hear this news, but it’s better to think about it now (and to reflect on what you can do to avoid it) than to continue without considering whether a delayed productivity forecast could apply to you.

Gartner says the two main reasons for its downward big data productivity revisions for companies are: (1) Big data tools and techniques are being acquired before businesses have the expertise and maturity needed to use them, and (2) businesses are struggling with how to ask the right questions and spot opportunities where they can harness big data for breakthrough insights.

The natural question is: Do big data champions in organizations have to accept the reality of extended time to big data productivity? The answer is no. You can take steps to avoid sand traps that seem to center more around a lack of organizational know-how about big data than a lack of tools or resources. Here are six recommendations.

1: Build your business case first

Nathan Gnanasambandam, Senior Research Analyst at Xerox, recently spoke spoke about “dream sessions” they had with their clients. In the sessions, clients are asked to come up with questions about their businesses that they’d always wanted to answer but never could with the information they had available.

Building your business cases for big data doesn’t have to be that involved, but no one should go shopping for a big data tool and system without a clear strategic direction on what the company expects to learn from the data.

2: Don’t get budget happy

Big data shopping is a lot like mobile device and cloud shopping was several years ago — everyone wants bragging rights in executive summits that their company “has” big data solutions. For IT, which frequently finds itself embroiled in budget battles, there is also the challenge to fund big data systems and tools while the budgetary sun is shining on big data. Try to avoid this until you have those strategic sessions with business leaders on what you want to derive from your big data.

3: Don’t “underdesign” pilot projects

The normal course in an IT project using new technology is to design a small pilot project first so you get a quick project success under your belt. The strategy should be no different with big data.

However, you still need to design that first project large enough so the methods you use can be applied to subsequent business cases. If you don’t design a proof of concept that you can leverage for future work, you could quickly find yourself (and your management) propelled into the disillusionment phase of big data that Gartner talks about. By Mary Shacklett Read more