big dataJohn Leonard of Computing.co.uk recently asked, “In 10 years’ time will big data analytics have replaced traditional relational business intelligence systems? That was one of the questions that Computing asked UK IT decision-makers in the quantitative survey that formed part of our recent big data research programme. The answer to that question was a pretty decisive ‘No’, with 68 per cent of respondents saying there will always be room for both technologies, while four per cent dismissed big data altogether, saying it’s a flash in the pan. Only eight per cent agreed that big data will eventually replace traditional relational business intelligence systems, with the remaining 20 per cent arguing that big data technologies will always be too complex for most organisations.”

Leonard continues, “Big data technologies are rarely seen as rip-and-replace solutions. Rather they work alongside existing solutions, finding niches where their ability to process voluminous or diverse data sets makes them a better fit than more traditional enterprise software. But while big data technologies might help to bring a wider range of data under the analytics umbrella, this is of little use unless the skills and the will are there to make use of the information. ‘BI is still hugely underutilised in its more traditional form,’ commented one respondent, and this undoubtedly is true. Many enterprises struggle to get their traditional BI systems to report on the fabled ‘single version of the truth’ as it is, hidebound as they are by incomplete coverage by their underlying datawarehouse systems, data sitting unanalysed in silos, and different premises or departments operating their own discrete systems.”

Big data technologies are rarely seen as rip-and-replace solutions. Rather they work alongside existing solutions, finding niches where their ability to process voluminous or diverse data sets makes them a better fit than more traditional enterprise software.

But while big data technologies might help to bring a wider range of data under the analytics umbrella, this is of little use unless the skills and the will are there to make use of the information.

“BI is still hugely underutilised in its more traditional form,” commented one respondent, and this undoubtedly is true.

Many enterprises struggle to get their traditional BI systems to report on the fabled “single version of the truth” as it is, hidebound as they are by incomplete coverage by their underlying datawarehouse systems, data sitting unanalysed in silos, and different premises or departments operating their own discrete systems.

It is also the case that many people don’t know what to do with the information revealed by their BI systems, with reports languishing unread in desk drawers, or ignored when the truth becomes inconvenient or impolitic. Adding new data sources where such a culture exists is unlikely to produce much growth in insight.

In short, unless an organisation is committed to being “data-led”, with people, skills and roadmap in place, there’s not much that new technology can do to help it.

Opening up to open source

bd-skills-fig-2-370x370Like cloud, open source was seen as a valuable tool to lowering one of the main barriers to entry – cost (figure 2). Many big data start-ups are based on offering open-source technology, and while it is certainly not cost free over the long term, many felt that the fact that such software is free to download and run was a significant advantage, especially for smaller organisations.

“There is no way that we could go out and buy a big IBM or big Oracle solution, that’s just out of our range,” said the head of IT at an online business services company.

It was generally felt by companies of all sizes that the open-source model has matured significantly over the past few years, building on the success of Linux, Apache web server and other widely adopted software, making it much more acceptable to budget holders. Developers, too, are increasingly likely to have cut their teeth on open-source tools.

“There is much more acceptance among analysts to look at open-source tools, also our skill sets are coming from university graduates and they are more comfortable using open-source tools. This is important from an innovation perspective as you get them [running] very quickly to test and learn. You can then migrate them to more robust enterprise technologies if needed,” a big data lead at a marketing firm explained.

Open source or proprietary, big data technologies are seen as increasing the size of the whole cake rather than gobbling up individual slices such as traditional relational BI.