big data It’s an engrossing thought for an idle moment: if you had all the computing power in the world at your disposal, what task would you want accomplished? Would you put it to work on a cure for cancer? Or perhaps the power could be used to discover whether there’s an absolute square root of pi.

There are plenty of issues that computers could address — and what’s becoming apparent is the virtually limitless nature of computing resources at the world’s disposal. Where 30 years ago an 8-bit computer and 256KB of RAM was a big deal, now we have quad-core phones storing gigabytes of information.

The huge repositories of information around the world, which get bigger every day, have given rise to the term ‘big data’ — and although few people seem to like the term, no-one has yet come up with a better one.

When talking about big data, it’s never long before Google and Amazon enter the conversation. These companies represent two of the biggest data stores in the world — data stores that are measured in exabytes and petabytes. But Google and Amazon are typical of today’s large enterprises in another way: we think of them as two very large companies, but in reality they comprise multiple smaller entities.

Take Amazon as an example. Within Amazon is a second-hand book company (AbeBooks), a download company (Audible), a publisher (Kindle Publishing), the IMDb movie database company, four shoe companies and many, many more. Amazon also recently bought The Washington Post.

Look at Google and the picture is similar. Again, the ‘search giant’ actually comprises many different entities doing many different things.

Amazon and Google measure their files on individuals and companies in the multiple millions, but how do they keep tabs on everything? Big databases help. I am an Amazon customer, and have been since 2002 when I bought two Star Trek DVD box sets for £122. I know that because all the details of that transaction are still in Amazon’s database. So are the details of every other transaction by all the millions of Amazon customers around the world. That’s big data in action. Now companies and individuals can keep details of everything because the world has enough storage space for everything.

The big question is: now we have all these petabytes of data, what can we do with it? By Colin Barker Read more