big dataMany of the latest big data technologies are developed by big data startups from around the world that have found a way to deal with the vast amounts of data. They have developed disruptive big data technology that can be used by organisations to obtain valuable insights and turn data into information and wisdom. Of course, also the large existing IT players have developed substantial amounts of big data technology in the recent years. Especially large corporations that want an all-inclusive package installed use those technologies. There are also many different types of analysis that these startups perform and each will have a different impact or result.

Some technologies integrate data from different sources directly into a platform, skipping the need for additional data warehousing, while being able to deliver real-time interactive charts that are easy to interact with or to understand. These PaaS or DaaS solutions allow end-users to work with the data without requiring technical knowledge.

Learning from visualizations

There are big data technology vendors that focus on delivering the optimal graphical representation of big data. Visualizing unstructured and structured data is necessary to make the data understandable and turn it into information, but it is also very challenging. New big data startups however seem understand the practice of visualizing and have developed different solutions. One example is visualization based on the visual cortex of the human eye. This maximizes the ability of pattern recognition for the human brain. It makes it easy to read and understand massive amounts of relational data. The use of color and different thicknesses of the threats shown within the cortex allow users to easily recognize patterns and discover abnormalities.

Another way of visualizing is to use a technique called topological data analysis. This type of analysis focuses on the shape of complex data and is able to identify clusters and any statistical significance that is present. Data scientist can use this to reveal inherent patterns in those clusters. This type of analyses is best visualized with 3D clusters that show the topological spaces and can be explored interactively. Ayasdi is a big data startup that is capable of performing such visualizations.

It is definitely not always a necessity to have complex, innovative and interactive graphical representations. Infographics are visual representations of information, data or knowledge and they can help to make difficult and complex material quickly understandable. Dashboards combining different data streams showing ‘traditional’ graphs (column charts, line charts, pie charts or bar charts) can also provide valuable insights and there are many startups offering such solution.

Sometimes, real-time updated simple graphs showing the status of processes already provide more valuable information to improve decision-making then complex innovative visualizations. Visualizations on mobile devices get a completely new meaning when a user is able to play intuitively with the data while swiping, pinching, rotating or zooming on a mobile device. By Mark van Rijmenam Read more