How to Manage Big Data with a Data Governance Policy
Companies, both large and small, learning to cope with big data are using a myriad of strategies from employing private and public cloud enabled technology to completely overhauling their view on support of data types, device types, and overall data strategy. One critical approach to managing the big data challenge is the creation of a data governance policy. While data governance has been used by IT for years to establish control over organizations’ numerous types of data, big data has unique characteristics that affect how it is governed. For example:
The sheer volume of data. Big data tends to grow exponentially. Without defined governance policies in place, it quickly can become impossible for organizations to search, classify, and manage huge amounts of information.
The wide variety of data. Traditionally, data governance has focused on information stored in relational databases. Big data, however, involves many different forms of information such as non-relational databases and other types of unstructured data, like information generated by social media.
Combining knowledge of big data challenges with a well-thought out data governance policy, however, enables organizations to increase the value of their information and transform it into a highly available view of a company’s legacy knowledge and intellectual property. When users enjoy access to data that is consistent, accurate and available when they need it, that translates into better business decision-making and faster responses to market trends and customer needs.
Neglecting Data Governance – A Risky Proposition
It’s not uncommon for organizations to view data creation as a separate activity from data governance. That approach can be risky, however, if data governance falls by the wayside and is forgotten or neglected. Dangers associated with ungoverned big data include:
Data that is difficult to search and analyze. When organizations don’t have a coordinated approach to data management and governance, business users often have a hard time finding the information they need to make decisions and they may not trust the validity of the data they can access. This problem is compounded as data volumes increase. On average a company’s data doubles every 18 months. This means that the time for getting control over this information is now.
Lack of compliance with regulations or internal controls. Regulations such as the Federal Rules of Civil Procedure (FRCP), the Federal Rules of Evidence (FRE), the Health Insurance Portability and Accountability Act (HIPAA), Sarbanes-Oxley (SOX), and others are complex because organizations must proactively demonstrate compliance with standards related to electronically stored information. If steps to ensure compliance are not articulated in a data governance policy and then followed, compliance issues can arise.
Potential for financial and reputational damages. If organizations don’t have a clear idea of the lifecycle for different types of information and the systems where information resides, the risk of data breaches and theft increases. That can result in fines, as well as reputational damage.Unclear retention policies for different types of information. Storing big data indefinitely can quickly become a costly matter. A robust data governance policy should define how long different types of data are retained. By Chris Grossman read more