Best Big Data Practices for 2018
Big data helps you make money by providing you improved analytic tools that leverage existing massive data sets you may not have utilized or you underutilized in the past. It provides enhanced marketing and sales insight. It helps identify consumer needs. It provides the basis for well-developed personas. It gives management finer grain detail into situations for better business decisions. It automates data gathering and basic analysis to provide enhanced insight to management who incorporate added value human observations regarding the basic analysis.
Since it’s only been a short time that big data moved from the confines of hard science to the world of business and marketing, its uses continue to develop. Until relatively recently, big data analysis required supercomputers. For scientists, that often meant vying for time on NASA systems. As computers develop by leaps and bounds, larger data sets can run on smaller systems, putting big data analysis in the hands of small business. With its growing ubiquity comes an increased need for appropriate principles and practices.
Doing it right from the beginning provides a solid foundation for what is quickly becoming the leading new management decision-making tool. Doing it right means implementing best practices from the start. Start with best practices in three phases: planning, implementation and maintenance.
- Collaborate by forming a business intelligence team comprised of data scientists and business people such as key managers and marketing staff. Have those from the data side educate leadership about what is and is not possible with big data. This helps close communication gaps before the pilot program starts.
- Begin with a pilot project. Choose one succinct area or topic on which you want to improve reach or decision-making. Tie your big data project to a business goal. Your pilot provides the test case for your planning, implementation and maintenance programs. Its results give you a starting point for an initial improvement powered by big data and a point of reference for program improvement. Ensure you have access to the appropriate external and internal data.
- Organize the business requirements of the big data pilot. Analysis and understanding of the business requirements enable proper data gathering and analysis.
- Evaluate data needs. Analyze available data aspects, its frequency and how it can benefit the business. This helps discover data holes that you can address after you successfully complete the pilot.
- Obtain private cloud storage during the planning phase. It provides better data protection than a public cloud. While it’s slightly more expensive, it’s the closest thing to on premise data storage.
- Plan the pilot and overall big data program with the European Union’s General Data Protection Regulation (GDPR) in mind. Even if you do not already do business there, it sets you up to scale to do so effortlessly. The GDPR sets the new standard for data protection so using it as a guideline ensures you use best practices data protection. As a bonus, you’ll avoid the non-compliance fines which go up to four percent of the firm’s annual revenue.
- Weave big data analytics and decision-making into daily business operations, especially with front-end employees. Beginning with the pilot project transitions staff into the daily data process, enhancing comfort levels.
- Anonymize data to help protect customer privacy. This technique still affords you the ability to perform trend analysis. You can create composite values or utilize data redaction or masking.
- Standardize your big data collection and analysis. Standardization and automation ease the burden of the current data scientist shortage. Even if you can afford a hefty salary to draw a quality scientist, standardization enables scalability.
- With business goal and guidelines in mind, let your data scientists design and code the pilot in their preferred programming languages. After the pilot project, the data scientists can work with the larger IT department to expand the scope. This may mean moving to other programming languages with more cross environment capabilities or enhanced security.
- Implement in an iterative manner. Work from the initial use case to scale efforts. Identify specific, small, high-value questions or problems to address using big data. Choose one and work through it to expand beyond the pilot. Expand by answering one problem at a time.
- Establish a Center of Excellence (CoE) to empower knowledge transfer. This aids wider organizational development and distributes hard and soft costs organizationally.
- Establish the data flow controls, including analytical modeling, in-database summarization, integration, post-processing and pre-processing.
- Combine your big data with enterprise data to best leverage actionable results. You can make data integration a natural product of your CoE. This benefits you organization-wide by breaking down data silos.
- At least annually, examine your cloud vendors’ privacy practices and data policies. Ensure they remain in concert with yours. If their practices falter, change vendors.
- Annually vet departmental data and privacy practices. Use external auditors, as well. Document non-compliance and rectify it.
- Annually conduct a social engineering audit to thwart employee sabotage. Individuals may share information outside of the organization that they should not. The auditor examines systems and accounts for phishing, phone and physical entry attacks. Address any anomalies with employee training. Include reprimands, if needed.
Your company’s big data program can speed and ease business decisions. It can help you reach new markets and enhance sales. Developing it from scratch using best practices ensures a better working program that succeeds.
In addition to being the editor at designrfix and writing about tech, web and graphic design among other subjects, I love “unplug” and be outdoors hiking and enjoying nature. If you can’t reach me, it’s probably because where I am at doesn’t have cell phone reception.