SQL-on-Hadoop EnginesThe IT industry is set to push past $3.5 trillion by the end of 2017, according to CompTIA. Findings from Research and Markets found that Big Data investments alone will account for over $57 billion this year. Every business leader must recognize the crucial role technology will play throughout their industry. And while many organizations have reevaluated their business intelligence to assist in areas such as production and operations proactively, the oil and gas industry is one sector that has been slow to adopt.

The oil and gas industry is known for innovative drilling technology and has built a well-established IT department. However, they have neglected to put investments into renovating existing legacy systems. IT has not recognized the value in an agile structure that allows them to respond to business changes far more rapidly. The industry is volatile by nature, and that takes an unforgiving toll on today’s IT team. Without a highly configurable, fully integrated, pattern seeking product, the IT department cannot maintain speed. Therefore, it becomes less about having well built-out IT teams. It is about equipping those teams with proper tools to do their job efficiently.

As lower oil prices have held steadfast, oil companies are now looking for every opportunity to produce more oil at the same costs. The best way to do this now is to empower the pulse of the organization. Equip all departments, like field operations, with the power of big data and mobility. By doing so, everyone from the production accounting team to the COO taps into the ability to truly manage their wells in a cost efficient manner.

As oil and gas companies begin their journey through digital disruption, much like governments and banks have already done, they are leveraging data like never before. They are now placing value on what needs to change. Oil and gas companies are onboarding software systems capable of not just aggregating data but also recognizing key business patterns that optimize production. Through trustworthy data collection, extracting useful patterns and the horsepower of cloud companies reduce downtime, improve production, analyze wells, get straight to the discrepancies; and ultimately improve their bottom line.

By placing value on what needs to change, oil companies are seeing the vastly untapped revenue generator in an efficient, optimized, transparent digital ecosystem.

Here are the three biggest ways big data is playing a role in this digital transformation:

Capture Data: Cloud Computing

Cost – of IT, you can still get the same scalability with the cloud – more mature cloud environments, e-commerce, even the government has all of their application – governments and banks, bureaucratic they’re ahead of you in the cloud / tech.

To take effective action, the entire production chain, from COO right down to on-site well engineers and pumpers, need to see very detailed cost and production data, narrowed down to the invoice and well level, respectively. By leveraging cloud computing capabilities, accuracy and transparency are achieved in the shortest amount of time to improve well-cost management drastically.  When organizations can reach this level of speedy insight into costs, they can do things like quickly tie downtime reduction to production optimization.

For oil and gas companies to survive, they must leverage tech like cloud computing. Prior to the current technological environment, the industry didn’t have the means to create value out of data in real time. Now, a worker in the field office can monitor and act on that data while it is happening. This improves the bottom line by preventing unnecessary spending on the downed well.

Understand the Data: Production Pulse in the Field

To optimize business operations, oil and gas companies must solve critical challenges across all departments. The easiest and most efficient way to accomplish this is to integrate a production optimization system that creates the opportunity for seamless communication between everyone from field workers to the C-suite. The ability to monitor, control, and enhance well-site performance in real time has a direct return on investment.

According to McKinsey, the potential impact of using advanced analytics for predictive maintenance is a decrease in maintenance costs of up to 13 percent. At one company, where maintenance costs accounted for 25 percent of operating expenses, this enabled preemptive equipment maintenance—in effect, vital equipment could be repaired before it broke down. This effort reduced costs by up to 27 percent while increasing reliability and uptime. Advanced analytics for energy and yield also has the potential to increase energy efficiency by as much as 10 percent.

Being aware of these variables allows companies the opportunity to address concerns in real-time, while workers are out in the field, which vastly reduces downtime and production loss. Furthermore, armed with more recent data and accurate insights between the field and office, teams can now create dynamic routing to double pumper efficiency by prioritizing routes based on highest impact wells. Exceptions, not the rule, should guide fast action.

Recognize Patterns with to Run a Faster Business: Pump by Exception

The biggest opportunity to cut costs in the field operations is a term the oil and gas industry calls ‘pump by exception.’ By applying machine learning and artificial intelligence, oil and gas companies are able to make recommendations on which well sites need attention. And more importantly, which ones do not.

Essentially, extracting meaningful patterns in data to match a driver with a ride, like Uber for oil and gas. This form of production optimization is far more efficient than the current system where field workers tend to every well, unaware of whether or not it actually has an issue. By associating a lease operator based on their location and which well sites may need attention based on the data analysis, this cuts out the unproductive guesswork.  So, now you can optimize production by tending to a down well at 8:00 am instead of 2:00 pm. Efficiently task pumpers daily.

The use of data and automation of well information on pumps allows oil and gas companies to perform at optimum capacity, directly impacting their bottom line. When well downtime is reduced, costs in contract labor with lease operators are reduced. There are dozens of variables that can save millions when machine learning is applied correctly.


Production engineers, logistics analysts, surface land managers, superintendents and lease operators know how to drive a profitable oil field in any market condition.  Equip them to drive significant and lasting value.  Disparate source systems, ungoverned information, and unreliable data block their view to operational excellence.  Give them tools to turn meaningful insights into shared action. Then, watch the impact on profitability!

The article is by Shiva Rajagopalan, Founder and CEO of Seven Lakes Technologies. He has grown Seven Lakes from a one-man tech firm to a rapidly growing force on a path to transforming the oil & gas industry.