Dawn of the data engineer?

Oil and gas has all the resources for a big data revolution – so what’s holding us back? Robert Dickson, Director – field development project excellence at io sets out his view.

Image from iStock.

Over recent years, in almost every industry, the concept of big data has moved sharply into focus, with Google searches for the term increasing over ten-fold since the beginning of 2011[1].

But what is big data? Perhaps the most helpful way to define it, for the purposes of the oil and gas industry, is the practice of using large data sets to reveal patterns and trends that can be used to optimize performance.

Currently, there are potentially enormous efficiencies that could be generated with the widespread adoption of big data techniques. Today, arguably the most prominent current application of big data is in seismic data interpretation, which has broadly served as the exploration and production industry’s introduction to the science of big data and pattern recognition.

Beyond this, a few other new and notable applications of big data are emerging. For example, the use of big data techniques is beginning to be seen in predictive asset maintenance (using advanced statistical techniques to predict the failure of plant equipment); the use of advanced software to implement a statistical approach to production optimization and the use of advanced statistical techniques on health, safety and environment data and other operational data to predict hazardous combinations of activities. Indeed, we at io combined many of these approaches when designing our own “systems thinking” field development approach.

Yet, despite a few positive exceptions, the implementation of big data within the exploration and production industry has to date been slow, and led by those with large enough budgets to conduct pilots and explore tools and techniques. Currently, the oil and gas industry is simply not effectively exploiting the data being collected, and there are a number of reasons why big data techniques do not yet appear to have been widely adopted in exploration and production.

At the core of the issue, there appears to be a reluctance of exploration and production companies to hire data engineers to work with front line engineers and scientists to explore combination of data sets and the value that can be derived from this. Additionally, it seems fair to say that big data requires a more curious or explorative culture based on determining what value can be potentially derived from large and disparate data sets, rather than a more focused approach of solving an identified problem. As such, organizations must consider how they can make sure to hire individuals with such a mind-set, and create a corporate culture that allows them to thrive.

The adoption of big data is not helped by a lack of access to new tools sets created to analyze and handle large data sets. Attempts are being made to address this issue, but so far only select individuals in exploration and production would claim to have access to processing software such as SAS, R platform or IBM Watson. Additionally, the availability structure and quality of data cannot yet be described as “standardized”.

Currently, only a small number of exploration and production companies have embraced big data/analytics at an organizational level, establishing centers of excellence to identify analytical techniques and tools and spread their use around the company.

The ‘C’ word

More broadly, there is also scope for much greater collaboration within the oil and gas industry. Traditionally, the competitive nature of the industry has meant that companies have gone to great lengths to guard their innovative processes and technologies. Whilst such thinking may lead to a competitive advantage in the short-term, it stifles innovation and leads to significant inefficiencies (the current lack of standardization is a striking example of this).

Instead, the industry would benefit from greater sharing of technology, and where possible should encourage engineers and data specialists to work together to create innovations that benefit the wider industry. For example, extensive work on establishing data standards has been undertaken by joint industry project Energistics as part of the organization’s wider objective to define open standards for the upstream oil and gas industry. The widespread adoption of these standards would allow for greater collaboration within and across organizations, smoothing out many of the road blocks that currently prevent the adoption of big data techniques.

Similarly, other regulated industries, from healthcare to finance, have benefited from the establishment of “Hack Days.” Borrowed from Silicon Valley’s tech upstarts, the term refers to an event where engineers and developers are encouraged to attend an event and work (collaboratively or competitively) to achieve a specific goal. This concept could be adopted in the offshore industry, and engineers equipped with the necessary data engineering skills could be brought together to collaborate on finding imaginative ways to improve efficiencies within organizations. There is also scope to give individuals with the right skills a wider remit, and involve them in front-line activities to work hand-in-hand with engineers to search for opportunities where big data might add significant value.

While the entire sector is undoubtedly feeling the pain of an uncertain oil price, the current economic environment also presents the oil and gas industry with a unique opportunity. The need to reduce costs to make projects economically viable means that the industry is finally talking about the need to implement widespread change. In this climate, it should now be possible to take the decisions required to implement much greater digitization throughout the industry, and make the world of big data a reality.

[1] – google.com/trends – “big data”

Robert Dickson is director – field development project excellence at io. He has 24 years’ experience in the oil and gas industry, having worked as a petroleum engineer and been in management teams on three major field developments from appraisal to first production. For the past 10 years, he has been a management consultant and E&P adviser covering field development, production and operations efficiency and digital oilfield strategy and implementation.