Taking augmented reality subsea

Mark Stevens and Bob Moschetta, of Oceaneering, discuss the benefits of using augmented reality for subsea training.

The augmented reality view combines the real-time video with the 3D model, which allows operations to occur in real time. This view demonstrates the overlay capability whereas the 3D model is overlaid into the real-time video which provides the augmented reality capability. The bottom view is the real-time video with the model rendered into the ROV operator’s screen. Images from Oceaneering.

Augmented and virtual reality environments are beneficial for offshore operations or any remote environment. The overall risk is reduced and job execution is improved. Specifically, safety is enhanced as the entire subsea environment can come to life and the job can be performed in the virtual environment. Thus, work instructions, procedures and processes which are intended for the actual job can be viewed and tested. This helps ensure problems that affect the equipment, the people and the execution of the job can be corrected ahead of time.

This increased visualization of how the job is supposed to occur provides the opportunity for the continuous improvement cycle to be used as equipment, process or work instructions can be modified based upon the virtual testing of executing the job, thereby reducing the risk and improving efficiency.

Situational awareness

Augmented reality takes into account additional sensor data, which enhances situational awareness. An example would be loading 3D models into a live video display. If the offshore visibility decreases due to fog or muddy water, the 3D models overlaid into the video would provide additional inputs to ensure the job can be completed effectively. Training employees on utilizing the augmented reality tools ensures that they are using all available means to complete the project.

Augmented training can ensure that personnel are aware of the potential hazards presented by the environment, equipment and procedures that will be utilized for jobs. This allows for hazards and inefficiencies to be identified and resolved, thus reducing the likelihood of managing the change, incurring delays and increased risk at the job site. Recommended changes can be communicated to the project managers and engineers to ensure the proposed changes to equipment and processes are adequate for the particular job.

The subsea field view as seen demonstrates the high level tactical view that can be used for subsea operations. This ensures a more complete picture for the ROV operator and subsea engineer and provides for effective field management in low visibility conditions. The use of survey positioning gear subsea allows the equipment to be positioned with centimeter level accuracy.

Training

Performance enhancements for training can be provided by augmented reality due to the increased realism provided by the latest augmented systems. The repeatability of the training scenarios and the utilization of an effective physics engine to replicate the real-world environmental factors (i.e., gravity, wind, and temperature effects) should ensure flawless execution of the actual job. An example of a physics engine would allow a trainee to simulate how an ROV would work as he/she is heading towards an object and the engine was cut off. The physics engine would show the ROV slowing down instead of an instant stop.

Skills and capabilities can be captured and replayed utilizing an augmented reality platform by capturing the user input via all of the control systems. This can be combined with other sensor data such as the position of equipment which can be reviewed with new technicians to demonstrate the most effective process to complete a mission or job. The capturing of the user input would allow an organization to set up a baseline mission profile that can be replayed back to a new user so that their inputs and controls are adjusted to meet or excel the baseline recording.

Equipment

The use of haptic controls and touch sensors should allow for remote operational missions to be achieved while not on-site. Haptic tools recreate the feel of actually touching objects even though the operator is remote. These controls provide realistic feedback to give the user, which creates the feeling they are present in close proximity to the system. Using the robotic optical cameras and controls from the remote system allows the operators to explore and identify hazardous conditions, such as a subsea equipment failure, which has the potential for a subsea release of oil or gas.

Additionally, the utilization of wearable augmented headsets should see an increase in usage in the commercial market. Various products over the next two years will be hitting the market on the consumer side (i.e, Oculus, Sony, and Microsoft) and these should accelerate the usage of the tools and how they can be used in the workplace.

Oceaneering’s experience

Oceaneering utilizes ROV simulators throughout our worldwide training centers. The use of the simulators ensures that the ROV pilots are familiar with the processes and controls for each upcoming mission and also allows for advanced testing of new ROV tooling that was developed for the specific job. Portable simulators are utilized offshore for recurrent training, thus ROV operators can assess, practice and review any changes to the mission. Moreover, potential hazards can be identified and corrected.

The use of augmented training helps to ensure the operator’s situational awareness is such that the loss of visibility that may occur on the actual job should not impede the mission and objectives. This is possible as the augmented visuals provided in the operator display will provide a clear picture of the situation to allow the mission to continue.

This is similar to the aviation community where the pilot’s heads up display (HUD) is augmented with the runway position in zero visibility and the pilot is comfortable completing the landing due to confidence provided by his/her training. Augmented reality training in the aviation world is so effective that the first time an airline co-pilot operates the controls of a commercial aircraft, it is typically with customers onboard. The Pilot-in-Command or senior pilot then provides monitoring of the co-pilot’s activities and this demonstrates the cost savings as well because a dedicated aircraft isn’t required.

Mark Stevens, Director of Oceaneering Communications Solutions (OCS), is an information technology industry veteran with extensive experience in the global oilfield communications segment. He has been with Oceaneering for more than 15 years and is responsible for growing OCS’s global technology business both organically and through strategic acquisitions that further extend the company’s hardware and software development expertise and resources.

Bob Moschetta is the Sr. Vice President of Health Safety Environmental-Training-Quality at Oceaneering International, Inc. He is experienced in integrating industry best practices, HSE management principles, quality improvement processes, and regulatory standards. Bob is a Certified Safety Professional with a master’s degree in safety management from West Virginia University.