The growing interest in autonomous subsea inspection
This article highlights the newest development of Velella’s project.
Globalization has shaped today’s industries, with seas and oceans playing a central role in resource production and logistics. Renewable energy, maritime transportation, ports, and offshore oil and gas operations all rely on extensive underwater infrastructure. These assets require regular inspection and maintenance to ensure their integrity and performance. While the costs associated with these operations are significant, so are the stakes.
At IVM Technologies, we have already helped bring inspections one step closer to efficiency through photogrammetry. Our solutions enable faster, more cost-effective inspections compared with traditional methods, while delivering high-quality visual data.
Today, operators across the subsea industry are pursuing greater automation and looking to apply it to inspection workflows. With artificial intelligence becoming a key component of subsea robotics, autonomous subsea inspection is just around the corner. By reducing operator workload, improving repeatability, accelerating inspection campaigns, and lowering mobilization costs and operational risks, autonomy has the potential to set a new standard for the industry.
Our commitment to research and development towards fully autonomous subsea inspections
Since 2023, we have been participating in the Velella project, whose objective is to validate key technologies and methodologies needed to enable the deployment of commercial floating offshore wind farms under optimal economic, social, and environmental conditions. As part of this project, IVM Technologies, contributes to the economical and industrial aspect by researching and developing subsea surveillance and inspection methods through a close-range navigation solution based on vision, with the goal of reducing infrastructure operation and maintenance costs.
In our previous article on the subject, we presented a key step of the project: the development of a neural network-based algorithm capable of autonomously identifying and inspecting a mooring chain within a simulated environment. By combining object recognition, inspection path planning, and autonomous camera control, the inspection system demonstrated its ability to perform inspection tasks without operator intervention. The system consists of a Hydro 300 subsea photogrammetry unit mounted on an ROV equipped with multiple monocular cameras, various acoustic sensors, including a multibeam sonar, DVL, and USBL, a camera tilt mechanism, and an inertial measurement unit (IMU).
While this project involves the autonomous control of a ROV, developing a fully autonomous vehicle is not IVM Technologies’ objective. Our ambition is to develop a reliable vision-based close-proximity navigation solution that can be integrated with other sensors and deployed on existing robotic platforms.
To achieve this, our photogrammetry technology was combined with standardized positioning sensors, namely acoustic and inertial systems. This multi-sensor approach leverages the strengths of each sensing modality, providing enhanced robustness, accuracy, and reliability in the challenging conditions of subsea environments.
Integrating our vision technology into a fully autonomous inspection workflow required autonomous control of the ROV throughout the mission. However, autonomous vehicle control was not an objective in itself; rather, it was a necessary step to validate the performance of our vision-based navigation solution under realistic operating conditions.
The next challenge was to bridge the gap between simulation and reality by transforming this theoretical autonomous behaviour into a system capable of operating in real-world subsea conditions.
Further reinforcing algorithms for better autonomy in real subsea contexts
From 3D model to physical mock-up
Following our first inspection campaign at the SEM-REV offshore test site on FLOATGEN’s mooring chains, the collected dataset was processed into a highly accurate 3D model. This digital representation served as the foundation for training our autonomous photogrammetry inspection algorithm.
However, because the SEM-REV site is located far from our offices, we needed a realistic test asset that would allow us to continue development and validation activities under controlled conditions This is essential to address the challenge bridging the gap between simulation and reality. While simulation provides an ideal environment for algorithm development, real-world operations introduce numerous variables, including changing hydrodynamic conditions, sensor noise, variation in visibility, and environmental disturbances.
To recreate these conditions as faithfully as possible, our R&D project management team used the 3D models generated from the mooring chain to manufacture physical replicas of selected points of interest. These full-scale mock-ups provided our team with a realistic testing environment in which they could evaluate, refine, and strengthen the autonomous algorithm before moving to field validation.
Autonomous inspection: testing in our tank
The first real-world tests of the algorithm were conducted in our test tank, a controlled environment where the team could closely monitor its behaviour and analyse its responses. This phase served as a critical intermediate step between simulation and field deployment, allowing us to progressively validate the algorithm’s performance in real-life scenarios. Additionally, it was a crucial step for debugging and performance assessment before exposing the system to more complex conditions. During this stage, several aspects were evaluated, including camera tracking performance, object recognition, consistency of path planning and repeatability of inspection.
These experiments provided valuable insights into the algorithm’s behaviour and helped identify areas for refinement before proceeding to open-water trials.
Towards greater operational realism: testing in port conditions
The next stage of development focuses on further strengthening the algorithm and evaluating its reliability in realistic inspection scenarios.
To improve the robustness of the neural network, it is essential to expose it to the many variables encountered in real-world operations, including turbidity, natural lighting variations, currents, environmental disturbances, positioning drift, and cluttered backgrounds.
By multiplying these challenges and continuously refining the system, we aim to develop a robust autonomous inspection solution capable of reliably inspecting mooring chains in subsea environments.
This summer, a new inspection campaign will be conducted at the site where the project began, bringing together the technological advances achieved throughout this development phase and providing a final opportunity to validate them in operational conditions.
From the Robotics Engineer’s Perspective
Moving from simulation to real-world deployment raises several technical challenges. To better understand this transition, we sat down with the robotics engineer in charge of the project.
Which aspects of the transition from simulation to real-world testing performed as expected?
The simulation was carried out using a simulation tool, complemented by a plugin. This made it possible to generate simulated interfaces with a BlueROV2. As a result, the connectivity and acquisition of the sensors native to this ROV (barometer, compass, IMU, etc.) worked directly upon startup and did not require any modifications.
The algorithms implemented to perform the photogrammetric inspection were tested in simulation through the entire processing chain. A 3D reconstruction of the model was generated using Metashape and produced good results. The same inspection trajectory was successfully followed in the real-world case (despite a few adaptations detailed in the next section). A final processing step was carried out in Metashape, which confirmed its feasibility under real conditions.
The real-time 3D reconstruction was quickly integrated and proved to be functional.
Which aspects of the system required the most adaptation when transitioning from simulation to real-world testing?
The transition from simulation to real-world conditions presents several constraints:
- In simulation, the interactions between the thrusters and the water are difficult to reproduce accurately and vary between fresh water in a pool and seawater. As a result, the transition to real conditions required extensive tuning of the various controller coefficients. For example, the same command sent to control the vertical speed of the ROV does not have the same effect between a simulated ROV in our context and the same ROV operating in real conditions.
- The Hydro images acquired in the simulation environment are very different from those obtained in real-world conditions. In addition, the real mooring chain represented in the simulation is very different from the one specifically built for the needs of the project. It was therefore necessary to train a recognition model specifically adapted to our real-world context.
- Although theoretically functional, the algorithms developed in simulation could not be directly applied in the pool due to space limitations. They therefore required minor adjustments, particularly affecting the length of the chain inspected during data acquisition.
- The Hydro 300 sensor simulated in Gazebo does not provide the same interfaces as the real sensor. It was necessary to develop a dedicated synchronized and latency-free data stream for this robotic application. Similarly, the autonomous management of its tilt angle required specific developments for the software control of this articulation.
What unexpected behaviours emerged when the system was exposed to real-world conditions?
In simulation, and similarly at sea, the mooring chain is the only element visible on the screen. In the test tank, however, the wall in the background can be clearly seen. This creates a problem when performing a 3D reconstruction, which aims to match visible elements between the two stereo cameras in order to estimate the distance between a point and the cameras. In the pool, some elements of the wall are taken into account, which distorts the distance measured to the chain. To overcome this issue, the chosen solution was to consider only the points belonging to the chain identified by the recognition model, thereby avoiding points matched on the wall.
In real-world conditions, the system can encounter currents, which are very difficult to simulate in our environment. Some controllers, particularly PID (Proportional-Integral-Derivative) controllers used to correct position errors, are not well suited to dealing with these currents and are unable to compensate for them effectively. It was therefore necessary to modify them by switching to sliding mode controller, which are more robust in the presence of currents because they act directly on velocity.
Reducing costs of subsea asset inspections
Contact us today to discuss your requirements, explore our solutions, and embark on a transformative journey in underwater inspection.