Breaking the Surface 2021 – Overview of field trial experiments

Nadir Kapetanović, Anja Babić, Ivan Lončar, Igor Kvasić, Vladimir Slošić, Barbara Arbanas

Members of the IEEE OES University of Zagreb SBC organized a significant number of field trials during Breaking the Surface international interdisciplinary field workshop of maritime robotics and applications that was held in Biograd na Moru, Croatia, during the week of 26 September to 3 October. as well as during the following week. This article brings a short overview of those activities.

HEKTOR project trials

Our SBC members working on the HEKTOR project are in charge of developing the heterogeneous autonomous robotic system in mariculture applications. Mariculture scenarios include an ASV Korkyra (developed by our team), Blueye Pro ROV integrated with the ASV by our team, and a UAV (developed by our colleagues from LARICS team at the University of Zagreb Faculty of Electrical Engineering and Computing). The main objective is to enable autonomous inspection of the fish cages from air, at and below the surface of the sea.

During the BtS 2021 workshop, the performance of vertical rope detection and mission control was tested at the seawater pool in Biograd na Moru, Croatia. A real fish cage net was strung across the pool to emulate a real-world environment, and the ROV was controlled by ROS2 running on a laptop, as shown in the following figure. The performance tests were successful, but the visual detection algorithm should be made even more robust to different lighting conditions, especially early in the morning and at dusk.

Extensive tests of ASV Korkyra’s power consumption were also performed. Minimum and maximum consumption of each subsystem or device were measured and corrected for the baseline consumption. It shows that the autonomy of the ASV Korkyra ranges from the theoretical worst-case of 3.22h (when all the motors, computers and subsystems are working at full power) to 20.58h in the best case. It is however realistic to conclude that the average expected autonomy would range from 10h to 11h.

Control of the ROV by the ASV needs position feedback. Thus, WaterLinked Underwater GPS (UWGPS) G2 SBL topside system with a bottomside Locator U1 system met all these requirements and was integrated into the ASV Korkyra during the week after the BtS 2021.

Since the integration of the UWGPS G2 with ROS2 was done before BtS 2021, this meant that frame transforms needed to be performed in the post-processing phase taking into account ASV’s known GPS position and orientation in 3D space.

Integration of the ROV onto the ASV and their cooperative path planning in autonomous inspection missions requires a tether management system (TMS) to be developed together with a docking mechanism (DM). During the BtS 2021 workshop, a prototype control box of the TMS was developed. It enables the manual control mode of the TMS motor but also an automatic control mode from the ASV’s main computer over serial communication.

As mentioned earlier, the UAV will perform fish cage inspections from the air. This will be done in coordination with the ASV Korkyra, so a landing platform (LP) is needed for the UAV to be docked onto the ASV. A great number of test runs were performed to detect possible issues with the LP. Extensive testing was first performed in the lab environment. Similar to the TMS, the LP has a control box allowing manual control, but also automatic control over serial communication integrated with ROS. During Breaking the Surface 2021, additional LP robustness tests were performed and were successful. These tests showed that the auto control mode is fully functional together with serial communication integration with ROS1.

Multifunctional smart buoys trials

The trials taking place after BTS 2021 were the first field trials for the Multifunctional Smart Buoy project, which started in April 2021. The team working on the project prepared two initial prototypes of the underwater sensor units that communicate with a floating buoy unit, forming a marine system for long-term environmental monitoring.

The aim of these initial field trials was to perform full integration of the developed subsystems – Cyclops fluorometer sensors for various water quality measurements, a small acoustic nanomodem for long-range underwater communication, the units’ main electronic boards including an SD-card based logger and a Real Time Clock, as well as a battery assembly composed of lithium-ion cells, all housed within a plexiglass enclosure and equipped with specially-made attachment points for a surface unit’s mooring line.

After integration, two units were vacuum-tested and left to perform an overnight experiment on land, logging their respective sensor data and battery voltage. Once this “test run” was successfully concluded, the units’ batteries were recharged and they were deployed in the seaside pool.
To serve as a surface node and an endpoint for the sensor units’ acoustic packets, a topside laptop was set up with an acoustic modem running into the pool. This laptop collected data sent by the units, parsed it, and sent it to a ThingsBoard IoT dashboard where it was presented in a user-friendly way. All the while, the deployed sensor units were monitored using underwater cameras affixed to the sides of the pool.

Differences in sensor unit behaviour were analysed each day, especially with regards to energy consumption. For the remaining three days of the field trials, the units were recharged and redeployed underwater with the goal of testing different sleep modes and sleep durations, in order to gather valuable data for ensuring long-term autonomy of the final buoy system, while achieving regular and persistent acoustic communication of all gathered sensor measurements.

The trials after BtS 2021 proved incredibly useful as both data and experience gathered during the first-ever deployment of the developed prototypes lead to excellent insights regarding the current state of the system, as well as extensive and specific plans for future work.

 

Autonomous multipurpose ship project trials

During and after the Breaking the Surface 2021 workshop, the initial trials for the Autonomous ship project took place. In the project, the plan is to build a multipurpose autonomous ship in collaboration with Brodosplit shipyard. The ship will be a fast response vessel for firefighting and oil recovery. The goal is to mitigate negative impact of maritime accidents on the environment. Since the project was still in the design phase as it started in July 2021, we could only rely on equipment available in LABUST.

The goal of the initial trials was to gather perception data from the maritime environment using a lidar. This data will be useful for testing of developed algorithms used for detection of objects commonly encountered in maritime environment and testing of readily available SLAM algorithms. Since the marina is an environment in which you can only rely on perception data for safe navigation, there is a great benefit on having real data for algorithm validation.

During the trials we integrated the lidar to the Korkyra catamaran vehicle. Lidar was mounted together with a camera to a landing platform used in project HEKTOR. For a test run, we first mounted the sensor rig to a box and pushed it around a seawater pool. Results of the lidar processing can be seen from the image below where a reconstructed map of the pool can be seen.

After the initial test, we set out to the sea to gather more relevant data from a marina in Biograd na Moru. The catamaran was driven manually around a marina with close supervision from a dinghy. The data collected will be used as a testing dataset in future publications.

ADRIATIC project trials

As part of the Breaking the Surface 2021 program, our IEEE OES SBC members held an introduction lecture and a hands-on tutorial in Unity simulator related to the ongoing project ADRIATIC – Advancing Diver-Robot Interaction Capabilities. The project focuses on finding innovative and intuitive ways of interaction between human divers and autonomous underwater vehicles (AUVs). Within the project the collaboration scheme between the diver and the AUV envisions the robot vehicle to take the place of a robotic diving buddy, with the prime goal of observing the diver and determining his physiological parameters such as breathing, hearth and motion rate and to allow the detection of critical diver states and in turn increase safety. One of the project scenarios includes the autonomous underwater vehicle as a diver navigator, where the vehicle uses its advanced localization and navigation capabilities to navigate the diver to a target or point of interest. The participants have had the chance to experience that scenario in a simulated environment using Unity 3D. The tutorial had multiple objectives:

• Showcasing the Unity marine simulator developed within the Laboratory and demonstrating its capabilities
• Evaluating the feasibility of using this type of underwater simulation for developing algorithms and training
• Gathering valuable mission data and feedback from a larger audience of experts from various marine-related fields

The simulator works as a first person-view game played from a diver’s perspective. The diver is controlled using “WASD” keyboard controls and mouse for movement, plus “C” and “SPACE” keys for diving and ascending when underwater. The task is divided into two separate missions. The mission starts as the dive commences at the diving vessel. The first objective is to navigate the diver underwater using keyboard controls to two separate targets at previously known positions. The first target is a sunken plane and the second is a shipwreck. In the first mission the participants navigate to the position using a diving compass and depth sensor gauge, which is the usual equipment that divers would have at their disposal.

At the dive start, the player is given only the direction and distance towards each target and has to overcome waves, sea currents and low visibility underwater in order to reach the targets.  In the second mission the goal is the same, but this time the navigation is aided by the diving robot. The robot uses its simulated proprioceptive navigational sensors to determine its location and positions itself on an imaginary safety circle between the diver and the target, pointing to the waypoint from the divers’ perspective. While the visualization and simulation parts are running in Unity, all the navigation algorithms are running in ROS and are connected to Unity, which generates the simulated sensor readings. At the end of the two missions, participants can view their results in the form of displaying the trajectories in 2D and 3D along with the statistics such as distance traveled and time of each mission. At the end, the participants were given a short output survey to express their feedback and help further develop the simulator. Future plans of the project include analyzing the navigation data and statistics, along with the gathered feedback, and publish the results in a future paper.

ROADMAP project trials

During the trials on BtS 2021, another experiment was performed in the field of underwater localization. Most of the projects that LABUST is involved in demand precise localization of the AUVs. Therefore, a new localization technique – breadcrumb localization-  was introduced as an upgrade to the localization algorithms that we already use at LABUST.

The main goal of the experiment was to collect data that will confirm the theoretical background of breadcrumb localization. The experiment was performed in the Olympic seawater swimming pool (dimensions: 50m by 25m). Equipment used:

• autonomous surface vehicle aPad (further in the text – ASV aPad) with mounted underwater acoustic pingers and camera looking downwards
• five underwater acoustic pingers in waterproof casing with anchors
• GPS RTK system for precise navigation and localization
• PC with ROS installed.

When it comes to underwater localization of the AUVs, a common problem regarding proprioceptive sensors, such as IMU, is a drift error that accumulates during time as AUV executes its mission. The idea of the breadcrumb algorithm was to minimize the error of the localization with so-called “breadcrumbs” that are actually underwater acoustic pingers deployed one after another during the mission. ASV aPad (or AUV in a different scenario) can measure the distance between itself and each of the breadcrumbs individually. Those measurements can be used to correct the localization estimate from IMU and other sensors that are mounted on/inside the AUV/ASV.

Before the experiment aPad was serviced, all of the needed sensors for experiments were mounted on it. Breadcrumbs housings were made at LABUST out of plexiglass tubes inside of which is the pingers’ electronics and battery.

During the experiment, there has been poor weather conditions that postponed the experiment for some time and caused a lower amount of recorded data. There were also some technical problems with the aPad’s IMU that wasn’t able to calibrate itself. One breadcrumb was replaced because of the seawater penetration inside of the housing due to a defective seal.

The recorded data will be used for the confirmation of the breadcrumb localization theoretical background and some other algorithms that are being developed and tested at the moment at LABUST.

Conclusion

BtS 2021 workshop and the field trials in the following week were a successful demonstration for many of our research projects. It gave us a chance to involve our fellow IEEE OES UNIZG SBC members to participate actively. During the field trials of 5 research projects, various vehicles and subsystems were tested: 2 ASVs, 1 ROV, 1AUV, 1 1UAV, 1 landing platform, 1 TMS, and a 2-node underwater sensor network (UWSN). Several cameras, lidar, environmental sensors, acoustical localization and communication systems, and sonars were used for performance testing of mission planning, navigation guidance and control systems, as well as long-term Internet of underwater things (IoUWT) modules. Furthermore, a transition from real-life into virtual reality (VR) experiments took place to test human-robot interaction (HRI) and robot-aided underwater diver navigation.

We are looking forward to our Spring/Summer field trials and of course yet another edition of Breaking the Surface in Autumn 2022. During this year’s BtS, HEKTOR project is planned to have final experiments of AS-ROV-UAV cooperative fish net pen inspection. BOVE project will have its final experiments as well, testing the long-term environmental monitoring by a UWSN in all configurations defined by the project. For the Autonomous ship project, the plan is to add either virtual or real obstacles in the sea to demonstrate collision avoidance algorithms while autonomously navigating a predefined route. Breadcrumb underwater localization algorithm will be tested in the scope of the ROADMAP project, this time with an AUV and a diver.