Nadir Kapetanović, Igor Kvasić, Anja Babić, Ivan Lončar
Despite the challenging time that we’re faced with, IEEE OES University of Zagreb Student Branch Chapter (UNIZG SBC) members are giving their best to continue with the usual research activities. One of the essential parts of these activities are periodic tests of the equipment, marine vehicles and various algorithms developed for those vehicles. At the beginning of April, our members participated in diver-robot interaction trials at our soon-to-be-opened research pool in the newly renovated Laboratory for Underwater Systems and Technologies space. The end of April was marked with our trials regarding underwater communication and localization in Split, Croatia. During those trials, our members also participated in a webinar organized by Blueye company from Norway to show the integration of their Blueye Pro ROV with our newly developed autonomous catamaran. Dissemination of our knowledge and experience and promotion of our SBC are also very important for us, so we participated in two such events in May. On May 13 our members presented their research on underwater human-robot interaction modalities on the seventh in the series of events “Coffee with EUMR” – EUMarineRobots – Marine robotics research infrastructure networks, this time hosted by the University of Zagreb. As every year our research activities, underwater vehicles and ongoing projects were presented at the 2021 Science Festival with the goal of making the public eye more aware of the areas of our work and research laboratories.
Adriatic project remote trials (March 29th to April 8th)
Marine robotics is one of the research fields that largely depends on experimental work. COVID restrictions and lockdown measures in 2020 brought physical collaboration possibilities, joint experiments, equipment sharing and field trials close to impossible. 2020 was the year that coincided with long-term LABUST plans for expanding and upgrading its laboratory facilities. With most of the people stuck working from home and project partners unable to travel abroad, a lot of initial concerns in designing a new experiment workspace went into addressing these challenges. One of the solutions aimed at enabling more work to be done remotely is equipping the indoor pool with underwater cameras, ceiling cameras and ultra-wideband localization systems. The idea of implementing such a setup is streaming as much data as possible and providing situational awareness to a remote user.
The first remote trials that put the described setup in the new laboratory to the test took place in late March and the beginning
of April 2021. These trials were performed in collaboration with the Auckland Bioengineering Institute (ABI) from the University of Auckland, New Zealand. The goal of the trials was to test novel diver-robot interaction modalities developed through ONR-G project Adriatic. The scenario consisted of a diver equipped with sensors embedded in the diving suit and a gesture recognizing diving glove in a pool in Auckland, and the D2 AUV in the LABUST pool in Zagreb collaborating together. The diver used the smart diving glove to recognize gestures and send them acoustically to a poolside PC. The commands are then sent over a http server to a poolside PC in Zagreb and again translated to acoustic commands transmitted to the underwater vehicle. The vehicle executed simple movement actions mapped to these commands. The underwater cameras, as well as the cameras mounted on the vehicle, were streamed to Auckland to provide situational awareness, and the diver received a haptic feedback through the glove when a gesture was recognized and sent successfully.
Considering the distance between ABI and LABUST laboratories and the current travel and collaboration restrictions in force, it was practically impossible to do a joint experiment in person. Introducing existing and new technologies towards enabling such remote collaboration experiences promotes equipment and infrastructure sharing and helps crossing barriers standing in the way of science.
CUV-ME trials (April 19th to 24th)
One of the significant challenges of underwater robotics is underwater localization on which many high-level functionalities such as mission/path planning and following rely. Trials that were organized from the 19th to 24th of April, 2021, in Split, Croatia, had inter-vehicle underwater localization and communication testing as the main objective. Degaussian station at Marjan peninsula, managed by the Republic of Croatia Armed Forces (OSRH), was a great location to work at with a full logistical support (location, work tent, divers, boats, etc.) provided by OSRH.
These trials were organized in the scope of the CUV-ME project (Cooperative Unmanned Vehicles in the Maritime Environment). Sea trials were a great opportunity for all vehicles to be deployed at the same time and to work cooperatively. The primary objective was to assess the current navigational accuracy of all the mentioned underwater platforms, and the ability to communicate between surface and underwater platforms.
It was planned that the ASV would track the AUVs in its downward looking camera as a ground truth, but also provide a USBL beacon for acoustic underwater localization of those vehicles. A firehose was spread along a 200m long transect along which LAUV Lupis traversed and detected the firehose in its downward looking camera and in its side-scan sonar.
The D2 AUV was used to detect and track LAUV Lupis in its forward looking sonar imagery. All these data will be post-processed by a sensor fusion method in order to quantify how much localization precision improvement does each and every additional sensor bring compared to the initial dead reckoning (with a DVL in the case of LAUV Lupis) navigation.
Another objective of these trials was to record datasets for diver detection in forward looking sonar imagery and for the detection of diver presence by detecting diver breathing/bubbles sounds recorded by a hydrophone.
The trials were successful at providing a proof of concept for cross-country USV-AUV communication and data exchange. Data collection was completed successfully with shown reproducibility and repeatability. Additionally, capability to exchange underwater mapping data formats between the University of Zagreb and OSRH was validated.
Potential future testing should focus on research in fully autonomous detection, acquisition and tracking of AUV targets in sonar imaging. Autonomous visual tracking from the surface would provide ground truth for evaluating navigation precision of AUVs. Including OSRH autonomous vehicles into this framework would be beneficial for showing easy transfer of technology between different vehicles and to evaluate the current readiness and navigation precision of OSRH vehicles.
HEKTOR project field tests (April 21st)
In the scope of the HEKTOR project (Heterogeneous autonomous robotic system in viticulture and mariculture, http://hektor.fer.hr/) our SBC members got a chance to acquire a Blueye Pro ROV from the Norwegian robotics company of the same name (https://www.blueyerobotics.com/).
HEKTOR is conceived as a modular and autonomous robotic system, adapted for various missions in viticulture and mariculture with the anticipated possibility of human intervention while performing various work, inspection and intervention tasks. The main objective of the HEKTOR project is to provide a systematic solution for the coordination of smart heterogeneous robots/vehicles (marine, land and air) capable of autonomously collaborating and distributing tasks in open unstructured space/waters.
One of the tasks of the HEKTOR project is to develop an autonomous surface vehicle (ASV) that could also be a docking station for the ROV, as well as provide a landing platform for an unmanned aerial vehicle (UAV). Because of these significant payload requirements, our previously developed ASV model (e.g., ASV Proteus mentioned above) had to be redesigned into a larger catamaran-shape ASV.
Since our members were already in at the Degaussian station, they also took time to launch the ASV Korkyra on its maiden voyage on the 21st of April, 2021, and test its manual and automatic controls and integration of the surveillance IP camera. Also, the ROS2 package that was developed by our members, which interfaces Blueye’s SDK with ROS2 for future autonomy tasks, was tested, i.e., data sending between the ROV and the operator’s PC with the ASV acting as a data relay in between the two.
Blueye webinar (April 22nd)
On the 22nd of April,, 2021 the Blueye company organized a one-hour webinar named “Customize your setup with the Blueye SDK and API”
(https://www.blueyerobotics.com/webinar/customize-your-setup-with-the-blueye-sdk-and-api) hosted by Andreas Viggen, company’s Senior Software Engineer. Blue ROVs are normally controlled by the Blueye app running on either iOS or Android devices. However, some of their customers have unique use-cases where they want to control the drones with their own software. Therefore, the Blueye company has developed the Blueye SDK — a python library that allows you to control the drone programmatically within minutes.
During the webinar, Andreas Viggen and Nadir Kapetanović (IEEE OES UNIZG SBC) have shown how to get started with the SDK. Andreas presented some stories from their customers who have utilized the SDK for controlling multiple drones simultaneously and others that have improved their post-processing sequence of the media and log files via the API.
Nadir presented the HEKTOR project use case for the Blueye ROV, and described the Blueye SDK-ROS2 integration as a necessary functionality for any kind of future autonomous inspection tasks. Also, the guests of the webinar had the chance to watch a live demo of the ASV Korkyra and Blueye Pro ROV deployment and manual control, where the ASV was basically a data relay between operator’s PC and the ROV. The complete recording of the webinar can be found on https://youtu.be/243JWUpxBY8.
Coffee with EUMR webinar (May 13th)
As partners in the European Union’s Horizon 2020 project EUMarineRobots (EUMR), the Faculty of Electrical Engineering and Computing, University of Zagreb, hosted the Seventh Coffee with EUMR webinar series, an online educational and training session with invited talks, exhibits and TNA experiments.
The scope of the EUMR project is to provide an access-infrastructure for the deployment of a full-range of aerial, surface, and subsurface marine robotic assets. UNIZG is part of the consortium that comprises 15 partners from 10 European countries who, collectively, can deploy a comprehensive portfolio of marine robotic assets with required associated support assets and expertise with a capital value well in-excess of €500M. Igor Kvasić presented LABUST’s approach on underwater human-robot interaction capabilities. Part of the presentation was dedicated to acquainting the audience with the soon to be opened laboratory facilities and subsystems that enable remote access work, as well as the latest joint remote experiment results with TNA partners.
Suleman Mazhar has been working as a professor in Information & Communication Engineering at Harbin Engineering University (China) since July 2019. He did PhD from Tokyo University (Japan) and postdoctorate from Georgetown University (Washington DC, USA). He had BS-CS from FAST-NUCES (Lahore) and MS from GIK Institute (Pakistan). He is TYSP young scientist fellow (Ministry of Science & Technology China) and have won several research grants from international organizations such as DAAD (Germany), ICIMOD (Nepal), NRPU (Higher Education Commission Pakistan), WWF (Worldwide Fund for Nature) Pakistan. His research focus is deep learning and signal processing applications for environmental monitoring, with particular focus on underwater acoustics, and marine mammal conservation. He is a reviewer for professional journals such as Journal of Acoustical Society (America), IEEE Journal of Oceanic Engineering, IEEE Sensors Journal, Applied Acoustics, IEEE Transactions on Intelligent Transportation Systems.
Peng Ren is a full professor with the College of Oceanography and Space Informatics, China University of Petroleum (East China). He is the director of Qingdao International Research Center for Intelligent Forecast and Detection of Oceanic Catastrophes. He received the K. M. Scott Prize from the University of York, the Natural Science award (first rank) from China Institute of Electronics, and the Eduardo Caianiello Best Student Paper Award from 18th International Conference on Image Analysis and Processing as one co-author. He has served as an associate editor of IEEE Transactions on Geoscience and Remote Sensing.
Mohd Rizal Arshad is a full professor at the School of Electrical and Electronic Engineering at Universiti Sains Malaysia (USM), Malaysia, where he specializes in ocean robotics technology and intelligent system. He received his B.Eng. in Medical Electronics & Instrumentation and PhD in Electronic Engineering from University of Liverpool, UK in 1994 and 1999, respectively. He completed his MSc. in Electronic Control Engineering from the University of Salford, UK in Dec 1995. He has supervised many postgraduate students and published extensively in local and international publications. He is a senior member of the IEEE, and was awarded IEEE OES Presidential Award in 2019.
Itzik Klein is an Assistant Professor, heading the Autonomous Navigation and Sensor Fusion Lab, at the Charney School of Marine Sciences, Hatter Department of Marine Technologies, University of Haifa. He is an IEEE Senior Member and a member of the IEEE Journal of Indoor and Seamless Positioning and Navigation (J-ISPIN) Editorial Board. Prior to joining the University of Haifa, he worked at leading companies in Israel on navigation topics for more than 15 years. He has a wide range of experience in navigation systems and sensor fusion from both industry and academic perspectives. His research interests lie in the intersection of artificial intelligence with inertial sensing, sensor fusion, and autonomous underwater vehicles.
John R. Potter (IEEE M’94, SM’02, F’18) graduated in the previous century with a joint honours Mathematics and Physics Degree from Bristol and a PhD. in Glaciology and Oceanography from Cambridge, UK studying Antarctic ice mass balance, where he spent four consecutive summers. This work helped underscore the non-linear fragility of polar ice to climate change and led to him receiving the Polar Medal from Queen Elizabeth II in 1988.
Nick is a Visiting Fellow at the UK National Oceanographic Center, Southampton His nomination was endorsed by the Underwater Acoustics Technology Committee. He had worked as a Research Associate and Lecturer at University of Birmingham and has been working as a Research Scientist at the Applied Research Laboratory, University of Texas, Austin. He has also served as a Program Officer at the Office of Naval Research Global. He is a senior member of IEEE (OES) and a Fellow of Acoustical Society of America (ASA). Nick has also been serving as Assoc. Editor for IEEE JoE and JASA. He is widely acknowledged for his expertise are seabed acoustics, parametric array modeling, sonar beamformer, underwater signal processing.
Maurizio Migliaccio (M’91-SM’00-F’17) is Full professor of Electromagnetics at Università di Napoli Parthenope (Italy) and was Affiliated Full Professor at NOVA Southeastern University, Fort Lauderdale, FL (USA). He has been teaching Microwave Remote Sensing since 1994. He was visiting scientist at Deutsche Forschungsanstalt fur Lüft und Raumfahrt (DLR), Oberpfaffenhofen, Germany. He was member of the Italian Space Agency (ASI) scientific committee. He was member of the ASI CosmoSkyMed second generation panel. He was e-geos AdCom member. He was Italian delegate of the ESA PB-EO board. He was Member of South Africa Expert Review Panel for Space Exploration. He serves as reviewer for the UE, Italian Research Ministry (MIUR), NCST, Kazakhstan and Hong Kong Research board. He lectured in USA, Canada, Brazil, China, Hong Kong, Germany, Spain, Czech Republic, Switzerland and Italy. He was Italian delegate at UE COST SMOS Mode Action. He is listed in the Italian Top Scientists. He is an IEEE Trans. Geoscience and Remote Sensing AE, International Journal of Remote Sensing AE, and was IEEE Journal of Oceanic Engineering AE Special Issue on Radar for Marine and Maritime Remote Sensing, IEEE JSTARS AE of the Special Issue on CosmoSKyMed, Member of the Indian Journal of Radio & Space Physics Editorial board. His main current scientific interests cover SAR sea oil slick and man-made target monitoring, remote sensing for marine and coastal applications, remote sensing for agriculture monitoring, polarimetry, inverse problems for resolution enhancement, reverberating chambers. He published about 160 peer-reviewed journal papers on remote sensing and applied electromagnetics.
He has developed various types of Autonomous Underwater Vehicles (AUVs) and related application technologies including navigation methods, a new sensing method using a chemical sensor, precise seafloor mapping methods, a precise seabed positioning system with a resolution of a few centimeters, a new sensing system of the thickness of cobalt-rich crust; and more. He has shown, by using these technologies that AUVs are practicable and valuable tools for deep-sea exploration.
Donna Kocak has had an outstanding career in defense and scientific projects developing and applying solutions in subsea optics, imaging and robotics. She graduated with an M.Sc in Computer Science in 1997 from the University of Central Florida; an MBA in 2008 from the University of Florida; and M.Sc in Industrial Engineering in 2011 from the University of Central Florida. She is currently a Senior Scientist, Advanced Concepts Engineering, and Fellow at the Harris Corporation in Melbourne, Florida, where she has developed novel optical imaging and communication solutions for under-sea defense and scientific projects. Prior to 2008 Donna Kocak was Founder and President of Green Sky Imaging, LLC (GSI) who developed laser/video photogrammetry software for underwater inspection and survey. Her earlier career positions were with Naval Training Systems Center, Florida; Harbor Branch Oceanographic Institution, Florida; eMerge Interactive; and the Advanced Technologies Group in Florida.
John Potter has a Joint Honours degree in Mathematics and Physics from Bristol University in the UK and a PhD in Glaciology and Oceanography from the University of Cambridge on research in the Antarctic, for which he was awarded the Polar Medal in 1988. John has worked on polar oceanography, underwater acoustics, ambient noise (including imaging), marine mammals, communications, IoUT, autonomous vehicles and strategic development. He has 40 years’ international experience working at the British Antarctic Survey in the UK, NATO in Italy, SIO in California, NUS in Singapore and most recently at NTNU in Norway. John is a Fellow of the IEEE and MTS, an Associate Editor for the IEEE Journal of Oceanic Engineering, IEEE OES Distinguished Lecturer, PADI Master Scuba Diver Trainer & an International Fellow of the Explorer’s Club.
Dr. James V. Candy is the Chief Scientist for Engineering and former Director of the Center for Advanced Signal & Image Sciences at the University of California, Lawrence Livermore National Laboratory. Dr. Candy received a commission in the USAF in 1967 and was a Systems Engineer/Test Director from 1967 to 1971. He has been a Researcher at the Lawrence Livermore National Laboratory since 1976 holding various positions including that of Project Engineer for Signal Processing and Thrust Area Leader for Signal and Control Engineering. Educationally, he received his B.S.E.E. degree from the University of Cincinnati and his M.S.E. and Ph.D. degrees in Electrical Engineering from the University of Florida, Gainesville. He is a registered Control System Engineer in the state of California. He has been an Adjunct Professor at San Francisco State University, University of Santa Clara, and UC Berkeley, Extension teaching graduate courses in signal and image processing. He is an Adjunct Full-Professor at the University of California, Santa Barbara. Dr. Candy is a Fellow of the IEEE and a Fellow of the Acoustical Society of America (ASA) and elected as a Life Member (Fellow) at the University of Cambridge (Clare Hall College). He is a member of Eta Kappa Nu and Phi Kappa Phi honorary societies. He was elected as a Distinguished Alumnus by the University of Cincinnati. Dr. Candy received the IEEE Distinguished Technical Achievement Award for the “development of model-based signal processing in ocean acoustics.” Dr. Candy was selected as a IEEE Distinguished Lecturer for oceanic signal processing as well as presenting an IEEE tutorial on advanced signal processing available through their video website courses. He was nominated for the prestigious Edward Teller Fellowship at Lawrence Livermore National Laboratory. Dr. Candy was awarded the Interdisciplinary Helmholtz-Rayleigh Silver Medal in Signal Processing/Underwater Acoustics by the Acoustical Society of America for his technical contributions. He has published over 225 journal articles, book chapters, and technical reports as well as written three texts in signal processing, “Signal Processing: the Model-Based Approach,” (McGraw-Hill, 1986), “Signal Processing: the Modern Approach,” (McGraw-Hill, 1988), “Model-Based Signal Processing,” (Wiley/IEEE Press, 2006) and “Bayesian Signal Processing: Classical, Modern and Particle Filtering” (Wiley/IEEE Press, 2009). He was the General Chairman of the inaugural 2006 IEEE Nonlinear Statistical Signal Processing Workshop held at the Corpus Christi College, University of Cambridge. He has presented a variety of short courses and tutorials sponsored by the IEEE and ASA in Applied Signal Processing, Spectral Estimation, Advanced Digital Signal Processing, Applied Model-Based Signal Processing, Applied Acoustical Signal Processing, Model-Based Ocean Acoustic Signal Processing and Bayesian Signal Processing for IEEE Oceanic Engineering Society/ASA. He has also presented short courses in Applied Model-Based Signal Processing for the SPIE Optical Society. He is currently the IEEE Chair of the Technical Committee on “Sonar Signal and Image Processing” and was the Chair of the ASA Technical Committee on “Signal Processing in Acoustics” as well as being an Associate Editor for Signal Processing of ASA (on-line JASAXL). He was recently nominated for the Vice Presidency of the ASA and elected as a member of the Administrative Committee of IEEE OES. His research interests include Bayesian estimation, identification, spatial estimation, signal and image processing, array signal processing, nonlinear signal processing, tomography, sonar/radar processing and biomedical applications.
Kenneth Foote is a Senior Scientist at the Woods Hole Oceanographic Institution. He received a B.S. in Electrical Engineering from The George Washington University in 1968, and a Ph.D. in Physics from Brown University in 1973. He was an engineer at Raytheon Company, 1968-1974; postdoctoral scholar at Loughborough University of Technology, 1974-1975; research fellow and substitute lecturer at the University of Bergen, 1975-1981. He began working at the Institute of Marine Research, Bergen, in 1979; joined the Woods Hole Oceanographic Institution in 1999. His general area of expertise is in underwater sound scattering, with applications to the quantification of fish, other aquatic organisms, and physical scatterers in the water column and on the seafloor. In developing and transitioning acoustic methods and instruments to operations at sea, he has worked from 77°N to 55°S.
René Garello, professor at Télécom Bretagne, Fellow IEEE, co-leader of the TOMS (Traitements, Observations et Méthodes Statistiques) research team, in Pôle CID of the UMR CNRS 3192 Lab-STICC.
Professor Mal Heron is Adjunct Professor in the Marine Geophysical Laboratory at James Cook University in Townsville, Australia, and is CEO of Portmap Remote Ocean Sensing Pty Ltd. His PhD work in Auckland, New Zealand, was on radio-wave probing of the ionosphere, and that is reflected in his early ionospheric papers. He changed research fields to the scattering of HF radio waves from the ocean surface during the 1980s. Through the 1990s his research has broadened into oceanographic phenomena which can be studied by remote sensing, including HF radar and salinity mapping from airborne microwave radiometers . Throughout, there have been one-off papers where he has been involved in solving a problem in a cognate area like medical physics, and paleobiogeography. Occasionally, he has diverted into side-tracks like a burst of papers on the effect of bushfires on radio communications. His present project of the Australian Coastal Ocean Radar Network (ACORN) is about the development of new processing methods and applications of HF radar data to address oceanography problems. He is currently promoting the use of high resolution VHF ocean radars, based on the PortMap high resolution radar.
Hanu Singh graduated B.S. ECE and Computer Science (1989) from George Mason University and Ph.D. (1995) from MIT/Woods Hole.He led the development and commercialization of the Seabed AUV, nine of which are in operation at other universities and government laboratories around the world. He was technical lead for development and operations for Polar AUVs (Jaguar and Puma) and towed vehicles(Camper and Seasled), and the development and commercialization of the Jetyak ASVs, 18 of which are currently in use. He was involved in the development of UAS for polar and oceanographic applications, and high resolution multi-sensor acoustic and optical mapping with underwater vehicles on over 55 oceanographic cruises in support of physical oceanography, marine archaeology, biology, fisheries, coral reef studies, geology and geophysics and sea-ice studies. He is an accomplished Research Student advisor and has made strong collaborations across the US (including at MIT, SIO, Stanford, Columbia LDEO) and internationally including in the UK, Australia, Canada, Korea, Taiwan, China, Japan, India, Sweden and Norway. Hanu Singh is currently Chair of the IEEE Ocean Engineering Technology Committee on Autonomous Marine Systems with responsibilities that include organizing the biennial IEEE AUV Conference, 2008 onwards. Associate Editor, IEEE Journal of Oceanic Engineering, 2007-2011. Associate editor, Journal of Field Robotics 2012 onwards.
Milica Stojanovic graduated from the University of Belgrade, Serbia, in 1988, and received the M.S. and Ph.D. degrees in electrical engineering from Northeastern University in Boston, in 1991 and 1993. She was a Principal Scientist at the Massachusetts Institute of Technology, and in 2008 joined Northeastern University, where she is currently a Professor of electrical and computer engineering. She is also a Guest Investigator at the Woods Hole Oceanographic Institution. Milica’s research interests include digital communications theory, statistical signal processing and wireless networks, and their applications to underwater acoustic systems. She has made pioneering contributions to underwater acoustic communications, and her work has been widely cited. She is a Fellow of the IEEE, and serves as an Associate Editor for its Journal of Oceanic Engineering (and in the past for Transactions on Signal Processing and Transactions on Vehicular Technology). She also serves on the Advisory Board of the IEEE Communication Letters, and chairs the IEEE Ocean Engineering Society’s Technical Committee for Underwater Communication, Navigation and Positioning. Milica is the recipient of the 2015 IEEE/OES Distinguished Technical Achievement Award.
Dr. Paul C. Hines was born and raised in Glace Bay, Cape Breton. From 1977-1981 he attended Dalhousie University, Halifax, Nova Scotia, graduating with a B.Sc. (Hon) in Engineering-Physics.