Satellite dark vessel detection for maritime domain awareness

Case Study · 22 November 2022 · 15 minute read

Abstract

The detection of vessels at sea that are not self-reporting their position using automated geolocation systems, i.e., dark vessels, poses a particular challenge to maritime domain awareness. Satellites have a unique advantage in dark vessel detection as they can scan larger ocean areas more often than any other surveillance technology. In this article, the science team of Starboard Maritime Intelligence reports insights from their dark vessel detection operations using optical, radio frequency and synthetic aperture radar satellite sensors. We found that an effective satellite dark vessel monitoring programme requires a multi-sensor approach and patrol assets should be available for verification and interrogation of dark targets. In conclusion, dark illegal fishing threatens the sustainability of fish stocks and the economy of nations relying on fisheries income, but evolving technology and the increasing number of satellites mean that satellite sensors are quickly closing the surveillance gap for large ocean areas.

This article was first published in the INFOFISH International Issue 6/2022 (November/December 2022).

Introduction

For fisheries operators, an accurate knowledge of ships in the vicinity, environmental conditions, and the location of fishing areas are essential for safe and legal conduct. Likewise, this situational awareness is indispensable for monitoring, control, and surveillance of fishing activity by governmental or regional fisheries management organisations.

Maritime domain awareness (MDA) is the effective understanding of anything associated with the maritime domain that could impact security, safety, the economy, or the environment. MDA encompasses many types of intelligence, but a fundamental requirement is knowledge of the locations of vessels at sea and information pertaining to their journey, such as their destination and the activities they are engaged in.

Fisheries-related vessel activities such as gear setting, steaming, and encounters for transshipment or resupply make up a significant fraction of maritime traffic. For example, in an area of the western equatorial Pacific north of Papua New Guinea, a quarter of vessels transmitting their positions are fishing boats (Figure 1).

Regulations of the International Maritime Organization require most large vessels to be equipped with automatic identification system (AIS) transponders providing geoposition and ship identification. Many small vessels, including recreational craft and fishing boats, are also AIS equipped voluntarily or because of regional regulatory requirements. AIS transmissions are received globally through networks of shore- and satellite-based receivers and are commercially available. This makes AIS an important component of MDA activities.

Some fishing vessels are not required to be fitted with AIS transceivers, but most countries and many regional fisheries organisations require fishing vessels active in their jurisdictions to be operating vessel monitoring system (VMS) transponders. Similar to AIS, VMS provides geolocations at set time intervals and other identifying information. However, VMS data is not public, but exclusive to the governing organisation.

A difficult challenge for MDA is to detect and monitor vessels that don’t report their positions, so-called dark vessels. Vessels may go dark because of technical failures of position transponders, gaps in coverage by signal receivers, or because they have been turned off purposefully by vessel operators. Finding a dark vessel in the vastness of the ocean, combined with the physical limitations of line-of-sight results in the proverbial “needle in a haystack” problem.

For example an average human standing at sea level can observe an area of about 70 km², with Earth’s curvature obscuring anything below the horizon about 5 km away. Observing from a 20 m lookout increases this area to about 800 km². In general, the same distance rules apply to radar installations used for maritime domain awareness, as well as for airborne remote sensing. For example, for a surveillance aircraft flying at 0.8 km altitude, the horizon is about 100 km away yielding a survey area of 32,000 km.

As the viewing radius increases with the observer’s altitude, satellite observations have a unique advantage: they can potentially observe vast areas of the ocean repeatedly. But all satellite sensors have specific limits to the size of the area that can be scanned (Figure 2), thresholds for the size or type of objects, and orbital dynamics dictate the timing and frequency of scanning opportunities.

In this article we describe our experiences from conducting numerous satellite surveillance campaigns at Starboard. We illustrate how satellites can support comprehensive monitoring of significant ocean areas both during operations where dark targets can be investigated by patrol assets, and for enhanced MDA over extended time periods. We first describe the advantages and limitations of three types of satellite sensors that are often used for vessel detection.

Satellite vessel detection

Most Earth observation satellites orbit our planet at altitudes between 300 to 1,000 km (low-Earth orbit) and scan the surface of our planet daily using a variety of different sensors. Three fundamental classes of sensors are routinely used for vessel detection: optical imagers, synthetic aperture radar (SAR), and radio frequency (RF) geolocation.

Optical imagers	Satellite-borne optical imagers produce the pictures of Earth’s surface that are commonly known as ‘satellite images’. The detection of vessels in satellite images is possible within the size limits given by pixel resolution. For example, vessels of 30 m length are visible in an image with 10 m pixel resolution (Figure 3a) and better pixel resolution is needed for smaller boats (Figure 3b). Optical data is available for free from satellites operated by large space agencies such as NASA and the European Space Agency (ESA). Such satellites cover large portions of the planet routinely, and images have a resolution of 10 m or more per pixel. Commercial providers offer better resolution images from about 0.3 m per pixel, but typically do not acquire data continuously. This makes it necessary to order images in advance so that satellites can be tasked to acquire data at certain times in areas of interest. Table 1: Information on a selection of satellites with optical imagers suitable for the monitoring of maritime domains (in order of decreasing footprint size). Satellite Operator Pixel resolution Imaging footprint Return period Tasking Sentinel-2 ESA 10 m 290 km swath 5 days PlanetScope Planet 3 m 25 x 16 km Almost daily SPOT Airbus 1.5 m 60 km swath Almost daily ✓ SkySat Planet 0.5 m 24 x 7 km Daily ✓ Pléiades Airbus 0.5 m 20 km swath Almost daily ✓ There is generally a tradeoff in coverage (imaging footprint) and pixel resolution. For example, ESA’s Sentinel-2 acquires data in continuous strips over land and coastal areas with 290 km swath width at 10 m pixel resolution, while PlanetScope images have a 25 by 16 km footprint at 3 m resolution. In other words, the resolution requirements to detect vessels simultaneously limits the maximum coverage area, so that ‘you need to know where to look’, which is often not applicable in the open ocean. An additional requirement for optical sensors is the need for a cloud-free view. Together, these limitations mean that optical imagers are of limited use for vessel detection at mission-critical times and areas of interest.
Synthetic aperture radar	Satellite-based synthetic aperture radar (SAR) systems transmit microwave radiation and then measure echoes received from the backscattered signal. By utilising longer wavelengths than optical systems, radar is able to penetrate clouds with minimal distortion. This enables measurements to be made in all weather conditions which is a significant advantage for most marine environments. A number of different SAR satellites are available, providing both free and commercial imagery over regions from 25 to 225,000 km2 at spatial resolutions between 0.35 and 50 metres. Some of the SAR satellites that have been used for ship detection include (in order of decreasing swath width): Radarsat-2: Canadian commercial C-band radar satellite with a dedicated large-scale ship detection acquisition mode at 450 km swath width; this satellite must be tasked to acquire data in areas of interest. Sentinel-1: ESA C-band radar satellite providing freely available data through the Copernicus Programme; Sentinel-1 scans land and coastal oceans at 250 km swath width and can not be tasked. TerraSAR-X/TanDEM-X: Twin satellite X-band constellation by the German Aerospace Centre; scanning modes for ship detection include ScanSAR at 100 km swath width and Wide ScanSAR at 270 km swath width. ICEYE: Finnish space startup offering tasking of its SAR constellation with a range of acquisition modes at a swath width of 5 to 100 km. Capella: Space startup company providing high-resolution SAR data at a swath width of 5 km. Capella achieves short revisit periods through a growing constellation of small satellites. There are very few satellites available that can deliver large-area SAR imagery with suitable resolution for vessel detection. A system that has demonstrated good results for large ocean regions is the Radarsat-2 satellite. Using the Ship Detection (DVWF) mode, Radarsat-2 can cover an area of about 225,000 km2 with a pixel size of 20 metres (Figure 4). The minimum size of a typical metal-hulled vessel that can be reliably detected from Radarsat-2 images in DVWF mode is approximately 30 metres. The selection of the system and operating parameters is critical in order to optimise vessel detection capabilities. A SAR image (Figure 4) is a reflectivity map where the intensity of the backscattered signal is dependent on the physical properties of the reflecting surface. Surfaces that are relatively flat and smooth, such as the ocean, reflect the transmitted energy away from the satellite; this results in dark areas within the SAR image. Rough and complex surfaces, such as a vessel, reflect energy back to the satellite, providing a greater return at the radar receiver; this results in bright spots within the SAR image. Other bright structures are caused by oceanographic and meteorological conditions. Separating out spots of high signal intensity that might represent ships from this noisy background is a processing-intensive task.
Radio frequency geolocation	Recent developments in satellite technology for maritime domain awareness have focused on methods to detect and geolocate vessels from their radio frequency (RF) emissions. The main source of RF signals on ships are navigational radars, powerful beacons that sweep beams of RF energy around the horizon at a rate typically between 20 and 60 times per minute. Most vessels whilst at sea will have one or more navigational radars continuously operating, emitting strong pulses in the S- and X-bands. Other types of RF emission may also occur, such as the use of VHF for radio communications. RF emissions from navigational radar propagate into space where satellites can detect them. If the angle-of-arrival at the satellite is measured, then the location of the emitter can be derived, to within some limited accuracy. Such data collections are made during a short interval (a few seconds) when a satellite in low-Earth orbit is making an overpass. The main advantage of RF geolocation over SAR imaging is that a larger area can be covered, generally at a lower cost. As such this is a promising new, albeit still maturing, technology. Some of the commercial companies providing RF detection products include (in alphabetical order): HawkEye 360: US geospatial analytics company using a constellation of small satellites to collect and geolocate RF signals for commercial use. The RF data collected includes UHF and VHF radio communications, X- and S-band marine radars, and L-band mobile satellite devices. Kleos Space: Luxembourg-based startup launching satellites for RF detection. Initially concentrating on the geolocation of VHF communication devices with plans to expand to marine navigational radars in the future. Unseenlabs: French company offering a radio frequency monitoring service using satellites. The RF data collected includes marine radar signals.

Dark vessel detection

The process of dark vessel detection starts with processing of the satellite data to reveal targets that could be ships. Data from optical and SAR satellite acquisitions are georeferenced images that have to be processed through feature detection algorithms to discern characteristic anomalies from the image background (Figure 4). RF scans on the other hand are processed by the data provider to deliver a list of geolocations of the radar emitter and their emission characteristics (Figure 5). While this is convenient, it does not allow assessment of the raw data background in case of suspected false detections.

The second step is matching satellite detections against known AIS and VMS ship positions. This matching has to take into account inherent uncertainty around the satellite-derived positions and possible gaps in the vessel track. For RF data, we consider a match to be a ship with an AIS message from within 5 km of the satellite detection. SAR data has better location accuracy than RF and we consider a match between a SAR detection and an AIS location within 1 km.

The end result is that satellite scans yield a list of detections matched to known vessel locations and a list of unmatched detections. As satellite sensors and processing techniques have the potential of creating false positives, i.e., making a vessel detection where no vessel is present, a dark satellite detection must be qualified as a ‘possible’ dark vessel, or a ‘dark detection’ unless it is independently verified to be a ship, for example by patrol assets.

The Starboard platform automates vessel detection processes, matching of satellite detections against known ship locations, and delivery of results to the end user. This means that actionable information is available almost immediately upon delivery of the satellite data, which is especially important when satellite acquisitions are in support of coordinated maritime patrol operations.

Case study: Dark vessel detection in the Tuvalu EEZ

In December 2021, the Starboard science team began to work with the Tuvalu Fisheries Department (TFD) on a long-term dark vessel detection programme of the nation’s 750,000 km² EEZ. The work was as part of a pilot project under the World Bank-funded Pacific Islands Regional Oceanscape Program (PROP) and included both routine data acquisitions, randomised over time and coverage area, as well as intensified acquisitions in support of patrol ship and aircraft operations. The TFD-Starboard team decided to task both SAR and RF detections to compare the different detection technologies and investigate optical imagery when the opportunity arose.

Dozens of RF scans from Unseenlabs (Figure 6) and 10 Radarsat-2 acquisitions were tasked for this project adding to a total survey area of 25 million km². The team decided in favour of more RF scans as they cover a larger area and cost less per acquisition than SAR. Upon receipt of the data from the satellite provider, the Starboard platform processed the data, matched detections to known vessel locations, and served the result to the web-based user interface (as seen already in Figures 4 and 5). Over the course of the programme, approximately 700 vessel detections were made in satellite data, most of which could be matched against vessels via known geolocations based on AIS and VMS transmissions. These matches indicated that the positional accuracy of satellite detections is excellent, with geocoordinates of RF detections typically within less than one kilometre of the ship location, and SAR detections within 500 metres.

Both types of satellite data can produce false positive and false negative detections. We found that SAR data was very unlikely to miss ships over 30 m length (false negative), but noise in the data may be falsely interpreted as a ship (false positive). RF scans may miss ships that are not operating their radar, or when the rotating emitters are pointing away from the satellite at the time of data capture.

The large number of dark RF detections was unexpected. Radar sources that may not be transmitting AIS or VMS include yachts and pleasure craft, military ships and even aircraft, whose weather radar operates in the frequency range detectable by Unseenlabs. However, we suspect that there were a number of false positive RF detections.

The insights gained from this surveillance campaign prove that the technology is evolving and limitations in the capabilities of the currently available commercial systems are being addressed.

Conclusion

Satellite sensors that can detect vessels at sea cover areas larger than any other surveillance technology in a single scan and can provide approximately daily global coverage. It is now easier than ever to order and quickly obtain satellite data at a fraction of the cost of ship or aircraft reconnaissance. Satellites have already become a regular tool to learn about the patterns of dark vessel activity, and to support fisheries monitoring, control and surveillance operations where the intelligence is used to direct surface and air patrol assets.

Satellite technology is also improving and expanding rapidly. Over the past year or so, more than a dozen new commercial vessel-detection satellites have been launched, and we expect dozens more before the end of 2023. This means that satellite sensors are quickly closing the surveillance gap. But the availability of a variety of sensing technologies, each with inherent advantages and limitations, requires careful planning of satellite campaigns. In our experience, an effective satellite dark vessel monitoring programme requires technical knowledge, close collaboration with regional experts, and flexibility to adapt to changing maritime traffic patterns and patrol asset availability.

At Starboard, we certainly foresee a future where global satellite surveillance provides continuous verification of ship locations at sea that is independent of self-reported positions. Space technology has already made it much harder to hide within the expanse of the open ocean and ubiquitous software platforms such as Starboard will continue to evolve to reveal suspicious activity, provide safety for those making their living at sea, and help to protect marine ecosystems.

About the authors

Dr. Moritz Lehmann is an oceanographer and remote sensing scientist who likes to squeeze every little bit of knowledge from ship tracks and vessel behaviour at sea. He leads Starboard’s biosecurity research and supports operations against illegal, unreported and unregulated fishing using satellite dark vessel detections. Moritz is a well-published scientist and a skilled communicator. His work has been featured in several interviews for radio, newspapers and TV.

Dr. Andrew Middleditch is a research scientist at Starboard with a background in mathematics and oceanography. He has spent much of his career in ocean radar research, developing signal processing algorithms for the measurement of currents and waves. Andrew currently works on Starboard’s dark vessel product, which focuses on the use of synthetic aperture radar (SAR) and radio frequency (RF) satellite technologies to detect non-reporting vessels.

Get a demo

Learn more about Starboard for fisheries analysis and investigations.

Read more

CASE STUDY

In May 2020 Starboard set out to provide an enhanced picture of southern bluefin tuna fisheries in the Tasman Sea. Combining real-time AIS with vessel detections by satellite for surveillance in this area of interest.

Read case study →

CUSTOMER CASE STUDY

Flagging vessels with heightened risks, such as foot and mouth disease, supports the planning and processes for border teams

Read case study →

CUSTOMER CASE STUDY

Sabina Goldaracena is a custodian of the South West Atlantic maritime domain. She connects the clues in her investigations, uncovers hidden activity and works towards transparency by sharing her stories.

Read case study →