Overview: Data Processing
Familiarize with the basics of hydroacoustic data processing in this overview.
Contents
# Terminology
# Applications: the uses of hydroacoustic data
# Procedures: a universal framework for data processing
# Techniques: tailoring the best approaches for your data
# Disclaimer
Related links
> Processing techniques for single-beam data
> Processing techniques for multibeam data
> Processing techniques for imaging-sonar data
> Discussions about these techniques on the Hydroacoustics User Forum
Terminology
The following terms and definitions are used throughout the Echoview website.
Hydroacoustics
This is defined as the study and use of sound in water. In the context of Echoview we are referring to active acoustics; this involves the active generation of underwater sound and the detection of echoes (backscatter). This is separate to passive acoustics, which involves underwater sound detection only and which we do not currently support in Echoview.
Application
This is defined as the reason or purpose for using the data. Hydroacoustics is widely used for the remote observation and assessment of underwater physical and biological phenomena. An example of a common hydroacoustic data application is the estimation of fish biomass for fisheries and ecosystem management purposes. See below for further examples.
Data processing
This is defined as the process of converting data to usable information. Synonymous and/or subset terms include data analysis and data conversion. We also make the distinction between two further components of data processing:
- Procedures - these are the universal steps that apply to the processing of any hydroacoustic dataset. See below for further details.
- Techniques - these are the specific approaches applied during a given data-processing procedure. The best technique to use will vary depending on the nature of the data. See below for further details.
Applications: the uses of hydroacoustic data
Hydroacoustics is a powerful tool for the study of marine and freshwater environments. The applications of this tool are diverse and include:
- Bathymetric (hydrographic) surveying for the creation of navigational charts
- Classification of bottom types for the description of biological habitats in ecosystem studies
- Monitoring of submerged structures for the maintenance of civil engineering projects
- Counting and tracking of fish for fisheries and ecosystem management purposes
- Characterization of fish schools and zooplankton aggregations for ecosystem studies
- Estimation of fish and zooplankton biomass for fisheries and ecosystem management purposes
Procedures: a universal framework for data processing
Despite the diversity of hydroacoustic applications, there is a universal sequence of procedures that applies to the processing of any hydroacoustic dataset.
These procedures and examples of associated techniques are given below. Click on a procedure to link to its description beneath.
- Loading and viewing of data
- Calibration
- Sample time
- Sample location
- Sample energy
- Background-noise removal
- Subtract
- Threshold
- Detection and filtering
- Resample/smooth/bin
- Detect seafloor/create other lines
- Detect single targets and single-target tracks
- Detect schools/create other regions
- Mask/filter
- Classification
- Delta Sv
- School attributes
- Thresholding
- Characterization
- Line attributes
- Cell attributes (echo integration)
- Single-target and track attributes
- School attributes
Calibration
Raw sounder data typically need calibrating so that the measured sample values are 'correct' with respect to time, location and echo energy. This requires information about the transducer location and orientation, the hardware-system properties (e.g. gain), the sound-related properties of the environment (sound speed and absorption coefficient) and any known problems with either the position or time feeds during data logging.
In Echoview this is achieved by entering the appropriate settings into the properties dialogs for the platform, transducer(s) and variable(s).
Note that analyses to estimate calibration parameters such as transducer gain are not part of the calibration procedure by this definition. Estimation of transducer gain, for example, involves the echo integration of reference-target data so would be categorized here as a characterization procedure.
Background-noise removal
This represents the first of the two 'cleaning' stages, whereby background noise is removed from the raw signal.
Simmonds and MacLennan (2005) define noise as the signal that would be measured by an echosounder or sonar with the transmit disabled and the receiver enabled. Based on this we can make two further distinctions; background noise and intermittent noise. Background noise describes noise that is relatively continuous within and over many tens of transmit/receive cycles (pings) or more; intermittent noise describes noise that is typically much stronger and exists over much shorter timescales, ranging from a few samples to a few pings. Intermittent noise is discussed further below.
Background-noise removal involves first estimating the noise level for any given sample and then removing it, either by subtraction (strictly the most correct approach) or by thresholding.
Detection and filtering
This represents the second of the two 'cleaning' stages, whereby the various intermittent-noise (see definition above) and signal phenomena present in the data are detected and filtered as required. Examples of intermittent noise include spikes and flecks. Examples of signal phenomena include single targets, aggregations (schools), the seafloor (primary and secondary bottoms, including 'ghost' bottoms), ping dropouts and surface phenomena (including air bubbles and transducer ringdown).
Prior to detection it may be necessary to resample/smooth/bin the raw data samples; this is typically performed either to mitigate stochastic sample-to-sample variation or to make the noise and/or signal phenomena more amenable to automated detection algorithms.
Detection requires some information about the characteristics of the target phenomena (threshold target strength or size, for example), whether to tune an automated detection algorithm or to facilitate decisions during manual delineation. For single targets, multiple instances of the same target can be identified and monitored over time as a track. Schools, tracks and other detected phenomena essentially describe regions that can be delineated in space and time.
The choice of which detected data to filter out and which to retain will depend on the application. A seafloor-habitat study, for example, may not require water-column data, while the reverse might be true for a fish survey. In most cases it is desirable to remove all detected noise phenomena.
Classification
During classification, the calibrated, cleaned data are partitioned into classes that best reflect their true nature. Examples of classes might be "sand" and "rock" for seafloor data, or "fish" and "zooplankton" for biological data.
Some knowledge of the acoustic characteristics of each class is required in order to search for classes in the data. Moreover, the ability to discriminate different classes requires that their defining characteristics are sufficiently different from each other. Class characteristics can be explored either empirically by ground truthing (direct sampling followed by comparison with the corresponding acoustic data) or theoretically by modelling based on acoustic scattering principles.
Characterization
Data processing is concluded by characterizing the various components of the data, which are now in the form of lines, single targets, single-target tracks (regions), schools (regions) or cells (arbitrarily-sized collections of samples). A line denoting the seafloor, for example, might be characterized by its depth and location, while a school has a range of characteristics related to its location, morphology, echo energy and environment. The core analyses in this regard include echo integration for cells and aggregations, and echo counting for single targets. In Echoview the results of these analyses are typically recorded via an export operation, where the requested calculations are written to a file. In some cases the results can be retained within the software to enable further processing and analysis as required.
Techniques: tailoring the best approaches for your data
Techniques are the specific methods or approaches used for a given data-processing procedure. The best technique to use will vary depending on the nature of the data.
The work and skill in hydroacoustic data processing is in deciding what techniques best suit your particular dataset. A major challenge for the hydroacoustics community is the development and sharing of robust techniques.
The Echoview website: a data-processing resource for the hydroacoustics community
The Echoview website provides a centralized place to discover, develop and share hydroacoustic data-processing techniques. Simply navigate the menu at left to find techniques for single-beam, multibeam and imaging-sonar data.
For each data type you will find a series of subpages, each dedicated to a particular technique. Each subpage (technique) consists of the following:
- Background information
- Links to literature and other resources
- Echoview templates and associated documentation
- A link to the corresponding topic on the Hydroacoustics User Forum
Be sure to check back regularly as new techniques, information and Echoview templates may be added at any time.
Naming convention for techniques and Echoview templates
Techniques are named incrementally according to their parent procedure in the order that they are posted to the Echoview website, e.g. "Background noise (1)" etc.
Echoview templates are uniquely labelled by:
- Data type
- Procedure
- The incremental technique number for each procedure
- The incremental template number for each technique
- The Echoview version it was saved in
e.g. "SingleBeam-BackgroundNoise1-01-v4.70.47.EV".
Share your work: a 'clearing house' for Echoview templates
We encourage users to share examples of their work, both for the benefit of others and to facilitate the development of standardized procedures. The Echoview website might be considered as a 'clearing house', where processing techniques are presented and Echoview implementations developed and discussed.
If you wish to share information, literature and/or Echoview templates describing your data processing and analysis algorithms, please contact us and we will make the appropriate arrangements with you to post your materials. This will be particularly useful if you have published your research and wish to enable others to repeat your methods.
Note that a technique can still be usefully posted even if an Echoview template hasn't been developed for it yet.
Disclaimer
Please note that techniques and Echoview templates are provided here "as is". While we try to ensure that these materials are free from mistakes, it is the responsibility of the user to verify the source, accuracy and appropriateness of the technique for their needs. You can read and discuss more about each technique is via its related topic on the Hydroacoustics User Forum.
