Button Text Button Text Button Text Button Text Button Text
Proba-1 view of Guam, ESA, CC BY-SA 3.0 IGO

Technical Background

The QA4EO guidelines provide a methodology to ensure reliable interpretations of environmental observations from satellites and in-situ measurements by requiring that associated uncertainty information is provided.

Earth observations are naturally multivariate: that is, any observational instrument will take several observations in different dimensions — for example, providing a time series of observations at a single location, or a spatial distribution of observations in an image. Scientists often combine different individual observations to create spatially and temporally gridded products, or to fit trends and interpolate between spatial observations. Other dimensions can be important too: optical sensors for example, make measurements in different spectral bands, which are combined in processing the raw data into observational products. For these reasons, any robust uncertainty analysis must include estimates of the error covariance in the data. (See introductory documents for more information).

The approaches defined within QA4EO enable the Earth observation (EO) community to develop quantitative characterisations of uncertainty in EO data. However, practically implementing these methods, especially in a computationally efficient manner, is not trivial and can be time-consuming. To facilitate this, the CoMet Toolkit provides a means to store and propagate uncertainty and error-correlation information. These tools allow the user to rely on quality-assured code, rather than having to 'reinvent the wheel' and, thus, lower barriers to entry for users new to handling uncertainties.

Effects Tables (see process guide) are a useful way to record and report the information required to fully parameterise an error-covariance effect. However, to use this information in a processing chain, it must be provided digitally. The CoMet Toolkit defines a mechanism for this, with a metadata standard that enables the creation of Digital Effects Tables in NetCDF files. In this way, uncertainty information can be written, read, and processed in a way that is machine-readable and preserved.

South Georgia, contains modified Copernicus Sentinel data (2018), processed by ESA LICENCE CC BY-SA 3.0 IGO

CoMet Toolkit has three core modules:

  • comet_maths mathematical algorithms, linear algebra calculations, and random (correlated) sample generation, for use throughout the CoMet Toolkit.
  • Obsarray extension to xarray for handling uncertainty quantified observation data, and storing uncertainty and covariance information in NetCDF files.
  • Punpy tool for 'Propagation of UNcertainties in Python'. Propagates uncertainties on input quantities through any measurement function defined in Python to uncertainties on the measurand, taking into account error-correlation information.

Punpy interfaces with obsarray to make uncertainty propagation as efficient and easy to use as possible. The digital effects tables produced with obsarray can be propagated through measurement functions using punpy, without the need for providing additional information. The data has thus been encoded with all relevant error-covariance information, though users don’t need to interact with it.

Together these tools enable both experienced and inexperienced users to efficiently include uncertainties throughout data processing, for reliability and ease of interpretation. Optional keywords provide users with the flexibility to deal with all kinds of complex use cases.

For further info, refer to the CoMet Toolkit website, as well as the punpy and obsarray documentation.