Building trust in Earth Observation products and services: the importance of calibration and validation procedures. These processes are key to the success of some new research in agritech. Co-creation and consultation are key pillars of the COALA project. In this project, the stakeholders’ wants and needs guide the development process.

Remote sensing

The COALA partners are developing products and services in a framework of scientific communication. COALA team members are building trust in Earth Observation products through validation and consultation. The University of Melbourne (UOM), with project partners, will conduct both airborne and field campaigns to validate their services. Importantly, the airborne campaign will be equipped with multispectral and thermal cameras.  Then, these experiments will help adapt and improve the core COALA algorithms. As a result of this calibration and validation process, COALA can provide detailed information on sub-paddock crop growth, nutrient and water stress conditions and spatially detailed crop water consumption.

In addition, high-resolution airborne remote sensing provides an accuracy assessment for products related to the water content of the soil-plant system derived from Sentinel-2 imagery.

The airborne system

The University of Melbourne’s Airborne Remote Sensing Facility, led by Prof. Pablo J. Zarco-Tejada, of the Hyperspectral Remote Sensing & Precision Agriculture Laboratory (HyperSensLab), has a payload of hyperspectral, thermal infrared and fluorescence sensors.

The UoM will deploy their airborne remote sensing system over a selection of pilot sites in the Murray Darling Basin. These sites are located where the high-resolution thermal infrared and multi-spectral imagery are most useful for validating and calibrating the suite of COALA products and services.

The airborne system of UoM has a payload of hyperspectral, thermal infrared and fluorescence sensors. Hyperspectral sensors cover the visible and near-infrared light range. We will undertake three campaigns (called phases). During Phases 1 and 2, we will conduct airborne campaigns three times within key growth stages of the irrigated farms. We have scheduled the three/phase airborne campaigns in December, January and February. These months are best to capture critical emergence, flowering and maturing, and key phenological stages of irrigated crops in these areas.

Calibration phases

The calibration phases will be integrated through in-situ soil/plant data that will be collected in field surveys. UoM will collect this data using a stratified sampling approach to minimise costs and maximise statistical representativeness.

In situ (A) thermal infrared (optional multi-spectral) and (B) meteorological monitoring stations operational in a selection of pilot farm fields in Goulburn-Murray and Coleambally region.

In the dryland cropping farms, equipment deployed in selected paddocks will provide multi-target data. These data include in-situ soil moisture data, ground-based soil and weather information. Each component is essential data to produce services from Sentinel-2 imagery that examine crop yield prospect and nitrogen deficiency.

The link between ground truthing and ‘validation’

Ground-truthing, or collecting georeferenced field data, is vital. Specifically, ground-truthing ensures the accuracy of the classification, interpretation, and prediction of physical and biological data from satellite information. Afterwards, we will utilise the reference data to calibrate sensors and aid in the analysis of satellite data. Later, these data act to verify information extracted from satellite imagery and assess the accuracy of associated products (Lillesand, Kiefer and Chipman, 2015; Pettorelli, 2019). The collection of reference data is expensive, and time and labour intensive. However, agricultural professionals increasingly turn to satellite data as a management tool. In turn, ensuring that this type of information is becoming more and more necessary (Pettorelli, 2019).

However, a major challenge in applying remote sensing products to agriculture is designing ways to validate tools and qualify data. This step is essential, so end-users understand the product they are using and its reliability for particular applications (Weiss, Jacob and Duveiller, 2020). While literature detailing this [what you see is not what you get satellite data phenomenon] is scarce, a lack of trust in data has been previously raised as in issue by Australian producers, service providers, and agriculture industry representatives (Barry et al., 2017).

Further, Schaeffer et al, (2013) detail perspectives around validating satellite data. Importantly, they emphasise the belief that traditional in situ measurements are more representative of the ‘truth’. 

Building trust in Earth Observation products through calibration and validation

A recent survey that COALA conducted to identify users needs shows the importance of trusting the images.   Farmers and extension officers were critical of many products based on satellite images or airbornes that are in the market. “it is hard to trust images from satellites when what it is observed in the field does not seem to match satellite data”, they said.

One interviewee summarised the feelings (and frustrations) of farmers around ‘understanding what they see’, and on timely delivery of services for decision making.

“.. the only way to go and check the image is to go and see what’s happening in the paddock, and then if you’re not seeing what’s happening in the paddock as the image, you just don’t trust it. So then you lose faith with the images, so for me to base a nitrogen decision off the image, I need to really trust that image, so they’re the types of things that I see too often is like you go out there, and the image is not matching up with what you’re seeing, and if it’s not doing that and that’s not right, and you can’t determine why it is or isn’t right, or you’re not going to let that make your decision for you, …”

“I had a trial paddock last year, and I was really struggling to find a good image of the trial that I was seeing [in] the paddock until I think it was September when I started to see it, and that’s just way too late for me to be making any decisions on it, so like I need to have that information in June, July, and that’s what I started to see those differences, and that’s probably one thing that I was, yeah, getting frustrated about, because I was seeing these things out in the paddock, but then I wasn’t seeing them when I had the information in front of me.”

COALA

In COALA, validation will be conducted to confirm from users that COALA services are exactly what they want. We base this project on a validation protocol and criteria. This step gives conclusions on effectiveness and cost-benefit, as well as further impact. It demonstrates if/that COALA services “make a difference” for the users and their environment. Building trust in Earth Observation products and services will be essential to the success of this project.

Dongreyol Rye, Graciela Metternicht, Rebecca Cross, Bonnie Teece