Conference Agenda

Smart-farming applications have seen a steep rise in the incorporation of EO data, mainly due to the availability of Sentinel 1 and 2, and to some extent the increased availability of affordable high-resolution satellite data. This increased uptake also results in ever increasing demands by the users in the level of detail as well as thematic accuracy of the products. While some of these demands can be met with high-resolution satellites, other demands and use cases remain too challenging. To fill this monitoring gap, Unmanned aerial vehicles (UAVs) or drones have become a popular and useful tool to identify and map (a)biotic stress at (sub)cm resolution. However, an important limitation remains the poor scalability due to the high costs and the need for manual, specialized, interventions. Here, a solution is proposed to circumvent these limitations, bringing UAV-based crop monitoring a step closer as a complementary companion to the satellite-based monitoring tools. As a first step, methodologies are presented with small, cheap drone systems that do not require elaborate training or licenses to operate. Secondly, a drone sampling methodology is proposed to optimize the cost-efficiency of data collection and processing, based on a direct georeferencing workflow to project the individual images on a regional Digital Surface Model. This allows for low or even negative image overlap which will be at least 70% more efficient compared to a traditional Structure-From-Motion workflow. Image analysis is performed on the individual images to identify plants or plant features like weeds, disease spots or flowers. To scale-up the adaptation of the drone sampling technology, the workflow has been integrated into MAPEO, VITO’s drone processing platform, culminating in the generation of consistently accurate and reliable data products. The overall workflow will be demonstrated with several use cases like agricultural insurances and invasive weed detection.

Spatially explicit information on land cover (LC) is commonly derived using remote sensing. The lack of training data remains a major challenge to produce accurate LC products. Although in-situ surveys are generally regarded as reliable information sources, there tend to be inconsistencies between in-situ LC data and the information derived from satellites due to the different view points.

Here, we develop a computer vision methodology to extract LC information from photos from the Land Use-Land Cover Area Frame Survey (LUCAS). A representative sample of 1120 photos covering eight major LC types across the European Union was selected. We then applied semantic segmentation to these photos using a neural network (Deeplabv3+) trained with the ADE20k dataset. For each photo, we extracted the original LC identified by the LUCAS surveyor, the segmented objects, and the pixel count for each ADE20k class. Using the latter as input features, we then trained a Random Forest model to classify the LC of the photo. Examining the relationship between the objects/features extracted by Deeplabv3+ and the LC labels provided by the LUCAS surveyors demonstrated how the LC classes can be decomposed into multiple objects, highlighting the complexity of LC classification from photographs. The results of the classification show a mean F1 Score of 89%. In a stage, we analyzed the semantic gap between this subset of LUCAS 2018 in-situ LC data and three high-resolution thematic LC products derived from satellite data, namely, Google’s Dynamic World, ESA’s World Cover, and Esri’s LC maps. Following an experimental analysis, we explore the feasibility of using geo-referenced street-level imagery to bridge the gap between information provided by field surveys and satellite data.

The spatial extent of croplands globally is a crucial input to global and regional agricultural monitoring systems. Although many new remotely sensed products are now appearing due to recent advances in the spatial and temporal resolution of satellite sensors, there are still issues with these products that are related to the definition of cropland used and the accuracies of these maps, particularly when examined spatially. To address the needs of the agricultural monitoring community, here we have created a hybrid map of global cropland extent at a 500 m resolution by fusing two of the latest high resolution remotely sensed cropland products: the European Space Agency’s WorldCereal and the cropland layer from the University of Maryland (GLAD croplands). Since the fallow class is included in the GLAD croplands map and we consider this relevant for food security applications we use this layer as in inital base map. We then aggregated the two products to a common resolution of 500 m to produce percentage cropland and compared them spatially, calculating two kinds of disagreement: density disagreement, where the two maps differ by more than 80%, and absence-presence of cropland disagreement, where one map indicates the presence of cropland while the other does not. Based on these disagreements, we selected continuous areas of disagreement, referred to in the paper as hotspots of disagreement, for manual correction by experts using the Geo-Wiki land cover application. The hybrid map was then validated using a stratified random sample based on the disagreement layer, where the sample was visually interpreted by a different set of experts using Geo-Wiki. The results show that the hybrid product improves upon the overall accuracy statistics of the individual layers, but more importantly, it represents a better spatially explicit cropland layer for early warning and food security assessment purposes.A version of the map has been used as part of JRC's early warning systems ASAP.

Ground truth reference data availability has long been a significant limiting factor for remote sensing data-driven model training. A vastly underutilized source of data comes from administrative data collected by mainly public service bodies. A demonstrated successful utilization is the EuroCrops project where we created training labels for crop type classification from farmers’ self-declarations on the crop types they cultivate. However, a key challenge of the project was the lack of domain-specific expert knowledge and incompatibilities between data released by different governments.

Recently, large language models (LLMs) have demonstrated remarkable abilities in information extraction and synthesis. We therefore propose training an LLM-based domain-specific foundation model to automate the data extraction and harmonization process from administrative data to ready-to-use training data.

This presentation will discuss a toy example of training a small language-based transformer model using the administrative data collected during EuroCrops with agriculture-based text data such as Wikipedia articles.

Preliminary experimentation with open-source not field-specific LLMs shows that they do offer some interesting domain-specific insights when decoding acronyms. However, they do not outperform traditional machine translation methods yet, such as Google translate, and at times output extremely believable but misleading results in the niche field of agriculture. Such bogus generations highlight the need for a grounded generation mechanism in the usage of LLMs.

.