Ask what would count as a fair test
Validation compares a map or model output with reference evidence suited to the claim. It is not simply a walk to several places where the map appears correct. A fair test needs a defined population, a sampling strategy and reference observations that are sufficiently independent of model development. If the same examples guide training, parameter tuning and final evaluation, the resulting score cannot honestly describe performance on unseen evidence.
Begin by stating the intended use. A map used to prioritise regional field visits may tolerate errors that would be unacceptable in a parcel-level legal decision. A habitat assessment may require different class definitions from an engineering survey. Define the relevant spatial unit and time period before choosing reference sites. Ground observations are powerful because they offer context, but they are not automatically perfect or directly comparable with the support of a satellite pixel.
Sample beyond convenient access
Roadside observations are easier to collect than observations in remote marshes, steep woodland or inaccessible private land. That convenience can produce a biased reference set. Probability-based sampling offers a basis for estimating accuracy across a defined area. Stratification can ensure that less common classes or candidate change areas receive attention. Record the sampling design, inclusion rules and any weights required for estimating area-level results. Sample size should follow the precision needed and the variability expected, not a universal rule of thumb.
Where access is impossible, document the missing reference information and consider appropriate alternatives, such as carefully interpreted high-resolution imagery or existing survey records. These alternatives introduce their own dates, resolutions and interpretation limits. Do not replace difficult sites only with nearby easy ones without considering the consequences. An inaccessible wetland margin may differ systematically from the accessible areas, so excluding it can remove exactly the conditions where the model is most uncertain.
Record the observation footprint
A location coordinate alone does not describe what was observed. Record positioning method, estimated positional uncertainty, observation time, surface conditions, photograph direction and the area represented by the reference label. A handheld Global Navigation Satellite System receiver can be affected by canopy, buildings and terrain. If position uncertainty is large relative to the pixel or feature, a disagreement may reflect misalignment rather than a classification error.
At mixed boundaries, describe the local composition rather than pretending that every site has one unambiguous class. A point on a hedge beside a field may not represent the surrounding satellite footprint. In places such as Dartmoor or the New Forest, vegetation mosaics and management boundaries can complicate simplified labels. Written field protocols help observers apply definitions consistently. A second interpretation of selected sites can reveal disagreement that should remain part of the uncertainty record.
Keep time and independence visible
A field visit after an event may not reconstruct the surface condition on the image date. Harvesting, grazing, flooding and vegetation recovery can all intervene. Record the time gap and explain why the reference remains suitable, or mark the comparison uncertain. For historical validation, archived aerial photographs and documented surveys may be more appropriate than present-day observation. More recent evidence is not necessarily better evidence for an older scene.
Reference interpretation should also avoid unnecessary influence from the prediction being tested. An observer who sees the model’s label first may unconsciously search for confirming details. Where practical, use independent or blinded interpretation and preserve ambiguous cases for review. The Committee on Earth Observation Satellites, through its Land Product Validation activities, provides a useful methodological context for thinking about validation. Product-level guidance still needs adaptation to the specific map, geography and decision.
Report errors as geographical information
A confusion matrix shows how reference and mapped categories agree or differ. Precision helps describe the reliability of a mapped class; recall helps describe how much of a reference class was detected. Neither tells the whole story alone. Include the reference definitions, sampling design and uncertainty around estimates. For mapped area totals, use an estimation approach consistent with the sampling design rather than assuming the raw count of classified pixels is unbiased.
Inspect where errors occur. Concentration near boundaries, under cloud shadows or within a particular land-management system can point to a useful improvement. Share known limitations alongside successes, and protect sensitive field locations where disclosure could harm species, communities or landholders. Fieldwork also requires permission, appropriate safety planning and respect for local knowledge. The goal is not to make the model look reliable. It is to establish which conclusions the evidence supports, which require revision and which should remain undecided.