Estimating Crowdsourced, Authoritative Observer, and Data Reputation

TitleEstimating Crowdsourced, Authoritative Observer, and Data Reputation
Publication TypeConference Proceedings
Year2020
AuthorsCalder, BR, Hoy, S
Conference NameCanadian Hydrographic Conference
Conference DatesFebruary 25-27
PublisherCanadian Hydrographic Association
Conference LocationQuebec City, Quebec, Canada

Non-authoritative use of marine volunteered geographical information (MVGI), also known as Crowdsourced Bathymetry (CSB), is commonplace in the fishing and recreational markets through companies such as Olex, Navionics, CMap, and others. Use of MVGI for authoritative purposes (i.e., for updates of official charting data) has been significantly less widespread. Although many Hydrographic Offices have conducted limited experiments, and despite the IHO’s enthusiastic endorsement, the liability, data availability, and processing workload associated with principles use of MVGI has led to slow uptake for routine use. The most commonly cited problem is lack of trust in the observations, particularly their vertical calibration. This paper therefore proposes to measure the reputation, or level of trust, that should be placed in observers across the spectrum from MVGI to Hydrographic Office surveyors, with the goal of dynamically adjusting the reputation of each observer over time as they make new observations that can be compared against other observers, or authoritative data. This model also extends to the data generated by the observers, which inherits the observer’s reputation, but can then be confirmed or degraded over time with other observations, leading to application to chart adequacy and resurvey priority. Using a widely applied model for competitive ranking (predominantly used in chess ratings), two ranking stages are outlined: one that ranks observers against reference to determine their individual reputation, and a second that monitors the reference for drift and time-decay. The model allows for uncertainty in the rankings, and the decay of certainty due to time between comparisons. This model requires estimates of observer self-noise and bias, which are determined from time-series analysis of the observations. Using MVGI from the IHO Data Center for Digital Bathymetry, the paper demonstrates observer uncertainty calibration, ranking of observers, and the effects of time, and new observations, on archival authoritative data.