This demo showcases the use of images from the Danish Mycological Society's SvampeAtlas and models how the GBIF community can serve as a mechanism for improvements in suggesting species identification and mediating machine vision training datasets to AI modellers.
Machine vision technology can already provide suggestions for identifying images for tens of thousands of species across a wide range of taxonomic groups—witness iNaturalist, which suggests species IDs to users in real-time as they create observation records. Rapid and ongoing advances in training of artificial intelligence (AI) will soon make the detection of species in video feeds or the use the camera in a mobile device to search for species-related content on the Internet commonplace.
The GBIF network has an important role to play in advancing and improving AI with respect to biodiversity data, cross-disciplinary collaboration and citation practice—not least because the GBIF infrastructure holds tens of millions of records associated with one or more images of labelled species, one of the largest datasets of its kind in the world.
As a community, GBIF has implemented not only key technical practices around data standards, but also important social and cultural improvements, including the adoption of open licences, guidance on data citation, and the development of a DOI-based system for tracking reuse of data. Currently applying these lessons alongside an expert team, GBIF is assisting research to increase machine vision's power and availability while seeking to improve understanding and accepted practice regarding the use of GBIF-mediated data in training for AI.
Training datasets are critical to achieving species recognition capability in any machine vision system. These datasets compile representative images containing explicit, verifiable identifications of the species they include. High-powered computers run algorithms to analyse the imagery, building complex models that characterize the defining features for each species or taxonomic group. Researchers can then apply the models to new images, offering predictions as to which species or group they likely contain.
Current research in machine vision is exploring:
- the use of location and date information to further improve model results
- identification methods beyond species-level into attribute, character, trait, or part-level ID, with an eye toward human interpretability
- expertise modeling for improved determination of “research grade” images and metadata
We propose the following approach:
-
To assist in developing and refining machine vision models, GBIF will provide AI training datasets and ensure that licensing and citation practices are respected. Each of these datasets receive a DOI, and all of contributing datasets will receive credit and attribution of uses through GBIF's DOI citation system.
-
To assist application developers, Google and Visipedia will build openly-licensed models and publish tutorials on how to adapt them for local use.
-
Together, the project partners will strive to ensure responsible and transparent use of AI training datasets to close the gap between machine vision scientists, application developers. We will also prepare taxonomic trees that clarify and clarify the confidence with which machine vision models can identify a taxon rank based on an image’s visual characteristics.