Indexing frequency and publication
Q: We have registered a new dataset and received confirmation that the first indexing run succeeded. When could we expect to see our data moved to the main GBIF site?
A: The GBIF data portal is based on the GBIF index database, which stores basic dataset metadata plus core data items for search support and result presentation. Because of performance requirements on one hand and requirements for processing of indexed data on the other, two copies of this database exist. One serves the public data portal, the other one runs indexing operations, both for updates of existing datasets and for first-time indexing of newly registered ones. This indexing database is periodically swapped with the public one in order to publish the data updates. However, rather extensive post-processing is required before this swap can be done (integration of taxonomic hierarchies, map generation and others). Because of these post-processing requirements, database swaps are currently only possible about every four to six weeks. Depending on the timing of first-time indexing of a new dataset, your data should be fully integrated into the public web portal within this time frame. In the meantime, a hidden page does exist that allows both a preview of your indexed – but not yet fully integrated –records, and viewing of the log entries generated during indexing. If you have not yet received notification of this URL, please kindly contact the Help Desk.
Q: How often is data from data publishers re-indexed by GBIF?
A: At the present stage, GBIF attempts to update each registered dataset at least once every three months. As the indexing database is non-public, and changes only get visible when a 'rollover' from the indexing database to the public web portal occurs, we are aiming for monthly rollovers. With the current indexing procedure, three-monthly updates on individual datasets are about the best we can manage, at least for the larger ones and following the general, scheduled procedures. Work to streamline the indexing process is under way, so that we should be able to arrive at more efficient and frequent indexing cycles over the coming year. All this said, it is always possible to insert an explicit indexing run for a dataset, especially if it is not a very large one, on request. So if you are aware of any significant changes or additions regarding your published dataset(s), please drop a quick note to portal_@If you can read this, please upgrade to a modern browser.gbif.org.
