Metadata Evaluation and Guidance

Purpose

To identify and compare existing metadata evaluation tools and mechanisms for connecting the results of those evaluations to clear, cross-community guidance.

Project plan

The following project plan is estimated to encompass work between May and October 2018.

• Populate Schemas, Tools and Guidance Information
• Review existing evaluation tools and identify what they are evaluating (completeness, consistency, linking, quality, other).
• Identify advantages/disadvantages of current evaluation tools and make recommendations for use
• Together with Project 5, create a guidance document for evaluation tool use for multiple communities and/or create a white paper or statement to the wider scholarly communications community
• Consider where there might be gaps in evaluation resources and where relevant, scope out the needs for ideal additional tools
• Complete Project 5’s best practices resources survey

Challenges

• Currently it is difficult to evaluate the completeness, consistency and accuracy of metadata deposited across multiple systems and, thus, to demonstrate evolution and improvements through time
• Researchers in particular do not have a good way of assessing the completeness of their metadata, and publishers have to take time to assess the metadata, leading to increased costs and delays
• Publishers do not have a clear idea of the compliance of metadata to multiple standards, and checking that compliance is often manual and time consuming

Possible solutions to explore

• Creation of a spirals-like metadata evaluation system for integration with submission systems, assessing the quality and completeness of metadata against different metadata standards
• Identify simple quantitative metrics that can be used to measure and monitor completeness, consistency and accuracy of metadata
• The Crossref Member metadata evaluation tool could be backed by Metadata 2020, with use cases of how publishers used the information to improve their own metadata. A similar approach could be looked at for other communities e.g. DataCite member metadata
• The creation of a catalog of metadata quality tools and further resources could be very useful to the community

Group participants

=======