The Crossref2024 annual meeting gathered our community for a packed agenda of updates, demos, and lively discussions on advancing our shared goals. The day was filled with insights and energy, from practical demos of Crossref’s latest API features to community reflections on the Research Nexus initiative and the Board elections.
Our Board elections are always the focal point of the Annual Meeting. We want to start reflecting on the day by congratulating our newly elected board members: Katharina Rieck from Austrian Science Fund (FWF), Lisa Schiff from California Digital Library, Aaron Wood from American Psychological Association, and Amanda Ward from Taylor and Francis, who will officially join (and re-join) in January 2025.
Background The Principles of Open Scholarly Infrastructure (POSI) provides a set of guidelines for operating open infrastructure in service to the scholarly community. It sets out 16 points to ensure that the infrastructure on which the scholarly and research communities rely is openly governed, sustainable, and replicable. Each POSI adopter regularly reviews progress, conducts periodic audits, and self-reports how they’re working towards each of the principles.
In 2020, Crossref’s board voted to adopt the Principles of Open Scholarly Infrastructure, and we completed our first self-audit.
In June 2022, we wrote a blog post “Rethinking staff travel, meetings, and events” outlining our new approach to staff travel, meetings, and events with the goal of not going back to ‘normal’ after the pandemic. We took into account three key areas:
The environment and climate change Inclusion Work/life balance We are aware that many of our members are also interested in minimizing their impacts on the environment, and we are overdue for an update on meeting our own commitments, so here goes our summary for the year 2023!
Metadata is one of the most important tools needed to communicate with each other about science and scholarship. It tells the story of research that travels throughout systems and subjects and even to future generations. We have metadata for organising and describing content, metadata for provenance and ownership information, and metadata is increasingly used as signals of trust.
Following our panel discussion on the same subject at the ALPSP University Press Redux conference in May 2024, in this post we explore the idea that metadata, once considered important mostly for discoverability, is now a vital element used for evidence and the integrity of the scholarly record.
For the third year in a row, Crossref hosted a roundtable on research integrity prior to the Frankfurt book fair. This year the event looked at Crossmark, our tool to display retractions and other post-publication updates to readers.
Since the start of 2024, we have been carrying out a consultation on Crossmark, gathering feedback and input from a range of members. The roundtable discussion was a chance to check and refine some of the conclusions we’ve come to, and gather more suggestions on the way forward. As in previous years, we were able to include a range of organisations, which led to lively and interesting discussions. See below for the full participant list.
Crossmark feedback
We started by presenting Crossmark and a summary of the consultation process. There are a number of areas where we have learned more about how the community operates or found that Crossmark needs to adapt. These include:
Implementation: Our members have struggled to implement Crossmark and uptake is low. At the same time, in many organisations the workflows for handling retractions are not well-defined because they are rarely used, if ever. The responsibility for updating Crossref metadata can be unclear and this may be a factor in the low uptake.
Education: There are different levels of understanding about how to handle retractions. Some members are very defensive when asked about retractions, others state they will never make updates to published works. How can we have a constructive conversation where the value of communicating updates appropriately is recognised?
Community engagement: Given the different scales, locations, disciplines, and technologies used by our members, it looks like one size will not fit all when it comes to updates. How can we get continual, representative feedback on new tools and processes?
Metadata assertions: Crossmark allows the deposit of metadata using custom field names, however this metadata seems to have low usefulness and is not highly valued by the community. Should we continue to collect it? Can we make some of the most-used field names part of our standard schema?
Changing the Crossmark UI: Although we didn’t specifically ask about it during the consultation, the look of the Crossref logo often came up, and concern that it is not recognised and not well-used. Can we change the look and behaviour so that it has more impact?
NISO Recommendations
Patrick Hargitt represented the NISO group on Communication of Retractions, Removals, and Expressions of Concern (CREC). The group’s recommendations were published earlier this year and cover how retractions are communicated. CREC arose from an earlier project, IRSRS. A large part of the motivation is that retracted works continued to be cited, with citing authors apparently unaware of the retraction.
Patrick presented the CREC recommendations, which cover:
Metadata receipt, display, and distribution,
Which metadata elements to communicate,
How to implement the recommendations,
Discussion of some special cases,
Key stakeholders and their responsibilities.
The two presentations prompted discussion, which was taken into the first of two workshops.
First workshop: Improving collection of retractions and Crossmark
The first workshop looked at proposed changes to Crossmark and how to encourage more members to deposit their retractions, corrections, and other post-publication updates. Several important themes emerged.
First, the question of whose responsibility it should be to provide metadata on retractions and similar updates. Crossref has a responsibility to work with the community to obtain high quality and complete metadata; publishers should take responsibility for handling issues of research integrity and reporting them to relevant downstream services, like Crossref; and platforms need to provide tools that allow easy reporting of retractions.
The value of Crossmark appearing in PDFs was reiterated. The fact that a PDF can be downloaded, and years later there is a way to tell whether it has been retracted or not is highly valued. There was also the suggestion that the Crossmark logo on web pages can indicate a change before it has been clicked. This is something that we have been considering at Crossref and it was useful to have the idea reinforced. Another suggestion was that a browser plugin would make a good complement to Crossmark.
Implementation issues with Crossmark were raised, including that it’s difficult to validate whether a specific implementation is complete. There are a number of different changes (to metadata deposit and content, and websites) that need to work together to have Crossmark fully functional. There were several questions and a discussion about Retraction Watch data. Some were about understanding its collection and validation. A number of participants are actively using the data and it was great to see the variety of applications.
Second workshop: Community use of retraction metadata
The second workshop focused on a broader set of downstream organisations that might want to make use of retraction metadata. We looked at stakeholders and their needs, and attempted to match them up with existing tools. Several gaps were identified as a result, which may provide opportunities for new services or collaborations to fill them.
We identified a number of tools available for publishers, editorial systems, metadata researchers, and readers. A good example is reference managers, many of which are now highlighting retracted works to authors. This can help to reduce the number of retracted works being cited. Publishing platforms are also providing support to editors, using tools that include retraction metadata.
Some of the stakeholders identified have limited tools for identifying retractions that are relevant to them. These include funders, archives and repositories, journalists, and institutions.
Often, there are pathways for retraction data to be communicated but they are not being sufficiently used. There needs to be a concerted effort to improve the quality of retraction metadata for tools to function better. For example, a second author on a paper might not know that a correction or retraction is planned for their article. If their email or ORCID isn’t included in the metadata, an alerting tool wouldn’t be able to let them know. A similar argument can be made for institutions or funders if they are not well-identified in the metadata.
The question of standardisation of metadata was raised. It seems too early to implement a full set of standards at the moment. CREC and similar initiatives have documented and accommodated for a range of practices while providing guidance and principles to work towards. More discussion is needed in the community to work out paths that could be applied across the broad spectrum of scholarly communication.
Conclusion
The event was very valuable in bringing up a range of topics related to retraction and communication of post-publication changes to scholarly works. We are grateful to all of the participants for their contributions and sharing their diverse experience and opinions with us.
Research integrity is an area of flux, with significant changes over the past few years. While there has been progress, there remain gaps in metadata and tools to communicate retractions. This is something that Crossref will continue to contribute to, and Crossmark clearly still has a role to play.
Some of the ideas and suggestions from the discussion can be implemented in the near future. Others need further development, and we will continue to engage the community. Reading this, there may be topics where you feel you have a role to play. We are keen to partner with other organisations in this space as we continue to improve the transparency and communication of metadata for post-publication updates.
Participants
Many thanks to the participants. Here is the full list of those that attended:
