Investing $4.9 million of our surplus in rebuilding the Crossref system

Helena Cousijn — Wed, 12 Aug 2026 00:00:00 +0000

The infrastructure that registers and distributes Crossref metadata has been running and growing for more than two decades. It now does considerably more than it was built to do, and it shows: two large, tightly coupled monoliths (Content System and REST API), where a change in one place risks breaking something in another.

In its July meeting, the Crossref board approved the use of Crossref surplus funds for a three-year project to accelerate the redesign of the Crossref system. Our plan is to use $1.6 million a year for three years (~USD $4.9M in total) from our surplus funds, which will help us set up a modern infrastructure consisting of separate services for the different functionalities that the Crossref system offers.

This infusion of time-limited resources will augment our existing team. We will continue to maintain the current system, add small improvements, fix bugs, and make schema changes. We’ve mapped out our time and capacity to ensure that we can keep the current system active and robust while building the new system at the same time. In the past, when we’ve attempted large redesign initiatives, we’ve done a code freeze on the existing system—we won’t be doing that this time. 25,000+ members and thousands of downstream services rely on the system daily, so we will continue to provide time and energy to support that while building for the future. As always, you can view our public roadmap to see progress with what we’re working on.

Background

Over the years, the board, leadership team, and staff have discussed the need to address long-term technical debt in the Crossref system and the need to move from a monolithic codebase to a more modern, manageable, resilient technology that recreates the Crossref system. We are aiming to do this through a set of connected services and functions that better meet our mission and enable the Research Nexus, our goal of a rich and reusable open network of relationships connecting research organisations, people, things, and actions.

Our colleagues Paul Davis and Sara Bowman described some of the current challenges and plans earlier this year in Hit refresh: redesigning our technical infrastructure. We migrated from a closed-source database to PostgreSQL in 2024, moved out of a physical data centre into the cloud in August 2025, and have steadily been deprecating older tools not fit for purpose, such as Event Data, Co-access and ‘old’ Metadata Manager while rebuilding replacements. That was some of the groundwork; now we’re rebuilding the centre.

The two largest components of the current architecture are the Content System (CS) and the REST API. Both function as monoliths. They use mono-repositories, and their internal components are tightly coupled. Over time, they have accumulated complexity, substantially expanding beyond their original purpose. Each of them is now responsible for many different functions, with different needs and characteristics. This monolithic and tightly coupled architecture makes it difficult to maintain, scale, or expand our system. Adding new functionality often requires workarounds, which further increases complexity and adds technical debt.

As a not-for-profit, any surplus we generate has three possible destinations: reserves, paying down debt, or reinvestment in the mission. We met our goal of holding twelve months of operating costs in reserve in 2023, and we carry no debt, so reinvestment is where the rest belongs. We aim to generate surpluses to ensure that we are operating within our means, but we don’t need to accumulate them.

Some of that reinvestment goes outside Crossref; this year we’ve supported other community initiatives and infrastructure that people rely on, totalling around $500,000. A large proportion of the rest should go where members will feel it.

Reinvestment also has to be the right shape. Under the Principles of Open Scholarly Infrastructure, time-limited funds are used only for time-limited activities: day-to-day operations are covered by day-to-day revenue, and one-off funds go to one-off projects. A three-year rebuild, staffed by fixed-term roles embedded in existing teams, meets that test.

What will we rebuild?

At the end of 2024, we introduced a new structure with Crossref work happening within three cross-functional programs: Co-creation and Community Trends (CCT), Contributing to the Research Nexus (CRN), and Open and Sustainable Operations (OSO). The redesign of the Crossref system will also happen within these three programs. The approach we’ve taken is that we’ve identified the different functionalities of the current system, and divided these between the three programs. In the tables below, you can see which services will be rebuilt within each program.

Co-creation and Community Trends (CCT)

Name of component	What is it?	What does it do?	What will change with the rebuild?
Internal APIs for UIs	Purpose-built backends to serve various user interfaces. This is an architectural approach that we have recently begun using to build the new Metadata Manager record registration tool.	Take data from sources such as the Postgres database or the data lake and provide it in a form that user interfaces can use.	This approach will allow us to decouple existing interfaces from the legacy system codebase and build new ones as part of the other projects listed below.
Member self-service	A new interface for members to perform routine tasks related to their member account and user management.	Currently, these tasks are performed in highly manual processes by the Membership team. In some cases, there are semi-manual processes in place, such as with title transfers, where members can fill out a request form on our website.	A self-service portal powered by the new authentication and authorisation service (mentioned below) will lighten the workload of the Membership team. For example, we estimate that about 15% of member support tickets in 2025 represented contact updates which had to be handled manually.
New Admin tool	A user interface for members to perform routine tasks related to their DOI records and metadata submissions.	Members can sign in and upload metadata submissions or queries in XML format, as well as view details about their past submissions. The current admin tool lives at doi.crossref.org.	We will replace the current Admin tool with an interface that integrates with the new authentication and authorisation service and uses interstitial APIs for its functionality. The new interface will also be more accessible and usable.
ORCID Auto-update	A workflow without a user interface or authentication with the Crossref system.	Auto-update pushes metadata about works in the Crossref database to the ORCID API and asks researchers’ permission for Crossref to push further updates and additions.	The main goal of rebuilding our ORCID Auto-update is to make it a standalone service, less entwined with the content system. We will also tweak functionality that we have been unable to make in recent years.
Reports	A series of tables and dashboards presenting metadata from the Crossref database, including DOI resolutions, conflict reports, error reports, and participation reports.	Most report interfaces are openly available via the Crossref website. They are used mostly by members to self-serve information about their metadata on the member, prefix, or title level.	We will change the data source of these reports to the data lake and unify their overall architecture. On the frontend side, there is much scope for improving the usability and accessibility of these interfaces for greater analysis by members and the community.

Contributing to the Research Nexus (CRN)

Name of component	What is it?	What does it do?	What will change with the rebuild?
Funder matching	Matching member-submitted funder metadata to a funder identifier.	For works with funders and no funder identifier, match the funder name to a funder registry entry.	Build a new matching architecture and match funder names to ROR identifiers. Matches are added directly to the REST API as asserted by Crossref.
Preprint matching	For registered journal articles, check if there is a corresponding preprint it can be linked to.	Looks for an exact match between the title and authors. Currently notifies the preprint member via email who is obliged to add the relationship to their metadata (which Crossref reciprocates from journal article records).	Research and test a new method for matching. Add matches directly to the REST API, marked as asserted by Crossref.
Reference matching	For references deposited without a DOI, look for a corresponding Crossref DOI and add those to the metadata (see our recent update on the scale of this service)	Several different processes are used, depending on the completeness of the deposited metadata. Matching is carried out during the deposit process and added to the XML record.	Research and test a new and further improved method for matching. Add matches to the REST API, marked as asserted by Crossref.
XML API	A public API to deliver metadata in XML format.	Use a DOI to look up metadata or construct an XML query.	Reimplement separately from the deposit system. Re-evaluate the features.
Metadata enrichment	Supplement member-submitted metadata with other information held by Crossref.	Crossref metadata records have `information` added during deposit, including member information, citation counts, and relationships.	Review current processes and identify those that can be separated from metadata submission. Where possible, build standalone services that create enrichments, preparing for additional third-party data sources.

Open and Sustainable Operations (OSO)

Name of component	What is it?	What does it do?	What will change with the rebuild?
Authentication & Authorisation	An identity management system	Authenticate and authorise users to access the different Crossref tools and services.	Improve security, allow for member and sponsor self-service to manage users and permissions across their teams and vendors.
Billing	A service to generate invoices and reports based on new record registrations per quarter.	Generate a .csv of billable submissions and upload to our accounting system to generate invoices.	Reduce technical debt and remove code from the legacy system; allow for point-in-time billing.
Notifications	A service to send emails or callbacks to members as new DOI records are registered or existing ones updated.	Transactional notifications on submission status. Additional notifications when works are cited by other works.	Reduce/eliminate the number of emails sent, reduce technical debt, remove code from the legacy system.
Handle system	Update and look up DOI in the Global Handle Registry.	Registers the DOI string and associated landing page URL for redirection (DOI resolution).	Implement better fallback and error handling to make connections with the Global Handle Registry more resilient.
Deposit system	System that receives XML deposits from members.	Processes deposits and makes them available to other systems, like the REST API.	Separate out the different components of the deposit system.

The three-year timeline refers to the duration of the additional investment that is now being made. We don’t expect all of the rebuild work to be complete within three years. And system maintenance and improvement is never done. The additional resources will accelerate our ability to address these projects and get them substantially complete or meaningfully underway within three years. We will start with the things that are easiest to separate out of the current system, and work our way through it. The deposit system will be the last part we’ll tackle— after the initial 3-year period. Exactly how we’ll break out or couple together its components will depend somewhat on how the rest of the work in all three programs proceeds.

Additionally, there will be work in the OSO program to pull out deprecated/unused code from the content system as other programs build new free-standing services.

Of course, an end goal is to rebuild all of this in open code repositories, and eventually turn our amber-coloured ‘open source’ to green in our biannual POSI audits.

What does this new investment and approach mean for our community?

Having seen the long list of things we’re planning to do, you may be wondering how this will impact you. Here are some things you should know:

You’ll be able to use our services throughout this project. Part of the reason why we are rebuilding different functions one by one is that we don’t want to interrupt the services we offer to our growing membership and users. The current services will continue to run until the new ones are in place, and we’ll have clear transition plans and documentation in cases where you need to update your workflows.

We won’t just rebuild like-for-like, but we’ll make improvements when we rebuild services. We want to continue to provide the functionality we currently offer to our members, but we also realise there are things that could be better. The clearest example of this is member self-service, which will be one of the main priorities for the CCT program next year. For example, as mentioned above, in 2025, about 15% of member support tickets were contact updates: changes members should be able to make themselves in seconds, but which our membership team handles by hand; that is the kind of thing that will improve life for both members and staff.

We are planning to deprecate some older/less-used services along the way, but we’ll give you plenty of notice. Crossref is probably running more services than you realise! We’re therefore evaluating all our services as part of the redesign, and keeping only the ones that are actively used and really add value for the community. If we decide to retire a service, we will contact all users so they have enough time to transition to an alternative service. Everything we’ve deprecated so far is also itemised on our website.

Community input will be a big part of the process. To ensure the services we offer meet your needs, we will need to keep talking with you. Our community managers and UX researchers will be reaching out to users of different services to understand use cases and ensure these continue to be met within the new system.

This blog is the first tagged ‘system-redesign’ so follow this category for ongoing updates. If you have feedback or questions, please head to the dedicated community forum space for system redesign and ask us in the open.

Two halves of the same decision

Later this week we’ll also announce the latest round of fee changes resulting from our Resourcing Crossref for Future Sustainability (RCFS) work. What we charge, and what we do with what we collect, are two halves of the same decision; fees are to come down at the same time as we commit the largest single investment in our infrastructure we have ever made. Both follow from the same principle: we set fees to cover the cost of running the infrastructure, and when we hold more than we need, it goes back to the community and into the longevity of the scholarly record.

Our recent position paper on PIDs in research infrastructure policy argues that the value of an identifier lies in the metadata and services around it, and in whether those can be maintained over the long term. This approach to futureproofing open scholarly infrastructure is an example of that maintenance in action.

Acknowledgements

We would like to thank Sara Bowman, Carlos del Ojo Elias, Bharath Govindarajan, Stewart Houten, Martyn Rittman, Lena Stoll and Patrick Vale for their work on the system investment proposal and contributions to this blog post.

System Redesign on Crossref