Blog

This blog post is from Lettie Conrad and Michelle Urberg, cross-posted from the The Scholarly Kitchen.
As sponsors of this project, we at Crossref are excited to see this work shared out.

The scholarly publishing community talks a LOT about metadata and the need for high-quality, interoperable, and machine-readable descriptors of the content we disseminate. However, as we’ve reflected on previously in the Kitchen, despite well-established information standards (e.g., persistent identifiers), our industry lacks a shared framework to measure the value and impact of the metadata we produce.

We have just released a bunch of new functionality for Crossref Metadata Search. The tool now supports the following features:

• A completely new UI
• Faceted searches
• Copying of search results as formatted citations using CSL
• COinS, so that you can easily import results into Zotero and other document management tools
• An API, so that you can integrate Crossref Metadata Search into your own applications, plugins, etc.
• Basic OpenSearch support- so that you can integrate Crossref Metadata Search into your browser’s search bar.
• Searching for a particular Crossref DOI
• Searching for a particular Crossref ShortDOI
• Searching for articles in a particular journal via the journal’s ISSN

At the moment, Crossref Metadata Search (CRMDS) is a Crossref Labs project and, as such, should be used with some trepidation. Our goal is to release CRMS as a production service ASAP, but we wanted to get public feedback on the service before making the move to a production system.

Since I’ve already blogged about this a number of times before here, I thought I ought to include a link to a fuller writeup in this month’s D-Lib Magazine of our nature.com OpenSearch service which serves as a case study in OpenSearch and SRU integration:

doi:10.1045/july2010-hammond

(Click image for full size graphic.)

I thought I could take this opportunity to demonstrate one evolution path from traditional record-based search to a more contemporary triple-based search. The aim is to show that these two modes of search do not have to be alternative approaches but can co-exist within a single workflow.

Let me first mention a couple of terms I’m using here: ‘graphs’ and ‘properties’. I’m using ‘property’ loosely to refer to the individual RDF statement (or triple) containing a property, i.e. a triple is a ‘(subject, property, value)’ assertion. And a ‘graph’ is just a collection of ‘properties’ (or, more properly, triples). Oh, and I’ll also use the term ‘records’ when considering ‘graphs’ as pre-fabricated objects returned within a result set.

[See this link if you’re short on time: facets search client. Only tested on Firefox at this point. Caveat: At time of writing the Crossref Metadata Search was being very slow but was still functional. Previously it was just slow.]

Following on from Geoff’s announcement last month of a prototype Crossref Metadata OpenSearch on labs.crossref.org, I wanted to show what typical OpenSearch responses might look like in a more mature implementation.

I have taken the liberty of modelling these on the response formats that we are already providing in our nature.com OpenSearch service which in turn are based on the draft syndication formats that I blogged here earlier.

Following on from my recent post about our shiny new nature.com OpenSearch service we just put up a cheatsheet for users. I’m posting about this here as this may also be of interest especially to those exploring how SRU and OpenSearch intersect.

The cheatsheet can be downloaded from our nature.com OpenSearch test page and is available in two forms:

• Cheatsheet (PDF, 65K)
  • Cheatsheet (PNG, 141K)
  Naurally, all comments welcome.

(Click panels in figure to read related posts.)

Following up on my earlier posts here about the structured search technologies OpenSearch and SRU, I wanted to reference three recent posts on our web publishing blog Nascent which discuss our new nature.com OpenSearch service:

1. Service

Describes the new nature.com OpenSearch service which provides a structured resource discovery facility for content hosted on nature.com.

2. Clients

Points to a small gallery of demo web clients for nature.com OpenSearch which all use the text-based JSON interface.

3. Widgets

Introduces the new nature.com search desktop widgets which interface with the nature.com OpenSearch service via an RSS feed. (See also the screencast posted to YouTube.)

We hope that this new search service will prove to be useful and may also provide a model for other implementations.

In an earlier post I talked about using the PAM (PRISM Aggregator Message) schema for an SRU result set. I have also noted in another post that a Search Web Service could support both SRU and OpenSearch interfaces. This does then beg the question of what a corresponding OpenSearch result set might look like for such a record.

Based on the OpenSearch spec and also on a new Atom extension for SRU, I have contrived to show how a PAM record might be returned in a coomon OpenSearch format. Below I offer some mocked-up examples for each of the following formats for review purposes:

As posted here on the SRU Implementors list, the OASIS Search Web Services Technical Committee has announced the release of drafts of SRU and CQL version 2.0:

• sru-2-0-draft.doc
  • cql-2-0-draft.doc
  The Committee is soliciting feedback on these two documents. Comments should be posted to the SRU list by August 13.

[Update - 2009.06.07: As pointed out by Todd Carpenter of NISO (see comments below) the phrase “SRU by contrast is an initiative to update Z39.50 for the Web” is inaccurate. I should have said “By contrast SRU is an initiative recognized by ZING (Z39.50 International Next Generation) to bring Z39.50 functionality into the mainstream Web“.]

[Update - 2009.06.08: Bizarrely I find in mentioning query languages below that I omitted to mention SQL. I don’t know what that means. Probably just that there’s no Web-based API. And that again it’s tied to a particular technology - RDBMS.]

(Click image to enlarge.)

There are two well-known public search APIs for generic Web-based search: OpenSearch and SRU. (Note that the key term here is “generic”, so neither Solr/Lucene nor XQuery really qualify for that slot. Also, I am concentrating here on “classic” query languages rather than on semantic query languages such as SPARQL.)

OpenSearch was created by Amazon’s A9.com and is a cheap and cheerful means to interface to a search service by declaring a template URL and returning a structured XML format. It therefore allows for structured result sets while placing no constraints on the query string. As outlined in my earlier post Search Web Service, there is support for search operation control parameters (pagination, encoding, etc.), but no inroads are made into the query string itself which is regarded as opaque.

SRU by contrast is an initiative to update Z39.50 for the Web and is firmly focussed on structured queries and responses. Specifically a query can be expressed in the high-level query language CQL which is independent of any underlying implementation. Result records are returned using any declared W3C XML Schema format and are transported within a defined XML wrapper format for SRU. (Note that the SRU 2.0 draft provides support for arbitrary result formats based on media type.)

One can summarize the respective OpenSearch and SRU functionalities as in this table:

<th width="33%" align="center">
  OpenSearch
</th>

<th width="33%" align="center">
  SRU
</th>

<td align="center">
  no
</td>

<td align="center">
  yes
</td>

<td align="center">
  yes
</td>

<td align="center">
  yes
</td>

<td align="center">
  yes
</td>

<td align="center">
  yes
</td>

<td align="center">
  no
</td>

<td align="center">
  yes
</td>

What I wanted to discuss here was the OpenSearch and SRU interfaces to a Search Web Service such as outlined in my previous post. The diagram at top of this post shows query forms for OpenSearch and SRU and associated result types. The Search Web Service is taken to be exposing an SRU interface. It might be simplest to walk through each of the cases.

(Continues below.)