The Second Wave
You might have been wondering why I've been banging on about XMP here. Why the emphasis on one vendor technology on a blog focussed on an industry linking solution? Well, this post is an attempt to answer that.
Four years ago we at Nature Publishing Group, along with a select few early adopters, started up our RSS news feeds. We chose to use RSS 1.0 as the platform of choice which allowed us to embed a rich metadata term set using multiple schemas - especially Dublin Core and PRISM. We evangelized this much at the time and published documents on XML.com (Jul. '03) and in D-Lib Magazine (Dec. '04) as well as speaking about this at various meetings and blogging about it. Since that time many more publishers have come on board and now provide RSS routinely, many of them choosing to enrich their feeds with metadata.
Well, RSS can be seen in hindsight as being the First Wave of projecting a web presence beyond the content platform using standard markup formats. With this embedded metadata a publisher can expand their web footprint and allow users to link back to their content server.
Now, XMP with its potential for embedding metadata in rich media can be seen as a Second Wave. Media assets distributed over the network can now carry along their own metadata and identity which can be leveraged by third-party applications to provide interesting new functionalities and link-back capability. Again a projection of web presence.
(Continues.)
XMP has much in common with RSS 1.0. They are both profiles of RDF/XML. They are both flawed in certain respects because of self-imposed limitations. But they both build on a robust and open data model for the web (RDF) and are reasonably open, at least they are extensible. One (RSS 1.0) was defined in an open process by committee, the other is an open (i.e published) specification provided by a vendor.
From our point of view both specifications are sufficiently advanced to be immediately useful. I'm not sure how one could interact with the further development of either specification. RSS 1.0 is essentially frozen with Atom being posed as a successor technology, although Atom does not conform to the RDF model. (The upshot is that an RSS 1.0 feed can be consumed completely by an RDF-aware application, while an Atom feed would need to be pre-processed before any RDF "goodness" could be gleaned from it.) By contrast, XMP is a vendor-defined technology and alive, if not perhaps kicking. I am unaware of any process to formally contribute to the XMP development apart from shouting from the terraces. None the less, both technologies are usable as is.
It is curious that no consistent packaging (and delivery) of metadata has yet been achieved with HTML, the original web interface. The HTML <title> and <meta> elements are employed by publishers with various degrees of consistency. There are also RDF islands that can be embedded within HTML comments (as used e.g. by CC licenses). And then there are COinS objects. But it's all a bit of a mish-mash to date. Certainly, I don't recall seeing any guidelines from CrossRef as to how machine readable metadata (even markup for the DOI itself) may be embedded within HTML pages, rather than on HTML pages for human readers.
This lack of uniform metadata deployment for HTML pages could be something to do with context. With RSS and XMP we are dealing with remote objects, whereas with HTML we are generally accessing this directly on the content server and so have a semantic context. It could be though that metadata delivery from HTML pages will finally be more uniformly available with the further development of standards such as microformats and especially RDFa, GRDDL, etc. It is also interesting to note that an XMP packet could just as easily be embedded within the HTML page, and if this technology were to be adopted more widely for embedding in other media assets then why not consider the same technology for ordinary web pages?
I can't help feeling though that XMP has a lot of promise and is very timely. There are only three real obstacles: creating XMP packets, writing them and reading them. To my mind, once one has a good grasp of XMP then creating the packets can be done with common tools. The same, more or less, for reading the packets. I have shown earlier that this is readily achievable. The only major block is writing the packets into media files although there is support for create/write (if patchy) by open source libraries, as well as there being support (perhaps limited) from products for create/write. But, anyway, it's certainly do-able.
Comments
Tony --
I'm having a difficult time fathoming the depth of the possibilities for XMP. In particular, I can't place it in any consistent context or get an accurate description of what it is/does. The Adobe site isn't much help -- it talks about how XMP is great inside Adobe applications, but it seems that you have wider expectations and desires.
In short, I guess: I don't have an answer to the question "Why should I care about XMP?"
Posted by: Peter Murray | September 11, 2007 04:19 PM
Tony - thanks for all the practical info on XMP. I take your point that CrossRef hasn't provided any guidelines in this area ("I don't recall seeing any guidelines from CrossRef as to how machine readable metadata (even markup for the DOI itself) may be embedded within HTML pages, rather than on HTML pages for human readers.")
I think there could be huge value in having standards on embedding machine readable metadata in HTML. This could vastly improve search engine indexing, for one.
So - we'll think about what CrossRef should do about this - any thoughts from our member publishers about this?
Posted by: Ed Pentz | September 12, 2007 06:49 AM
Peter:
I had hoped that my post was a credible attempt to address the question of "Why XMP?". Maybe I missed something.
According to the XMP spec XMP provides for three things: a data model, a storage model, and schemas. The major technology contribution is the second - the storage model. To define a means for embedding XML packets within arbitrary binary and teext file formats is a key step forward. This means that metadata lodged in media files is done so in an "open" fashion and free of all the restrictions and impediments that other mechansims bring along: TIFF, EXIF, IPTC Headers, ICC Profiles, Photoshop, etc. XMP allows for a single means of emnedding metadata irrespective of file format and which can be manipulated with standard web-based tools (XML editors, and the like).
The fact that XMP uses RDF/XML (more later) is also a major plus. RDF is an open data model which allows multiple data sources to be merged. The same cannot be said of generic namespaced XML. Different XML documents conform to differnet schemas and can not readily be merged. Because XML is more packaging and syntax than data model. (Unless you appeal to the Infoset which is a pretty low level and murky affair.) RDF/XML, on the other hand, uses XML as a carrier for data conforming to a model. An RDF parser can read RDF from multiple sources and multiple formats (XML, N3, RSS, JSON, SPARQL, etc) and abstract the data and fit it to the RDF data model. However, XMP defines only a subset of RDF proper and does not recognize some of the later constructs. It's still RDF though.
That's good. But, where XMP realy screws up is the schemas which provide fossilized views of metadata schemas mapped to the RDF data model. Let's consider the Dublin Core schema which XMP has stamped it's imprint on. (Bit of a shame for those who would like to more meningfully describe objects with DC in XMP.) As a very simple example let's consider the property "dc:identifier". I may have an identifier which is a URI but I cannot express that within XMP's definition of rthe DC schema. There it must forever be a string. I know what I want to do. I know what is right. But XMP won't allow me to express this. Hard to see why really. Doesn't impact on the XMP data model, nor the XMP storage model. It might impact on the XMP Toolkit (but who really cares about that) or on certain applications that make use of XMP. I don't see why it should impact on XMP itself. Toolkit and apps are adjuncts.
So, what do we have? We have a bastardized version of RDF (represented in RDF/XML) which can be fitted into arbitrary file formats. OK, so I think there's still a lot of mileage in that. And as I tried to make clear, I would see that as being not dissimlar from the case of RSS 1.0 which itself is a bastardized version of RDF (represented in RSS). Still might useful despite the handicaps.
Seems to me there are a number of options:
We had some similar issues with RSS 1.0 (e.g. RSS 1.0 proscribes repeated prroperties which we chose to ignore for our dc:creator fields). At the end of the day, the issue is one of prgamatics. To me it seems very useful.
Posted by: Tony Hammond | September 12, 2007 07:48 AM
Ah, now this is the key point I think I was missing:
Inclusion of metadata into arbitrary files is a big deal. I redoubled my efforts and finally found the XMP Specification (XMPSpecification.pdf, 948K), so I'll take a deeper look. I knew about embedding XMP into JPEG2000 (which is easy given its file format's run-length encoding and provision for arbitrary content 'boxes'), but seeing how they work with PNGs, JPEGs, and other file formats should be instructive.
Thanks for the additional discussion points!
Posted by: Peter Murray | September 12, 2007 11:26 AM
Tony, thanks for the posting. I'd be really interested to hear a CrossRef specific use case for using XMPP.
Posted by: Ed Summers | September 13, 2007 09:11 AM
Hi Ed:
Just to be clear, I've been talking about Adobe's XMP (Extensible Metadata Platform), not XMPP (Extensible Messaging and Presence Protocol). The description technology not the messaging protocol of which I remain cheerfully ignorant.
CrossRef interest should be in the embedding of DOIs into arbitrary media files which can then be propagated beyond the origin content platform and propelled out into the wider world with a keen sense of their own identity. Like nametagging a schoolchild's items of clothing. If there's additional metadata added to provide context then so much the better, but otherwise the DOI as a digital identity will suffice. That provides the "phone home" hook.
Tony
Posted by: Tony Hammond | September 14, 2007 08:34 AM