Outgoing

Just the Links, Please

We haven't done much with JSON in VIAF yet, but Ralph came up with a new feature for VIAF for which JSON seemed a natural fit: requesting a view of a VIAF cluster that just shows the links.  Here's the JSON for this new view of the Mark Twain cluster, http://viaf.org/viaf/50566653/justlinks.json: 

 

{ "viafID":"50566653",
    "BAV":["ADV10188047"],
    "BIBSYS":["x90056487"],
    "BNE":["XX945992"],
    "BNF":["http://catalogue.bnf.fr/ark:/12148/cb11927291n"],
    "DNB":["118624822"],
    "EGAXA":["vtls000823270"],
    "JPG":["500020427"],
    "LAC":["000002392","0105C0556"],
    "LC":["340758"],
    "NDL":["00459304"],
    "NKC":["jn19981002263"],
    "NLA":["000035028957"],
    "NLIara":["000160730"],
    "NLIcyr":["000156897"],
    "NLIheb":["000175478"],
    "NLIlat":["000133341"],
    "NUKAT":["vtls000634910"],
    "PTBNP":["46269"],
    "RERO":["vtls000164542"],
    "RSL":["nafpn-000083946"],
    "SELIBR":["98057"],
    "SUDOC":["027171876"],
    "VLACC":["000002392"],
    "WKP":"Mark_Twain"}

We return an array for each source because some may have multiple IDs (e.g. LAC in the example).  The array labels are VIAF's abbreviations (e.g. http://viaf.org/authorityScheme/LAC) for each of the source files, with WKP standing for Wikipedia.

For those of you that prefer content-negotiation to mangling URIs, the mime-type 'application/json+links' 'application/vnd.oclc.links+json' should also work.

Already built into VIAF is the ability to go from the source ID to the VIAF cluster: http://viaf.org/viaf/sourceID/SUDOC|027171876 will take you to the VIAF cluster http://viaf.org/viaf/50566653, and http://viaf.org/viaf/sourceID/SUDOC|027171876/justlinks.json will give you just the links for that page.

If you want ALL the links out of VIAF then visit the VIAF Dataset page at http://viaf.org/viaf/data.  It has a pointer to a file that lists each of the 27 million VIAF ID to source ID links.

We're having fun here at VIAF Central!, especially Ralph LeVan who thought of and implemented this.

--Th

 Update: 2012.05.09  changed mime type (old one should still work)

VIAF Dataset

VoID The VIAF dataset is now available for public consumption!  http://viaf.org/viaf/data describes and links to the files involved and describes how we expect the ODC-By license to be applied.  We are not sure just how popular the files will be, so if the site appears slow, please stop downloading and come back later.  From my machine here at OCLC my browser is estimating 20-30 minutes to download the larger files, from my home it was double that.

For more about this, see a previous post: http://outgoing.typepad.com/outgoing/2012/04/viaf-developments.html.

One question that has come up was whether it would be possible to incorporate VIAF identifiers and information into a dataset that is released under a CC-0 license.  The short answer is yes.  Here's a longer answer:

• We would like to see acknowledgement of VIAF as a source somewhere on your site
• We encourage the use of VIAF URI's where appropriate and they can be considered acknowledgement
• Incorporation of those VIAF URI's and associated information from VIAF should not prevent you from releasing your dataset under CC-0, since the URI's are considered sufficient acknowledgment

--Th

 

 

Uniform Titles in VIAF

Uniform Titles have been added to the Virtual International Authority File!

Now VIAF covers*:

• Personal names
• Corporate names
• Geographic names (only jurisdictional names so far)
• Uniform titles

There are a few more types of names in authority files, such as families (VIAF has a few since some sources consider these 'personal'), trademarks, and non-jurisdictional places. And then there are authority records for deities and imaginary people that start to be closer to concepts than names. Currently we don't consider concepts such as 'beauty' or 'agriculture' in scope for VIAF, but just about everything else in authority files is.

Adding uniform titles was a big step for VIAF and I expect that we are far from done with it. Titles are different from people and corporations. Compared to people and corporations, titles are not authors, but many of them have authors. In addition uniform titles recognize the FRBR concepts of works and expressions. Of course, since uniform titles have been around much longer than the current FRBR understanding of works and expressions, the relationships are not quite as explicit as we might like, but the basic relationships are there and needed to be recognized in VIAF.

Another aspect of uniform titles is that much of the information about them is in associated bibliographic records, not in authority records. This is true even to the extent that many uniform titles are not in the authority files at all. Since VIAF already uses information from bibliographic records to expand the information in other types of clusters, we initially planned to pull that information into VIAF for uniform titles. Unfortunately we found that creating VIAF clusters that do not have any actual authority records related to them started to open up issues that were growing more and more complicated as we got into them. In view of the limited time we felt we could spend getting uniform titles into production (~3 months), we decided to cut back to only using the information in the authority records and to spend our time trying to recognize and directly link the relations implicit in those records. Now that we have gotten this far we are starting to see some ways we could accommodate 'recordless' uniform titles, so we may get to that in the future.

Even without information from bibliographic records, we think uniform titles can make interesting VIAF clusters. Take a look at a well know author, such as Jane Austen. Expanding the 'Uniform Title Links' section you should see a number of works, and under those, expressions of the works. A well known work such as Pride and Prejudice will have its own work cluster and show several different expressions, such as Pride and Prejudice in Spanish.

It is not as clear how to handle things like 'selections'. As you can see in Jane Austen, we are treating both selections and 'works' as their own separate work. Also in Jane's list you can see works that have expression records, but no work record (e.g. the Polish version of the Wit and Wisdom of Jane Austen). We considered creating a VIAF cluster for those authority-recordless works, but that too opened complications, so currently such works do not have a VIAF cluster.

We hope uniform titles in VIAF will be useful, and would very much like to hear what people think of them and how they can be improved.

---------

NOTE: Since the uniform titles are so new, they should probably be considered 'beta' until people have a chance to look at them in more detail.

* From a FRAD (Functional Requirements for Authority Data) point of view, these are all names, but VIAF treats them as identifying what the name is about, for example http://viaf.org/viaf/96987389 represents Tolstoy himself, not his name or a record about his name. To a great extent VIAF is driven by the data it ingests, but does its best to reformulate it in a linked-data friendly way without forgetting current library applications that depend on it.

--Th

ANOTHER NOTE:  This was actually written in the fall of 2011, right after uniform titles were added.  I recently found it and thought it was still of interest.

--Th

xA

To a great extent, VIAF is a pure reflection of its source authority files and the bibliographic records linked to them.  We have dedicated our limited VIAF resources towards improving the automatic processes and error reporting capabilities of VIAF rather than spending much time manually fixing problems with clusters.  However, we knew from the beginning that VIAF would need methods for merging and splitting clusters and that has been handled with internal tables and procedures.  In fact, we get a steady stream of emails noting problems, mostly about two clusters that should be merged, and do our best to deal with them, whether it requires reporting problems to one of the VIAF participants (we never edit source authority records here at VIAF central), changes to the code that merges records into clusters, or adding to our internal tables.

Two years ago, after giving a talk about VIAF at a RERO meeting, there were several questions about how they could not only participate in VIAF, but have a direct way of adding and modifying VIAF clusters. Thinking about how we could do this we realized that we might be able to build something that addressed both external and internal issues with VIAF.  Not surprisingly, we started with our internal issues, with an eye towards the possibility of more general use.

We are calling the result xA (for Extended VIAF Authority) and it is now public at http://viaf.org/hosted/xa/. Internally the records are stored in MADS 2.0.  Interaction with the site uses SRU for retrieval, XSLT for formatting, Atom Publishing Protocol (AtomPub) for editing operations such as create, update and delete, and a little help from JavaScript and JQuery. Much like VIAF itself, there isn't anything hidden here.  Anyone could write an interface to xA using the (as yet undocumented but straight-forward) calls that we are using.

An example record in xA is the one for Barack Obama.  Since we were unable to algorithmically match some of the source authority records with each other, we manually pulled together most of them into a single cluster using an internal table.  Now we have a xA MADS record which identifies the source authority records, adds the President's birth month and day, and adds an additional source record that wasn't in the internal table.  Currently we have the capability of creating personal and corporate authority records.

xA gets integrated into VIAF as an additional source file.  No special privileges were needed (we treat any link between source records as a 'forced' link which is almost always respected).  It is probably worth noting that changes in xA are not direct changes to VIAF.  We rebuild VIAF about once/month, and it is at that point that xA records get their chance to affect VIAF clusters.  To give more immediate feed-back to VIAF, we have considered flagging clusters that we expect to be affected by new xA records, but have not pursued that yet.

By the way, xA is the 24th source file we are merging in VIAF.

Right now record creation and editing in xA is only available internally, although we do have a external authorities project we expect to use the same framework.  How and if the process of controlling and adding to VIAF in this manner can be made more public is something we will be exploring.  Right now we are just happy to have finally gotten xA into production and that we no longer need to deal with those internal merge tables!

No direct linked data yet either, although since xA shares much with VIAF, that should not be too difficult.

You are free to poke around in xA.  I should warn you, however, that it will let you try to edit and create records, but will challenge you before you can save your work, so you won't be able actually change anything.  We do have a site at http://viaf.org/hosted/sandbox that will let you save things if you know the secret user name and password (guest/guest).

Thanks to:

• Ralph LeVan (SRU, AtomPub, Web)
• J.D Shipengrover (Interface)
• Karen Smith-Yoshimura (Requirements, suggestions, enthusiasm)
• Jenny Toves (Requirements, integration into VIAF)
• Jeff Young (MADS advice, protocols)
• Eric Childress (General advice)

--Th