Outgoing

More ships in WorldCat Identities

I've written before about ships in WorldCat Identities, but never tried to find all the ships for which there are Identity pages.  Recently Peter McCracken suggested that ShipIndex.org might link to appropriate Identities pages (and vice-versa), so we went ahead and tried to locate all the ships in Worldcat.

Not surprisingly, identifying all the ships took a little work.  The task reminds me of trying to find fictitious characters in WorldCat, and we used a similar approach looking at the parenthetical qualifier that describes the name.  For instance the Confederate sloop Alabama is controlled as Alabama (Screw sloop).  The trouble is that there are quite a few different types of ships and boats, from the Albatross (Steamer) to Zingarella (Bark).  Here is the basic list we are currently using:

• barge
• bark
• barque
• boat
• brig
• carrier
• catamaran
• corvette
• cruiser
• destroyer
• dredge
• freighter 
• frigate
• galleon
• ironclad
• ketch
• packet
• schooner
• ship
• sloop
• steamer
• submarine
• sweeper
• tender
• trawler
• trimaran
• tug
• vessel
• whaler
• yacht

I'm sure there are more words that describe ships, so please let me know if you notice anything missing.

Of course you have to be careful not to include airships or boat seaplanes or ship replicas or townships or Steamboat Springs, so I also have a list of patterns to skip, and then a list of overrides for the skips.

Altogether I found just under 40,000 Identity pages for ships in WorldCat Identities.  About 6,700 of them have an LC/NACO authority record associated with the name.  Approximately 11,000 had two or more citations and 6,000 had three or more.

How successful we will be in linking the databases remains to be seen, but it was interesting to try to find all the ships.

Here are the regular expressions (in Python):

#Always accept these:

simpleRE = re.compile(r'\(battle.?ships?\)|\(.*cargo.?ships?\)|\(cruise.?ships?\)|\(motor.?ships\)|\(passenger.?ships?\)|\(steam.?ships?\)|\(war.?ships?\)|\(ships?\)', re.IGNORECASE)

#Skip these

skipRE = re.compile(r'\(airship\)|\(air.ship\)|\(.*boat seaplane.*\)|\(.*boat work.*\)|\(.*fellowship.*\)|\(.*friendship.*\)|\(.*govenership.*\)|\(.* key.*\)|\(.*partnership.*\)|\(.* replica.*\)|\(.*ship[a-z].*\)|\(.*springs.*\)|\(.*township.*\)|\(.*twonship.*\)|\(.*voivodeship.*\)', re.IGNORECASE)

# The main regular expression

shipsRE = re.compile(
    r'\(.*barges?\b\)|\(.*bark\b\)|\(.*barkentine.*\)|\(.*barque.*\)|\(.*boat.*\)|\(.*brig\b\)|\(.*carrier.*\)|\(.*catamaran.*\)|\(.*corvette.*\)|\(.*cruiser.*\)|\(.*destroyer.*\)|\(.*dredge.*\)|\(.*freighter.*\)|\(.*frigate.*\)|\(.*galleon.*\)|\(.*ironclad.*\)|\(.ketch.*\)|\(.*packet.*\)|\(.*schooner.*\)|\(.*ship.*\)|\(.*sloop.*\)|\(.*steamer.*\)|\(.*submarine.*\)|\(.*sweeper.*\)|\(.*tender.*\)|\(.*trawler.*\)|\(.*trimaran.*\)|\(.*tugs?\b\)|\(.*vessel.*\)|\(.*whaler.*\)|\(.*yacht.*\)', re.IGNORECASE)

--Th

VIAF and OpenSearch

Ralph LeVan recently added OpenSearch capability to the Virtual International Authority File (VIAF).  Since the VIAF site is built on top of SRU, the easiest way to that was to use proposed SRU extensions. Here is Ralph's description of it:

Currently, an SRU request generates a searchRetrieveResponse.  The SRU committee has proposed the addition of a new parameter to an SRU request that can indicate to the server that the client would like some other form of response.  For OpenSearch, that means an RSS response.  We have implemented the new parameter and have succeeded in providing OpenSearch access to VIAF.

Here is an example of an SRU query for the name “Dempsey” in VIAF: http://viaf.org/search? query=cql.any+all+ "dempsey"+ &maximumRecords=100 &stylesheet=xsl/results.xsl &sortKeys=holdingscount.  This returns an SRU searchRetrieveResponse, though that may not be obvious to you because an XSLT stylesheet was used to render that response into HTML.  To turn that search (or any SRU search of VIAF) into an OpenSearch query simply add &http:accept=application/rss%2bxml to the end of the URL.

http://viaf.org/search?query=cql.any+ all+"dempsey"+ &maximumRecords=100 &stylesheet=xsl/results.xsl &sortKeys=holdingscount &http:accept=application/rss%2bxml

Support for RSS is built into most browsers and clicking on that link should get you a screen with the results of the search and options to subscribe to that search.  There’s no point in using the subscription option yet, but sometime in the near future you’ll be able to use that search and see only the records that have changed in the results.

OpenSearch is a popular protocol for search aggregators, such as A9.  In addition, we are now providing a hint to browsers when they display VIAF records that the database is available via OpenSearch, so you may see VIAF offered as a potential search engine in your browser’s search box.  OpenSearch requires an OpenSearch Description Document.  We have one at http://viaf.org/allFieldsSearch.xml.  All you have to do is point an OpenSearch client and that URL and it will know everything it needs to search VIAF.

One of the weaknesses of OpenSearch is that it doesn’t support a query grammar.  In our OpenSearch implementation we had to pick an SRU/CQL query that we could drop OpenSearch search terms into.  We chose the generic “All Fields” search that we expose on the VIAF home page.  Other pre-defined queries could be supported with other Description Documents, but we don’t have enough experience with exposing multiple Description Documents to know how the clients will expose the various search options made available.

Our SRU server, which has built in support for Lucene and DSpace databases (SOLR coming soon) is available as open source at Google Code (http://code.google.com/p/oclcsrw/).  The new feature to support the http:accept parameter will be committed to Google Code soon.

If you have any questions about this, contact Ralph LeVan (levan@oclc.org)

Ralph

New PURL Server running

As promised earlier, we did in fact install a new version of the PURL server early on July 27, 2009.  We had a couple of surprises, but they have been fairly minor, and with some adjustments we are now completely transitioned to the new software.

The software is open source and was written for us by Zepheira.  The original specifications for the new software allowed for some variation in the exact operation of PURLs.  Our thinking was that part of the reason we were redoing the software was to make improvements to PURLs, and we wanted to impose as few restrictions on that as we could.  As we attempted the transition, however, reality collided with our best intentions and we found that changes that seemed quite reasonable had too great an impact on existing users.  Working through those issues took quite a bit of work and a lot of testing, but I think we finally managed it.

Probably the most important new feature in PURLs is the ability to create advanced PURLs for which you can control the HTTP status code (e.g. the new 303-See Other redirection code used in the semantic Web).  You can also delete PURLs.  Once you have done this the PURL is tombstoned so there is a record in the PURL server, even though the PURL no longer is available for redirection.

In case you are interested, here are some of the problems we encountered:

• Non-standard characters.  Some characters (e.g. the dollar sign $) are valid is some parts of a URI, but not in others.  We still have a few unusual PURLs with valid but unusual characters we aren't handling correctly.
• Undocumented features.  There were some cases where the old PURL server would try to 'do the right thing' even though the 'right thing' was neither documented nor expected, for instance treating a PURL as a partial indirect even though it was not.  In almost all cases we were able to fix these problems by more conventional PURL techniques.

• DNS problems.  There was at least one case where someone had redirected a domain name to purl.org.  We have actually used that trick in-house, but did not realize anyone else was doing it.  The new software is more aware of where requests originate than the old system and blocks this sort of thing.  In the case in question we were able to accommodate them, but it isn't something we are encouraging.

The DNS problem in particular shows how difficult it can be to thoughly test this type of software.  PURLs are very much embeded in the Web and it is difficult to predict all the ways various PURLs intereact.  We did get better at it with (rather painful) experience, however.

Since purl.org resolves something close to two million PURLs/day (about 20/second), we do everything we can to avoid disrupting that flow, and had only minimal interruptions in this transition.

--Th

Library hand

I noticed a reference to Library Notes on the Library History Buff Blog.  At Illinois I must have taken a course that talked about the history of library education because I wrote a paper about library hand.  Library hand, like almost everything else in libraries at the end of the 19th century was strongly influenced by Melvil Dewey, so I was interested to see that an article in Dewey's Library Notes Vol. 1, #4, March 1887 pp. 273-282 about library hand is available in Google Books.

As I remember, I was able to find some original material about library hand in the archives at Illinois and the school also had a good collection of old library school brochures.  As far as I could tell, students at many schools were expected to master library hand at least up until World War II.  Even in 1887 when the Library Notes article came out it was recognized that 'mechanical writing' would probably dominate in libraries, but legible handwriting was still considered very important.

Ever since writing that paper I've been interested in Dewey, who was quite a character.

OCLC publishes the DDC.

--Th

New PURL server

We are planning on installing a new version of the PURL software at around 7:00 AM ET this coming Monday, July 27, 2009.

We've done a lot of testing on this, so we are confident it will work.  There are some differences in some PURLs that were using undocumented 'features' of the old PURL software.

I'll post again about our experience after the cut over.

--Th

Expanding VIAF

We have been busy adding new files to the  Virtual International Authority File, streamlining our handling of files, adding new features and getting that last bug out of the system.

Currently we are loading personal name records from 16 different authority files:

Bibliothèque nationale de France
National Library of the Czech Republic
Deutsche Nationalbibliothek
National Library of Israel
Library of Congress
National Library of Sweden
National Library of Australia
Bibliotheca Alexandrina (Egypt)
Istituto Centrale per il Catalogo Unico (Italy)
Biblioteca Nacional de Portugal
Biblioteca Nacional de España
Swiss National Library
Vatican Library

That may look like only 13 files, but Israel has four authority files based on the script of the names.  Some of the files are still small test files, but should be full files soon.

We still have lots of work to do, both in the interface and in the record matching, but we are pleased with what we have accomplished so far.

--Th

Changes to VIAF

The Virtual International Authority File (VIAF) is a project jointly administered by LC, the BnF, the DNB and OCLC.  The National Library of Sweden is also a participant, soon to be joined by several other libraries.  We currently have about 7.8 million VIAF records built from 9.2 million source records.

The VIAF site has recently had a major overhaul.  What you now search are records created from a merge of matching source authority records.  Within this record you can see what source records were used to create it, along with cross references and other information gleaned both from the authority records and from associated bibliographic records.

In addition to the VIAF record there are MARC-21 and UNIMARC views of the data.  For example, here is the record for Barbara Tillett (one of the main forces behind VIAF) : http://viaf.org/viaf/77390479.  By adding a file extension you can get a MARC-21 version: http://viaf.org/viaf/77390479.m21, and a UNIMARC view: http://viaf.org/viaf/77390479.unimarc.  The MARC export is new, so any error reports are welcome.

In the not too distant future we expect to offer a 'linked data' view of VIAF.

The whole site is just a thin layer of XSLT stylesheets and URL rewrite rules over an SRU database. Other than some JavaScript, virtually everything else is XSL transforms on XML data returned from SRU queries.  The graph is put up using a JavaScript vector graphics package jsGraphics.  Mouse overs, etc. are currently working better in Firefox than IE.

We know we have work to do to refine the matching process, but are interested in errors people notice, both names that should have been brought together and those that should not have, as well as interface problems you may have.

--Th

Ships in WorldCat Identities

A colleague was in my office wondering if Captain Cook's ships are in WorldCat Identities.  Sure, they are.  Here are the ones I found:

Resolution
Discovery
Adventure

and Endeavour (suggested by Debbie below).

In honor of Darwin's 200 birthday here is the HMS Beagle.  And of course the Titanic.

If ships, why not spaceships?  The key word is spacecraft:

• Hubble Space Telescope
• Apollo 13
• Columbia
• Vostok

There are even imaginary space vehicles (e.g. Rama).

Mostly people write about ships, but under AACR-2 the ships themselves can be considered authors (e.g. of their logs).  If you look at the LC/NACO authority record for the Hubble Space Telescope you will see it established as a corporate name.  Which almost makes sense!  Rama however, is established only as a subject, so it can't author anything.

--Th

Popular authors

In the UK, EU authors get paid a few pence for every loan of their books, so lending is tracked nationally.  Lorcan pointed me at a Guardian article about the latest book-lending statistics which show that James Patterson (an American mystery author) was the most borrowed author in the UK last year.

Here's the list (linked to WorldCat Identity pages):

1. James Patterson
2. Jacqueline Wilson
3. Daisy Meadows
4. Nora Roberts
5. Francesca Simon

The article points out that part of the trick is being prolific.  I read recently that Nora Roberts had 100 books on the New York Times bestseller list over the last ten years.

Looking at what these authors write, it's easy to see what people read: mysteries, love stories, children's books, mystery love stories, young adult mysteries, young adult love stories, and the occasional young adult Christmas paranormal love story mystery.

--Th

T-Mobile pricing

I was thinking about upgrading my T-Mobile/HTC Shadow phone to the new G1 that runs Google's Android operating system.  Before it came out they offered existing customers a pretty good deal on it, but last weekend I looked and they now want $350 dollars ($400 without a 2 year extension).  Pretty expensive, but still "special pricing that's just for T-Mobile customers."

To see just how special the pricing was, I checked the page for new customers: $180 (with a 2 year contract), about half the price.

Now I can understand why they are doing this.  They have this phone that lots of people would like to try, and see it as a way to bring in new customers.  When my current plan expires next year though, I expect that I will be looking around for what's new, rather than a happy T-Mobile G1 user like I should be.

--Th