About Taxonomies, Thesauri and Drupal

Background Reading on Taxonomies etc.

A Partial Bibliography

The need for shareable, interchangable, common taxonomies and vocabularies os a hot topic in knowledge management. Many partial solutions, or at least definitions of the problem, have been put forward. A good primer on this is Metadata? Thesauri? Taxonomies? Topic Maps! Making sense of it all
By: Lars Marius Garshol

Ian Dickson put out the call for a centralized 'Taxonomy Server' for Drupal, describing how such a project may be constructed.

Theory

According to academic papers on the subject, alternative vocabularies used to group different sets or axes of terms are labelled 'facets'. Lots of talk about it, especially in library circles, seems to have been done, but little is available on notation or communication of these concepts.

A heavy-duty, but comprehensive read is The ANSI Standard Guidelines for the Construction, Format, and Management of Monolingual Controlled Vocabularies Z39-19-2005

As it describes in section 5.3.4, "Facet Analysis" is the task of choosing how to construct your vocabularies, which terms should be grouped with which in the Drupal 'Categories' admin section.

Facets are a kind of structural metadata. Attributes that might be selected as facets for content objects are:
• Topic – the subject of the content object
• Format – the format of material (e.g., text, image, sound, etc.).
• Target audience – the appropriate reader for the content (e.g., Children, Adults)

That document also contains excellent recommendations on term selection (Grammar, Plural form, Capitalization etc, section 6) and illustrates a dozen alternative textual ways that taxonomies/thesauri may be notated (and hence could be useful as import/export formats).

Some possible ways of rendering taxonomies are available for inspections from, eg Library of Congress: Thesaurus for Graphic Materials I: Subject Terms

An Example Entry, LOC thesauri notation:

-------------------------------
  MT: Alphabets (Writing systems)
  UF: Letters of the alphabet
  BT: Writing systems
  NT: Initials
  NT: Phonetic alphabets
  Control No.: lctgm000270
-------------------------------
Mini-Glossary/explanation:
MT
Term
UF
Used For
BT
Broader Term
NT
Narrower Term

Some possibly useful canonic thesauri are accessable for browsing (but not convenient download) at the Library of Congress
In light of current research, the schemas used and defined there are positively archaic ... although they provide an interesting list of terms.

A much larger collection of thesauri is indexed at http://www.taxonomywarehouse.com/ or http://www.schemas-forum.org/registry/registry.html , including terms used by the United Nations and various governments.
... However these are just indexes of external sites, and resources found there are often only 'browsable' but not downloadable, and when they are, are each rendered in their own, usually proprietary markup notation scheme! Plus various curious licensing restrictions ... on word lists! Obviously there is a need for a useful, interoperable notation scheme!

The English Heritage National Monuments Record Thesauri Collection looks like a nice clean resource, listing thesauri for ['Monument Types', 'Building Materials', ' Historic Aircraft Type' and more ]. Again, it's browsable, not downloadable.

W3C published Quick Guide to Publishing a Thesaurus on the Semantic Web which does recommend a method, (which looks very much like what I ended up doing) but this doesn't seem to have caught on anywhere outside of their own glossary project (however that's cool as glossaries go).

Also from the W3C in 2008 (after this generation of taxonomy_xml was designed) there is Best Practice Recipes for Publishing RDF Vocabularies.
Rameau (FRENCH) is working example of this theory, and The Library of Congress "Authorities and Vocabularies" now provides a public service we can hook into also!
Hooray for getting into standards early! The taxonomy_xml lookup client was built before there were any servers in the world for it to talk to. When the servers were built a year later - This client started working! (gobsmacked)
The Library of Congress service also publishes its data in RDFa over HTML.

Historical Initiatives.

... include XFML (An XML representation of structured Thesauri) ... which appears to have totally died. Apparently giving way to as-yet-undefined RDF-based solutions.

There once even was a Drupal XFML module, long since retired apparently.

The syntax almost lives on in 'facetmap', an application and XML dialect that pretty much does the job, only it calls the multiple 'vocabularies' found in Drupal 'facets' and the 'terms' within them 'maps' (?). Original XFML at least called them 'topics', which was workable.

The Vocabulary Definition Exchange Appears to define a schema for representing terms and relationships within a vocabulary. Although it looks a bit like an awkward attempt, and I've not seen any actual examples of it in use.

An academic thesis, Migrating Thesauri to the Semantic Web gives some good case studies listing existing thesauri :

Zthes was supposed to be the answer to ANSI/NISO Z39.50 (a specification for interoperable subject searches - maintained by the Library of Congress, and closely related to Library OPAC systems) but it never got anything working or useful. They tried to publish an XML schema for representing taxa, and it's actually OK. But there are no references to anyone using it in the wild.

Current Implementation of Taxonomy import/export for Drupal (Oct 2007)

I've referred to Wordnet/RDF + Web Ontology Language (OWL) for the target dialect of XML used in this export schema.
Words and Terms come from, and are uniquely identified by the existing wordnet vocabulary, and their relationships are described using the RDF Schema 'ParentOf' and 'ChildOf' terms etc.

This modification of the taxonomy_xml.module is intended for two uses.

  1. To assist in migrating taxonomies between cloned sites, eg dev and live copies of essentially the same site. To this end, some effort has been put into maintaining vocabulary IDs and term IDs, because once they get out of synch, cloning and replication is almost a lost cause.
  2. To become a foundation for a Taxonomy Interchange initiative [Taxonomy Server] and therefore, I guess, somewhat similar to all those other 'taxonomy warehouses' but we intend to publish, for import/export, these shared taxonomies in a way that allows Drupal sites (or other related technologies) to share this data.

Sources of Taxonomies

The following sites provide downloadable taxonomies, Thesauri or Glossaries that are at least partly compatable with this import tool.