LinkedData.html 26 KB

Raw Blame History Permalink

<html xmlns="http://www.w3.org/1999/xhtml">
  <head>
    <meta name="generator" content=
    "HTML Tidy for Mac OS X (vers 31 October 2006 - Apple Inc. build 13), see www.w3.org" />
    <meta content="text/html; charset=us-ascii" http-equiv=
    "Content-Type" />
    <title>
      Linked Data - Design Issues
    </title>
    <link rel="Stylesheet" href="di.css" type="text/css" />
    <link rel="icon" href="diagrams/lod/597992118v2_64x64_Back.png"/>
    <meta http-equiv="Content-Type" content="text/html" />
  </head>
  <body style=
  "color: rgb(0, 0, 0); background-color: rgb(221, 255, 221);">
    <address>
      Tim Berners-Lee<br />
      Date: 2006-07-27, last change: $Date: 2009/06/18 18:24:33
      $<br />
      Status: personal view only. Editing status: imperfect but
      published.
    </address>
    <p>
      <a href="./">Up to Design Issues</a>
    </p>
    <hr />  <!-- http://www.cafepress.co.uk/w3c_shop.480759174  http://www.cafepress.com/+shirt,480756337 -->
    <a href="http://www.cafepress.com/w3c_shop">
    <img alt="Get a 5* mug" border="none" src="diagrams/lod/597992118v2_350x350_Back.jpg" align="right"/>
    </a>
    <h1>
      Linked Data
    </h1>
    <p>
      The Semantic Web isn't just about putting data on the web. It
      is about making links, so that a person or machine can
      explore the web of data. &nbsp;With linked data, when you
      have some of it, you can find other, related, data.
    </p>
    <p>
      Like the web of hypertext, the web of data is constructed
      with documents on the web. However, &nbsp;unlike the web of
      hypertext, &nbsp;where&nbsp;links are
      relationships&nbsp;anchors in hypertext documents written in
      <small>HTML</small>, for data they links&nbsp; between
      arbitrary things described by <small>RDF</small>,. &nbsp;The
      <small>URI</small>s identify any kind of object or&nbsp;
      concept. &nbsp; But for <small>HTML</small> or
      <small>RDF</small>, the same expectations apply to make the
      web grow:
    </p>
    <ol>
      <li>
        <p>
          Use <small>URI</small>s as names for things
        </p>
      </li>
      <li>
        <p>
          Use <small>HTTP</small> <small>URI</small>s so that
          people can look up those names.
        </p>
      </li>
      <li>
        <p>
          When someone looks up a <small>URI</small>, provide
          useful information, using the standards (RDF*, SPARQL)
        </p>
      </li>
      <li>
        <p>
          Include links to other <small>URIs</small>. so that they
          can discover more things.
        </p>
      </li>
    </ol>
    <p>
      Simple. &nbsp;In fact, though, a surprising amount of data
      isn't linked in 2006, because of problems with one or more of
      the steps. &nbsp;This article discusses solutions to these
      problems, details of implementation, and factors affecting
      choices about how you publish your data.
    </p>
    <h2>
      The four rules
    </h2>
    <p>
      I'll refer to the steps above as rules, but they are
      expectations of behavior. &nbsp;Breaking them does not
      destroy anything, but misses an opportunity to make&nbsp;
      data interconnected. &nbsp;This in turn limits the ways it
      can later be reused in unexpected ways. &nbsp;It is the
      unexpected re-use of information which&nbsp;is the value
      added by the web.
    </p>
    <p>
      The first rule, to identify things with
      <small>URI</small>s,&nbsp; is pretty much understood by most
      people doing semantic web technology. &nbsp;If it doesn't use
      the universal <small>URI</small> set of symbols, we don't
      call it Semantic Web.<br />
      <br />
      The second rule, to use <small>HTTP</small>
      <small>URI</small>s,&nbsp; is also widely understood.
      &nbsp;The only deviation has been, since the web
      started,&nbsp; a constant tendency for people to invent new
      <small>URI</small> schemes (and sub-schemes within the
      <span style="font-family: monospace;">urn:</span>
      scheme)&nbsp; such as <small>LSID</small>s and handles and
      <small>XRI</small>s and <small>DOI</small>s and so on, for
      various reasons. &nbsp;Typically, these involve not
      wanting&nbsp;to commit to the established Domain Name System
      (<small>DNS</small>) for delegation of authority but to
      construct something under separate control. &nbsp; Sometimes
      it has to do with not understanding that <small>HTTP</small>
      <small>URI</small>s are names (not addresses) and that
      <small>HTTP</small> name lookup is a complex, powerful and
      evolving set of standards. This issue discussed at length
      elsewhere, and time does not allow us to delve into it here.
      [ @@ref TAG finding, etc])
    </p>
    <p>
      The third rule, that one should serve information on the web
      against a <small>URI</small>, is, in 2006, well followed for
      most ontologies, but, for some reason, not for some major
      datasets. &nbsp;One can, &nbsp;in general,&nbsp; look up the
      properties and classes one finds in data, and get information
      from the <small>RDF</small>, <small>RDFS</small>, and
      <small>OWL</small> ontologies including the relationships
      between the terms in the ontology.
      </p><p>
      The basic format here for RDF/XML, with its popular alternative
      serialization N3 (or Turtle).  Large datasets provide
      a SPARQL query service, but the basic linked data should
      br provided as well.
    </p>
    <p>
      Many research and evaluation projects in the few years of the
      Semantic Web technologies produced ontologies, and
      significant data stores, but the data, if available at all,
      is buried in a zip archive somewhere, rather than being
      accessible on the web as linked data. &nbsp;The Biopax
      project, the CSAktive data on computer science research
      people and projects were two examples. [The CSAktive data is
      now (2007) available as linked data]
    </p>
    <p>
      There is also a large and increasing amount of
      <small>URI</small>s of non-ontology data which can be looked
      up. &nbsp;<a href=
      "http://ontoworld.org/wiki/Semantic_wiki">Semantic wikis</a>
      are one example. The "Friend of a friend"
      (<small>FOAF</small>) and&nbsp;<span style=
      "font-style: italic;">Description of a Project</span>
      (<small>DOAP</small>) ontologies are used to build social
      networks across the web.&nbsp; &nbsp; Typical <a href=
      "http://en.wikipedia.org/wiki/List_of_social_networking_websites">
      social network portals</a> do not provide links to other
      sites, nor expose their data in a standard form.
    </p>
    <p>
      LiveJournal and Opera Community are two portal web sites
      which do in fact publish their data in <small>RDF</small> on
      the web. &nbsp; (Plaxo has a trail scheme, and I'm not sure
      whether&nbsp;they support <span style=
      "font-style: italic;">knows</span> links). This means that I
      can write in my <small>FOAF</small> file that I know
      H&aring;kon Lie by using his <small>URI</small> in the Opera
      Community data, and a person or machine browsing that data
      can then follow that link and find all his friends.
      <i>[Update:]</i>
      Also, the Opera Community site allows you to register
      the RDF URI for yourelf on another site.  This means
      that public data about you from different sites can be linked
      together into one web, and a person or machine starting with
      your Opera identity can find the others.


      <!--
      &nbsp;
      Well, all of his friends? &nbsp;Not really: &nbsp;only his
      friends who are in the Opera Community. &nbsp;The system
      doesn't yet him store the <small>URI</small>s of people on
      different systems. So while the social network is open to
      incoming links, and while it is internally browseable, it
      doesn't make outgoing links.
      -->
    </p>
    <p>
      The fourth rule, to make links elsewhere,&nbsp; is necessary
      to connect the data we have into a web, a serious, unbounded
      web in which one can find al kinds of things, &nbsp;just as
      on the hypertext web we have managed to build.
    </p>
    <p>
      In hypertext web sites it is considered generally rather bad
      etiquette not to link to related external material. &nbsp;The
      value of your own information is very much a function of what
      it links to, as well as the inherent value of the information
      within the web page. &nbsp;So it is also in the Semantic Web.
    </p>
    <p>
      So let's look at the ways of linking data, starting with the
      simplest way of making a link.
    </p>
    <h3>
      Basic web look-up
    </h3>
    <p>
      The simplest way to make linked data is to use, in one file,
      a <small>URI</small> which points into another.
    </p>
    <p>
      When you write an <small>RDF</small> file, &nbsp; say
      &lt;http://example.org/smith&gt;, then you can use local
      identifiers within the file, say &nbsp;#albert, #brian and
      #carol. &nbsp;In N3 you might say
    </p>
    <pre>
&lt;#albert&gt;  fam:child &lt;#brian&gt;, &lt;#carol&gt;.
</pre>
    <p>
      or in <small>RDF/XML</small>
    </p>
    <pre>
&lt;rdf:Description about="#albert"<br /> &lt;fam:child rdf:Resource="#brian"&gt;<br />  &lt;fam:child rdf:Resource="#carol"&gt;<br />&lt;/rdf:Description&gt;
</pre>
    <p>
      The <small>WWW</small> architecture now gives a global
      identifier &nbsp;"http://example.org/smith#albert" to Albert.
      &nbsp;This is a valuable thing to do, as anyone on the planet
      can now use that global identifier to refer to Albert and
      give more information.&nbsp;
    </p>
    <p>
      For example, in the
      document&nbsp;&lt;http://example.org/jones&gt; someone might
      write:
    </p>
    <pre>
&lt;#denise&gt;  fam:child &lt;#edwin&gt;, &lt;smith#carol&gt;.
</pre>
    <p>
      or in <small>RDF/XML</small>
    </p>
    <pre>
&lt;rdf:Description about="#denise"<br /> &lt;fam:child rdf:Resource="#edwin"&gt;<br />  &lt;fam:child rdf:Resource="http://example.org/smith#carol"&gt;<br />&lt;/rdf:Description&gt;
</pre>
    <p>
      <br />
      Clearly it is reasonable for anyone who comes across the
      identifier 'http://example.org/smith#carol" to:
    </p>
    <ol>
      <li>Form the <small>URI</small> of the document by truncating
      before the hash
      </li>
      <li>Access the document to obtain information about #carol
      </li>
    </ol>
    <p>
      We call this dereferencing the <small>URI</small>. &nbsp;This
      is basic semantic web.&nbsp;
    </p>
    <p>
      There are several variations.
    </p>
    <h3>
      Variation: URIs without Slashes and HTTP 303
    </h3>
    <p>
      There are some circumstances in which dividing identifiers
      into documents doesn't work very well. &nbsp; There may
      logically be one global symbol per document per document, and
      there is a reluctance to include a # in the
      <small>URI</small> such as&nbsp;
    </p>
    <p>
      http://wordnet.example.net/antidisesablishmentarianism#word
    </p>Historically, the early Dublin Core and <small>FOAF</small>
    vocabularies did not have # in their URIs. &nbsp; In any event
    when <small>HTTP</small> <small>URI</small>s without hashes are
    used for abstract concepts, and there is a document that
    carries information about them, then:<br />
    <ol>
      <li>An <small>HTTP</small> <small>GET</small>&nbsp; request
      on the <small>URI</small> of the concept returns
        <span style="font-family: monospace;">303 See Also</span>
        and gives in the Location: header, the <small>URI</small>
        of the document.&nbsp;&nbsp;
      </li>
      <li>The document is retrieved as normal
      </li>
    </ol>
    <p>
      This method has the advantage that <small>URI</small>s can be
      made up of all forms. &nbsp;It has the disadvantage that an
      <small>HTTP</small> request mBrowse-ableust be made for every
      single one. &nbsp;In the case of Dublin Core, for example,
      dc:title and dc:creator etc are in fact served by the same
      ontology document, but &nbsp;one does not know until they
      have each been fetched and returned HTTP redirections.
    </p>
    <h3>
      Variation: FOAF and rdfs:seeAlso
    </h3>
    <p>
      The <a href=
      "http://foaf-project.org/">Friend-Of-A-Friend</a>&nbsp;convention
      uses a form of data link, but&nbsp; not using either of the
      two forms mentioned above. &nbsp;To refer to another person
      in a <small>FOAF</small> file, the convention was to give two
      properties, one pointing to the document they are described
      in, and the other for identifying them within that document.
    </p>
    <pre>
&lt;#i&gt;  foaf:knows  [<br />       foaf:mbox &lt;mailto:joe@example.com&gt;;<br />        rdfs:seeAlso &lt;http://example.com/foaf/joe&gt; ].
</pre>
    <p>
      Read, "I know that which has email&nbsp; joe@example.com and
      about which more information is in
      &lt;http://example.com/foafjoe&gt;".
    </p>
    <p>
      In fact, for privacy, often people don't put their email
      addresses on the web directly, but in fact put a one-way hash
      (<small>SHA-1</small>) of their email address and give that.
      This clever trick allows people who know their email address
      already to work out that it is the same person, without
      giving the email away to others.
    </p>
    <pre>
&lt;#i&gt;  foaf:knows  [<br />       foaf:mbox_sha1sum "2738167846123764823647";  # @@ dummy<br />  rdfs:seeAslo &lt;http://example.com/foaf/joe&gt; ].
</pre>
    <p>
      This linking system was very successful, forming a
      &nbsp;growing social network, and dominating, in 2006, the
      linked data available on the web.
    </p>
    <p>
      However, the system has the snag that it does not give
      <small>URI</small>s to people, and so basic links to them
      cannot be made.
    </p>
    <p>
      I&nbsp; recommend (e.g in weblogs on <a href=
      "http://dig.csail.mit.edu/breadcrumbs/node/62">Links on the
      Semantic Web</a> , <a href=
      "http://dig.csail.mit.edu/breadcrumbs/node/71">Give yourself
      a URI</a>, and and <a href=
      "http://dig.csail.mit.edu/breadcrumbs/node/72">Backward and
      Forward links in RDF just as important</a>) that those making
      a <small>FOAF</small> file give themselves a
      <small>URI</small> as well as using the <small>FOAF</small>
      convention.&nbsp; &nbsp; &nbsp;Similarly, when you refer to a
      <small>FOAF</small> &nbsp;file which gives &nbsp;a
      <small>URI</small> to a person, use it in your reference to
      that person, so that clients which just use
      <small>URI</small>s and don't know about the
      <small>FOAF</small> convention can follow the link.
    </p>
    <h2>
      Browsable graphs
    </h2>So now we have looked at ways of making a link, let's look
    at the &nbsp;choices of when to make a link.<br />
    <p>
      One important pattern is a set of data which you can explore
      as you go link by link by fetching data. &nbsp; Whenever one
      looks up the URI for a node in the RDF graph, the server
      returns information about the arcs out of that node, and the
      arcs in. &nbsp;In other words, it returns any RDF statements
      in which the term appears as either subject or object.
    </p>
    <p>
      Formally,&nbsp; call a&nbsp;graph G <span style=
      "font-style: italic;">browsable</span> if, for &nbsp;the URI
      of any node in G, if I look up that URI I will be returned
      information which describes the node, where describing a node
      means:
    </p>
    <ol>
      <li>Returning all statements where the node is a subject or
      object; and
      </li>
      <li>Describing all blank nodes attached to the node by one
      arc.
      </li>
    </ol><br />
    <p class="detail">
      (The subgraph returned has been referred to as "minimum
      Spanning Graph (MSG [@@ref] ) or &nbsp;RDF molecule [@@ref],
      depending on whether nodes are considered identified if they
      can be expressed as a path of function, or reverse inverse
      functional properties. A concise bounded description, which
      only follows links from subject to object, &nbsp;does not
      work.)
    </p>
    <p>
      In practice, when data is stored in two documents, this means
      that any <small>RDF</small> statements which relate things in
      the two files must be repeated in each. &nbsp;So, for
      example, in my <small>FOAF</small> page I mention that I am a
      member of the <small>DIG</small> group, and that information
      is repeated on the <small>DIG</small> group data. Thus,
      someone starting from the concept of the group can also find
      out that I am a member. &nbsp;In fact, someone who starts off
      with my <small>URI</small> can find all the people who are in
      the same group.
    </p>
    <h3>
      Limitations on browseable data
    </h3>
    <p>
      So statements which relate things in the two documents must
      be repeated in each. This clearly is against the first rule
      of data storage: don't store the same data in two different
      places: you will have problems keeping it consistent.
      &nbsp;This is indeed an issue with browsable data. &nbsp; A
      set of &nbsp;of completely browsable data with links in both
      directions has to be completely consistent, and that takes
      coordination, especially if different authors or different
      programs are involved.
    </p>
    <p>
      We can have completely browsable data, however, where it is
      automatically generated. &nbsp;The <a href=
      "http://dig.csail.mit.edu/2006/dbview/dbview.py">dbview</a>
      &nbsp;server, for example,&nbsp; provides a browsable virtual
      &nbsp;documents containing the data from any arbitrary
      relational database.
    </p>
    <p>
      When we have a data from multiple sources, then we have
      compromises. &nbsp;These are often settled by common sense,
      asking the question,
    </p>
    <blockquote>
      <p>
        "If someone has the URI of that thing, what relationships
        to what other objects is it useful to know about?"
      </p>
    </blockquote>
    <p>
      Sometimes, social questions &nbsp;determine the answer.
      &nbsp;I have links in my <small>FOAF</small> file that I know
      various people. &nbsp;They don't generally repeat that
      information in their <small>FOAF</small> files. Someone may
      say that they know me, which is an assertion which, in the
      <small>FOAF</small> convention, is theirs to assert, and the
      reader's to trust or not. &nbsp;
    </p>
    <p>
      Other times, the number of arcs makes it impractical. &nbsp;
      A <small>GPS</small> track gives thousands of times at which
      my latitude, longitude are known. Every person loading my
      <small>FOAF</small> file can expect to get my business card
      information, but not all those trackpoints. It is reasonable
      to have a pointer from the track (or even each point) to the
      person whose position is represented, but not the other
      way.&nbsp;
    </p>
    <p>
      One pattern is to have links of a certain property in a
      separate document. &nbsp; A person's homepage doesn't list
      all their publications, but instead puts a link to it a
      separate document listing them. &nbsp;There is an
      understanding that&nbsp;<span style=
      "font-family: monospace;">foaf:made</span> gives a work of
      some sort, but <span style=
      "font-family: monospace;">foaf:pubs</span> points to a
      document giving a list of works. &nbsp;Thus, someone
      searching for something <span style=
      "font-family: monospace;">foaf:made</span> link would do well
      to follow a <span style=
      "font-family: monospace;">foaf:pubs</span> link.&nbsp; It
      might be useful to formalize the notion with a statement like
    </p>
    <pre>
foaf:made  link:listDocumentProperty foaf:pubs.
</pre>
    <p>
      in one of&nbsp;the ontologies.
    </p>
    <h3>
      Query services
    </h3>
    <p>
      Sometimes the sheer volume of data makes serving it as lots
      of files possible, but cumbersome for efficient remote
      queries over the dataset. &nbsp;In this case, it seems
      reasonable to provide a <small>SPARQL</small> query service.
      &nbsp;To make the data be effectively linked, someone who
      only has the &nbsp;<small>URI</small> of something must be
      able to find their way the <small>SPARQL</small>
      endpoint.&nbsp;
    </p>
    <p>
      Here again the <small>HTTP</small> 303 response can be used,
      to refer the enquirer to a document with metadata about
      which query service endpoints can provide what information
      about which &nbsp;classes of <small>URI</small>s.
    </p>Vocabularies for doing this have not yet been
    standardized.<br />
    <h2>Is your Linked Open Data 5 Star?</h2>
    (Added 2010).
    This year, in order to encourage people  -- especially
    government data owners -- along the road
    to good linked data, I have developped this star rating system.
    </p><p>
    Linked Data is defined above. Linked <em>Open</em> Data (LOD) is
    Linked Data which is released under an open licence, which
    does not impede its reuse for free.  Creative Commons CC-BY is an example
    open licence, as is the UK's <a href="http://www.nationalarchives.gov.uk/doc/open-government-licence/">
    Open Government Licence</a>.
    Linked Data does not of course in general have to be open -- there is a
    lot of important use of lnked data internally, and for personal and group-wide
    data. You can have 5-star Linked Data without it being open.
    However, if it claims to be Linked Open Data then it does have to be open,
    to get any star at all.
    <p></p>
    Under the star scheme, you get one (big!) star if the information
    has been made public at all, even if it is a photo of a scan of
    a fax of a table -- if it has an open licence.
    The you get more stars as you make it progressively more
    powerful, easier for people to use.
    <p>


<style>
.stars {color: gold; font-size: 18pt; text-align: right; margin-right: 20px;}
</style>
<table>
<tr>
<td class="stars">&#x2605;</td>
<td>Available on the web (whatever format) <i>but with an open licence, to be Open Data</i></td>
</tr>
<tr>
<td class="stars">&#x2605;&#x2605;</td>
<td>Available as machine-readable structured data (e.g. excel instead of image scan of a table)</td>
</tr>
<tr>
<td class="stars">&#x2605;&#x2605;&#x2605;</td>
<td> as (2) plus non-proprietary format (e.g. CSV instead of excel)</td>

</tr>
<tr>
<td class="stars">&#x2605;&#x2605;&#x2605;&#x2605;</td>
<td>All the above plus, Use open standards from W3C (RDF and SPARQL)
to identify things, so that people can point at your stuff</td>
</tr>
<tr>
<td class="stars">&#x2605;&#x2605;&#x2605;&#x2605;&#x2605;</td>
<td>All the above, plus: Link your data to other people&#8217;s data to provide context</td>
</tr>
</table>
    <p>
    How well does your data do?  You can buy <a href="http://www.cafepress.co.uk/w3c_shop.480759174">
    5 star data mugs</a>, T-shirts
    and bumper stickers from the W3C shop at cafepress: use them to
    get your colleages and fellows conference-goers thinking 5 star linked data.
    (Profits also help W3C :-).
    </p>
    <p>
    Now in 2010, people have been pressing me, for governmet data,
    to add a new requirement, and that is there should be metadata about the
    data itself, and that that metadata should be
    availble from a major catalog. Any open dataset (or even datasets which are not
    but should be open) can be regisetreed at ckan.net.
    Government datasets from the UK and US hsould be regisetred at
    data.gov.uk or data.gov respectively.
    Other copuntries I expect to develop their own registries.
    Yes, there should be metadata about your dataset.
    That may be the subject of a new note in this series.
    </p>

    <h2>
      Conclusion
    </h2><br />
    <p>
      Linked data is essential to actually connect the semantic
      web. &nbsp;It is quite easy to do with a little thought, and
      becomes second nature. &nbsp; Various common sense
      considerations determine when to make a link and when not to.
    </p>
    <p>
      The <a href=
      "http://dig.csail.mit.edu/2005/ajar/ajaw/tab">Tabulator</a>
      client (running in a suitable browser)&nbsp; allows you to
      browse linked data using the above conventions, and can be
      used to check that your linked data works.
    </p>
    <p>
      References
    </p>
    <p>
      [Ding2005] Li Ding, et. al.,&nbsp; <a href=
      "http://ebiquity.umbc.edu/paper/html/id/240/"><span style=
      "font-style: italic;">Tracking RDF Graph Provenance using RDF
      Molecules</span></a>, UMBC Tech Report TR-CS-05-06
    </p>
    <hr />
    <h2>
      Followup
    </h2>
    <p>
      2006-02 Rob Crowell adapts Dan Connolly's DBView (2004) which
      maps SQL data into linked RDF, adding backlinks.
    </p>
    <p>
      2006-09-05 Chris Bizer et al adapt <a href=
      "http://sites.wiwiss.fu-berlin.de/suhl/bizer/d2r-server/">D2R
      Server</a> to provide a linked data view of a database.
    </p>
    <p>
      2006-10-10 Chris Bizer et al produce the <a href=
      "http://sites.wiwiss.fu-berlin.de/suhl/bizer/ng4j/semwebclient/">
      Semantic Web Client Library</a>, "Technically, the library
      represents the Semantic Web as a single Jena RDF graph or
      Jena Model." The code feteches web documents as needed to
      answer queries.
    </p>
    <p>
      2007-01-15 Yves Raimond has produced a <a href=
      "http://moustaki.org/swic/">Semantic Web client for SWI
      prolog</a> wit similar functionality.
    </p>


    <p>I have a talk at the 2009 O'Reilly eGovernment 2.0 conference
    in Washington DC, talking about "Just a Bag of Chips" @@ref, and talking about the 5 star scheme.
    Following that,
    From  InkDroid blogged summary (and CSS) of my 5 star sceheme adapted here
</p>
    <hr />
    <p>
      <a href="Overview.html">Up to Design Issues</a>
    </p>
    <p>
      <a href="../People/Berners-Lee">Tim BL</a>
    </p>
  </body>
</html>