index.html 89.9 KB

Raw Blame History Permalink

<?xml version='1.0' encoding='utf-8' standalone="yes"?>
<!DOCTYPE html PUBLIC "-//W3C//DTD XHTML 1.0 Transitional//EN"
                      "http://www.w3.org/TR/xhtml1/DTD/xhtml1-transitional.dtd">

<html xml:lang="en-US" xmlns="http://www.w3.org/1999/xhtml" lang="en-US">
  <head>
    <title>Image Annotation on the Semantic Web</title>
    <meta http-equiv="Content-Type" content="text/html; charset=utf-8"/>
    <style type="text/css">

.new { color: #FF0000 }

.example {font-family: monospace; }

.figure {
         font-weight: bold;
         text-align: center; }

div.example {
    padding: 1em;
    margin: 0.1em 3.5em 0.1em 0.1em;
    background-color: #efeff5;
    border: 1px solid #cfcfcf; }

div.exampleOuter {
   margin: 0em;
   padding: 0em;
}

div.exampleInner {
   color: black;
   background-color: #efeff5;
   border-top-style: double;
   border-top-color: #d3d3d3;
   border-bottom-width: 1px;
   border-bottom-style: double;
   border-bottom-color: #d3d3d3;
   padding: 4px;
   margin: 0em;
}

div.exampleInner pre {
  margin-left: 0em;
  margin-top: 0em;
  margin-bottom: 0em;
  font-family: monospace;
}

div.c1 {text-align:center}

  </style>
  <link
   href="http://www.w3.org/StyleSheets/TR/W3C-WD.css"
   type="text/css" rel="stylesheet"/>
 </head>

 <body>
  <div class="head">
   <p>
    <a href="http://www.w3.org/">
     <img alt="W3C" src="http://www.w3.org/Icons/w3c_home"
      height="48" width="72"/>
    </a>
   </p>

   <h1>Image Annotation on the Semantic Web</h1>
   <h2>W3C Working Draft 22 March 2006</h2>

   <dl>
    <dt>This version: </dt>
    <dd>
     <a href="http://www.w3.org/TR/2006/WD-swbp-image-annotation-20060322/"
       >http://www.w3.org/TR/2006/WD-swbp-image-annotation-20060322/</a></dd>
    <dt>Latest version: </dt>
    <dd>
     <a href="http://www.w3.org/TR/swbp-image-annotation/"
       >http://www.w3.org/TR/swbp-image-annotation/</a></dd>


    <dt>Editors: </dt>
    <dd><a href="http://homepages.cwi.nl/~jrvosse/">Jacco van Ossenbruggen</a>, Center for Mathematics and Computer Science (CWI Amsterdam)</dd>
    <dd><a href="http://homepages.cwi.nl/~troncy/">Rapha&#235;l Troncy</a>, Center for Mathematics and Computer Science (CWI Amsterdam)</dd>
    <dd><a href="http://www.image.ntua.gr/~gstam/">Giorgos Stamou</a>, IVML, National Technical University of Athens</dd>
    <dd><a href="http://dl-web.man.ac.uk/~panz/">Jeff Z. Pan</a>, University of Aberdeen (Formerly University of Manchester)</dd>

    <dt>Contributors: </dt>
    <dd><a href="http://www.mindswap.org/~chris/">Christian Halaschek-Wiener</a>, University of Maryland</dd>
    <dd><a href="mailto:nsimou@image.ece.ntua.gr">Nikolaos Simou</a>, IVML, National Technical University of Athens</dd>
    <dd><a href="mailto:tzouvaras@image.ntua.gr">Vassilis Tzouvaras</a>, IVML, National Technical University of Athens</dd>

    <dt>&#160; </dt>
    <dd>Also see <a href="#acknowledgments">Acknowledgements</a>.</dd>
      </dl>

<p class="copyright"><a href="http://www.w3.org/Consortium/Legal/ipr-notice#Copyright">Copyright</a> &#169; 2006 <a href="http://www.w3.org/"><acronym title="World Wide Web Consortium">W3C</acronym></a><sup>&#174;</sup> (<a href="http://www.csail.mit.edu/"><acronym title="Massachusetts Institute of Technology">MIT</acronym></a>, <a href="http://www.ercim.org/"><acronym title="European Research Consortium for Informatics and Mathematics">ERCIM</acronym></a>, <a href="http://www.keio.ac.jp/">Keio</a>), All Rights Reserved. W3C <a href="http://www.w3.org/Consortium/Legal/ipr-notice#Legal_Disclaimer">liability</a>, <a href="http://www.w3.org/Consortium/Legal/ipr-notice#W3C_Trademarks">trademark</a> and <a href="http://www.w3.org/Consortium/Legal/copyright-documents">document use</a> rules apply.</p>

      <hr/>
    </div>

    <h2>
      <a id="abstract" name="abstract">
    Abstract
      </a>
    </h2>

    <p>
      Many applications that involve multimedia content make use of
      some form of metadata that describe this content.  The goals of
      this document are (i) to explain what the advantages are of
      using Semantic Web languages and technologies for the creation,
      storage, manipulation, interchange and processing of image
      metadata, and (ii) to provide guidelines for doing so. The document
      gives a number of use cases that illustrate ways to exploit
      Semantic Web technologies for image annotation, an overview of RDF
      and OWL vocabularies developed for this task and an overview of
      relevant tools.
    </p>

    <h2>
      <a id="status" name="status">Status of this document</a>
    </h2>

<p><em>This section describes the status of this document at the time of its publication. Other documents may supersede this document. A list of current W3C publications and the latest revision of this technical report can be found in the <a href="http://www.w3.org/TR/">W3C technical reports index</a> at http://www.w3.org/TR/.</em></p>

    <p>
      This document is a First Public Working Draft
    produced by the <a
    href="http://www.w3.org/2001/sw/BestPractices/MM/">Multimedia
    Annotation in the Semantic Web Task Force</a> of the <a
    href="http://www.w3.org/2001/sw/BestPractices/">W3C Semantic
    Web Best Practices &amp; Deployment Working Group</a>.  This group
    is part of the <a href="http://www.w3.org/2001/sw/">W3C
    Semantic Web</a> Activity.</p>

    <p>Discussion of this document
    is invited on the public mailing list <a
    href="mailto:public-swbp-wg@w3.org">public-swbp-wg@w3.org</a>
    (<a
    href="http://lists.w3.org/Archives/Public/public-swbp-wg/">public
    archives</a>). Please start the subject line of the message with
    the text "comments: [MM]".</p>

    <p>After reviewing comments and further feedback, the Working Group
    may publish new versions of this document or may advance the
    document to Working Group Note.</p>

<p> This document was produced by a group operating under the <a href="http://www.w3.org/Consortium/Patent-Policy-20040205/">5 February 2004 W3C Patent Policy</a>. The group does not expect this document to become a W3C Recommendation. This document is informative only. W3C maintains a <a rel="disclosure" href="http://www.w3.org/2004/01/pp-impl/35495/status">public list of any patent disclosures</a> made in connection with the deliverables of the group; that page also includes instructions for disclosing a patent. An individual who has actual knowledge of a patent which the individual believes contains <a href="http://www.w3.org/Consortium/Patent-Policy-20040205/#def-essential">Essential Claim(s)</a> must disclose the information in accordance with <a href="http://www.w3.org/Consortium/Patent-Policy-20040205/#sec-Disclosure">section 6 of the W3C Patent Policy</a>. </p>

    <p>Publication as a
    Working Draft does not imply endorsement by the W3C
    Membership. This is a draft document and may be updated,
    replaced or obsoleted by other documents at any time. It is
    inappropriate to cite this document as other than work in
    progress.
    </p>

<hr />

    <h2>
      <a id="roadmap" name="roadmap">
    Document Roadmap
      </a>
    </h2>
    <p>
      After reading this document, readers may
      turn to separate documents discussing individual image
      annotation <a
      href="http://www.w3.org/2001/sw/BestPractices/MM/resources/Vocabularies.html">vocabularies</a>,
      <a
      href="http://www.w3.org/2001/sw/BestPractices/MM/resources/Tools.html">tools</a>,
      and other <a
      href="http://www.w3.org/2001/sw/BestPractices/MM/resources/Resources.html">relevant
      resources</a>.
      Note: many current approaches to image annotation are not based on Semantic
      Web languages. Interoperability between these technologies and
      RDF and OWL-based approaches is <em>not</em> the topic of this document.
    </p>

    <h2>
      <a id="targetaudience" name="targetaudience">
    Target Audience
      </a>
    </h2>

    <p>
      This document is target at everybody with an interest in image
      annotation, ranging from non-professional end-users that are
      annotating their personal digital photos to professionals
      working with digital pictures in image and video banks,
      audiovisual archives, museums, libraries, media production and
      broadcast industry, etc.
    </p>

    <h2>
      <a id="objectives" name="objectives">
    Objectives
      </a>
    </h2>

    <ul>
      <li>
    To illustrate the benefits of using semantic technologies in image annotations.</li>

      <li>
    To provide guidelines for applying semantic technologies in this area.</li>

      <li>
    To collect currently used vocabularies for Semantic Web-based
    image annotations.</li>

      <li>
    To provide use cases with examples of Semantic Web-based
    annotations.</li>
    </ul>

  <div class="toc">
   <h2 class="notoc">
    <a id="contents" name="contents">Table of Contents</a>
   </h2>

   <ul id="toc" class="toc">
    <li class="tocline"><a href="#introduction"><b>1. Introduction</b></a>
     <ul class="toc">
      <li class="tocline"><a href="#annot_intro">1.1 Image Annotation Issues</a></li>
      <li class="tocline"><a href="#semweb_intro">1.2 Semantic Web Basics</a></li>
     </ul>
    </li>
    <li class="tocline"><a href="#use_cases"><b>2. Use Cases</b></a>
     <ul class="toc">
      <li class="tocline"><a href="#world_images">2.1. World Images</a></li>
      <li class="tocline"><a href="#culture_images">2.2. Culture Images</a></li>
      <li class="tocline"><a href="#media_images">2.3. Media</a></li>
      <li class="tocline"><a href="#scientific_images">2.4. Scientific Images</a></li>
     </ul>
    </li>
    <li class="tocline"><a href="#vocabularies"><b>3. Vocabularies for Image Annotation</b></a></li>
    <li class="tocline"><a href="#tools"><b>4. Available Tools for Semantic Image Annotation</b></a></li>
    <li class="tocline"><a href="#examples"><b>5. Example Solutions to the Use Cases</b></a>
     <ul class="toc">
      <li class="tocline"><a href="#solution_personal">5.1. Use Case: Management of Personal Digital Photo Collections</a></li>
      <li class="tocline"><a href="#solution_culture">5.2. Use Case: Cultural Heritage</a></li>
      <li class="tocline"><a href="#solution_TVarchive">5.3. Use Case: Television News Archive</a></li>
      <li class="tocline"><a href="#solution_NASA">5.4. Use Case: large-scale image collections at NASA</a></li>
     </ul>
    </li>
    <li class="tocline"><a href="#conclusions"><b>6. Conclusions</b></a></li>
    <li class="tocline"><a href="#references"><b>References</b></a></li>
    <li class="tocline"><a href="#acknowledgments"><b>Acknowledgments</b></a></li>
   </ul>
  </div>

    <h2>
      <a name="introduction">
    1. Introduction
      </a>
    </h2>

    <p>
      The need for annotating digital image data is recognized in a
      wide variety of different applications, covering both
      professional and personal usage of image data. At the time of writing,
      most work done in this area does not use semantic-based technologies partly
      because of the differences between the multimedia and the web communities
      and their underlying standardization organizations.
      This document explains the advantages of
      using Semantic Web languages and technologies for image
      annotations and provides guidelines for doing so. It is
      organized around a number of representative use cases, and a
      description of Semantic Web vocabularies and tools that could be
      used to help accomplish the task mentioned in the uses cases.
      The remainder of this introductory section first gives an
      overview of image annotation in general, followed by a short
      description of the key Semantic Web concepts that are relevant
      for image annotation.
    </p>

    <h3>
      <a name="annot_intro">
    1.1 Image Annotation Issues
      </a>
    </h3>

    <p>
      Annotating images on a small scale for personal usage can be
      relatively simple. The reader should be warned, however, that
      large scale, industrial strength image annotation is notoriously
      complex. Trade offs along several dimensions make the professional
      multimedia annotations difficult:
    </p>

    <ol>
      <li>
<p>
  <em>Production versus post-production annotation</em>
</p>
<p>
  A general rule is that it is much easier to annotate earlier rather than later.
  Typically, most of the information that is needed for
  making the annotations is available during production
  time. Examples include time and date, lens settings and
  other EXIF metadata added to JPEG images by most digital
  cameras at the time a picture is taken, experimental data in
  scientific and medical images, information from scripts,
  story boards and edit decision lists in creative industry,
  etc.  Indeed, maybe the single most best practice in image
  annotation is that in general, adding metadata during the
  production process is much cheaper and yields higher quality
  annotations than adding metadata in a later stage (such as
  by automatic analysis of the digital artifact or by manual
  post-production data).
</p>
      </li>
      <li>
    <p>
      <em>
        Generic vs task-specific annotation
      </em>
    </p>
    <p>
      Annotating images without having a specific goal or task in
      mind is often not cost effective: after the target
      application has been developed, it turns out that images
      have been annotated using the wrong type of information, or
      on the wrong abstraction level, etc. Redoing the annotations
      is then an unavoidable, but costly solution.  On the other
      hand, annotating with <em>only</em> the target application
      in mind may also not be cost effective.  The annotations may
      work well with that one application, but if the same
      metadata is to be reused in the context of other
      applications, it may turn out to be too specific, and
      unsuited for reuse in a different context.  In most
      situations the range of applications in which the metadata
      will be used in the future is unknown at the time of
      annotation.  When lacking a crystal ball, the best the
      annotator can do in practice is use an approach that is
      sufficiently specific for the application under
      development, while avoiding unnecessary application-specific
      assumptions as much as possible.
    </p>
      </li>
      <li>
    <p>
      <em>
        Manual versus automatic annotation and the "Semantic Gap"
      </em>
    </p>
    <p>
      In general, manual annotation can provide image descriptions
      at the right level of abstraction. It is, however, time
      consuming and thus expensive. In addition, it proves to be
      highly subjective: different human annotators tend to "see"
      different things in the same image. On the other hand,
      annotation based on automatic feature extraction is
      relatively fast and cheap, and can be more systematic. It tends
      to result, however, in image descriptions that are too low
      level for many applications. The difference between the low
      level feature descriptions provided by image analysis tools
      and the high level content descriptions required by the
      applications is often referred to, in the literature, as the
      <em>Semantic Gap</em>. In the remainder, we will discuss use
      cases, vocabularies and tools for both manual and automatic
      image annotation.
    </p>
      </li>

      <li>
    <p>
      <em>
        Different types of metadata
      </em>
    </p>
    <p>
      While various classifications of metadata have been described in
      the literature, every annotator should at least be aware of the
      difference between annotations describing properties of the
      image itself, and those describing the subject matter of the
      image, that is, the properties of the objects, persons or
      concepts depicted by the image.  In the first category, typical
      annotations provide information about title, creator,
      resolution, image format, image size, copyright, year of
      publication, etc.  Many applications use a common, predefined
      and relatively small vocabulary defining such properties.
      Examples include the <a href="#DublinCore">Dublin Core</a> and
      <a href="#VraCore">VRA Core</a> vocabularies. The second
      category describes what is depicted by the image, which can vary
      wildly with the type of image at hand.  In many applications, it
      is also useful to distinguish between objective observations
      ('the person in the white shirt moves his arm from left to
      right') versus subjective interpretations ('the person seems to
      perform a martial arts exercise).  As a result, one sees a large
      variation in vocabularies used for this purpose. Typical
      examples vary from domain-specific vocabularies (for example,
      with terms that are very specific for astronomy images, or sport
      images, etc) to domain-independent ones (for example, a
      vocabulary with terms that are sufficiently generic to describe
      any news photo). In addition, vocabularies tend to differ in
      size, granularity, formality etc.  In the remainder, we discuss
      the above metadata categories.  Note that in the first type it
      is not uncommon that a vocabulary only defines the properties
      and defers the definitions of the values of those properties to
      another vocabulary.  This is true, for example, for both Dublin
      Core and VRA Core.  This means that, typically, in order to
      annotate a single image one needs terms from multiple
      vocabularies.
    </p>
      </li>

      <li>
    <p>
      <em>
        Lack of Syntactic and Semantic Interoperability
      </em>
    </p>
    <p>
      Many different file formats and tools for image annotations
      are currently in use.  Reusing metadata developed for one set
      of tools in another tool is often hindered by a lack of
      interoperability. First, different tools use different file
      formats, so tool A may not be able to read in the metadata
      provided by tool B (syntax-level interoperability).  Solving
      the problem is relatively easy if the inner structure of
      both file formats are known by developing a conversion tool.
      Second, tool A may assign a different meaning to the same
      annotation as tool B does (semantic
      interoperability). Solving this problem is much harder, and a first
      step to provide a solution is to require that the
      vocabulary used be explicitly defined for both tools.
    </p>
      </li>
    </ol>

    <h3>
      <a name="semweb_intro">
    1.2 Semantic Web Basics
      </a>
    </h3>

<p>
This section briefly describe the role of Semantic Web technologies in image annotations. The aim of the Semantic Web is to augment the
existing Web so that resources (Web pages, images etc.) are more easily interpreted by programs (or "intelligent agents"). The idea is to
associate Web resources with semantic categories which describe the contents and/or functionalities of Web resources.
</p>

<p>
Annotations alone do not establish the semantics of what is being marked-up. One way generally followed
to introduce semantics to annotations is to get an agreement to carefully define what a set of
concepts are and what terms have to be used for them.
</p>

<p>
This agreement can be only "informal", that is, relies on natural language for defining the meaning
of a set of information properties. For example, the <a href="#DublinCore">Dublin Core</a> Metadata
Element Set provides 15 "core" information properties, such as "Title", "Creator", "Date", with
descriptive semantic definitions (in natural language). One can use these information properties in,
e.g., <a href="#RDF">RDF</a> or META tags of HTML.
</p>

<p>
For example, the following RDF/XML code represents the statements "there is an image <tt>Ganesh.jpg</tt> created by <tt>Jeff Z. Pan</tt> and whose
title is <tt>An image about the Elephant Ganesh</tt>. The first four lines define the <a href="#XML-NS">XML namespaces</a> used in this
description. A good starting point for having more information on RDF is the <a href="#RDF-Primer">RDF Primer</a>.
</p>

<table width="90%" border="0" cellspacing="20" bgcolor="#EEEEEE">
  <tr>
    <td>
        <pre>
&lt;rdf:RDF xml:base=&quot;http://example.org/&quot;
         xmlns=&quot;http://example.org/&quot;
         xmlns:dc=&quot;http://purl.org/dc/elements/1.1/&quot;
         xmlns:rdf=&quot;http://www.w3.org/1999/02/22-rdf-syntax-ns#&quot;&gt;

  &lt;rdf:Description rdf:about=&quot;Ganesh.jpg&quot;/&gt;
    &lt;dc:title&gt;An image about the Elephant Ganesh&lt;/dc:title&gt;
    &lt;dc:creator&gt;Jeff Z. Pan&lt;/dc:creator&gt;
  &lt;/rdf:Description&gt;
&lt;/rdf:RDF&gt;</pre>
    </td>
  </tr>
</table>

<p>
A complementary approach is to also use ontologies to specify formally the meaning of Web resources
and thus get a "formal" agreement. <em>Ontology</em> is a term borrowed from philosophy
that refers to the science of describing the kinds of entities in the world and how they are related.
In computer science, ontology is, in general, a representation of a shared conceptualization of a
specific domain. It provides a shared and common <em>vocabulary</em>, including important concepts,
properties, their definitions and <em>constraints</em>, sometimes referred to as background assumptions
regarding the intended meaning of the vocabulary, used in a domain that can be communicated between
people and heterogeneous, distributed application systems. The (formal) ontology approach, though more
difficult to develop, is more powerful than the informal-only agreement approach because users can
thoroughly define the vocabulary using axioms expressed in a logic language and machine can use this
formal meaning for reasoning, completing and validating the annotations. Ideally, the concepts and
properties of an ontology should have both formal definitions and natural language descriptions to be
unambiguously used by humans and software applications.
</p>

<p>
There exists a standard Semantic Web Ontology Language <a href="#OWL">OWL</a>, which is a W3C
recommendation. We provide below an example of this language in its RDF/XML syntax.
Given that there exists a <tt>Image</tt> class and a <tt>hasSize</tt> property in an ontology, one can
use the following OWL statements to define a new OWL class called <tt>BigImage</tt> as the set of all
members of the class <tt>Image</tt> such that the size of the image is equal to <tt>Big</tt>.
For more information, the <a href="#OWL-Guide">OWL Guide</a> provides a good overview of the OWL
language.
</p>

<table width="90%" border="0" cellspacing="20" bgcolor="#EEEEEE">
    <tr>
     <td>
      <pre>
&lt;rdf:RDF xml:base=&quot;http://example.org/&quot;
         xmlns=&quot;http://example.org/&quot;
         xmlns:rdf=&quot;http://www.w3.org/1999/02/22-rdf-syntax-ns#&quot;&gt;
         xmlns:rdfs=&quot;http://www.w3.org/2000/01/rdf-schema##&quot;&gt;
         xmlns:owl=&quot;http://www.w3.org/2002/07/owl#&quot;

  &lt;owl:Class rdf:about=&quot;BigImage&quot;/&gt;
    &lt;owl:intersectionOf rdf:parseType=&quot;Collection&quot;&gt;
      &lt;owl:Class rdf:about=&quot;#Image&quot;&gt;
      &lt;owl:Restriction&gt;
        &lt;owl:onProperty rdf:resource=&quot;#hasSize&quot;&gt;
        &lt;owl:cardinality rdf:datatype=&quot;&amp;xsd;nonNegativeInteger&quot;&gt;1&lt;/owl:cardinality&gt;
        &lt;owl:allValueFrom rdf:resource=&quot;#Big&quot;&gt;
      &lt;/owl:Restriction&gt;
    &lt;/owl:intersectionOf&gt;
  &lt;/owl:Class&gt;
&lt;/rdf:RDF&gt;</pre>
     </td>
    </tr>
   </table>

<p>
The next section presents some representative use cases that highlight some requirements for image
annotation tools, vocabularies, and practices.
</p>

    <h2>
      <a name="use_cases">
    2. Use Cases
      </a>
    </h2>
    <p>
      Image annotation is relevant in a wide range of domains,
      organizations and applications; it cannot be covered in a
      single document such as this. This document, instead, describes a number of use cases
        that are intended as a representative set of
      examples. These use cases will be used later to discuss the
      vocabularies and tools that are relevant for image annotation on
      the Semantic Web. Example scenarios are given in <a
      href="#examples">Section 5</a>.
    </p>

    <p>
      The use cases are organized in four categories, which reflect
      either the topics depicted by the images or their usage community.
      These criteria often determine the tools and vocabularies used in the annotation process.
    </p>

    <h3>
      <a name="world_images">
    2.1 World Images
      </a>
    </h3>
    <p>
      This section provides two use cases with images that could
      potentially depict any subject: management of a personal photo
      collection and that of a news press photo bank. The other use
      cases will focus on images from a specific domain.
    </p>

    <h4>
      <a name="photo_collection">
      Use case: Management of Personal Digital Photo Collections
      </a>
    </h4>

    <p>
Many personal users have thousands of digital photos from vacations, parties, traveling,
conferences, everyday life etc. Typically, the photos are stored on personal computer hard drives
in a simple directory structure without any metadata. The user wants generally to easily access
this content, view it, use it in his homepage, create presentations, make part of it
accessible for other people or even sell part of it to image banks. Too often, however, the only
way for this content to be accessed is by browsing the directories, their name providing usually
the date and the description with one or two words of the original event captured by
the specific photos. Obviously, this access becomes more and more difficult as the number of
photos increases and the content becomes quickly unused in practice.
More sophisticated users leverage simple photo organizing tools allowing them to provide
keyword metadata, possibly along with a simple taxonomy of categories. This is a first step
towards a semantically-enabled solution. <a href="#solution_personal">Section
5.1</a> provides an example scenario for this use case using Semantic Web technologies.
    </p>

    <h4>
      <a name="photo_bank">
      Use case: Press Photo Bank
      </a>
    </h4>
    <p>
<!--
      TO DO: IPTC / News / Sport / Entertainment. e.g Corbis, Associated Press, Reuters
-->
    </p>

    <h3>
      <a name="culture_images">
    2.2 Culture Images
      </a>
    </h3>
    <p>
      This section contains a single use case from the cultural
      heritage domain. This domain is characterized by a long
      tradition in describing images, with many standardized methods
      and vocabularies.
    </p>

    <h4>
      <a name="cultural_heritage">
      Use case: Cultural Heritage
      </a>
    </h4>
    <p>
      Let us imagine that a museum in fine arts has asked a specialized company to produce
      high resolution digital scans of the most important art works of
      their collections. The museum's quality assurance requires the
      possibility to track when, where and by whom every scan was
      made, with what equipment, etc. The museum's internal IT
      department, maintaining the underlying image database, needs the
      size, resolution and format of every resulting image. It also
      needs to know the repository ID of the original work of art. The
      company developing the museum's website additionally requires
      copyright information (that varies for every scan, depending on
      the age of the original work of art and the collection it
      originates from). It also want to give the users of the website
      access to the collection, not only based on the titles of the
      paintings and names of their painters, but also based on the
      topics depicted ('sun sets'), genre ('self portraits'), style
      ('post-impressionism'), period ('fin de si&#232;cle'), region ('west
      European'). <a href="#solution_culture">Section 5.2</a>
      shows how all these requirements can be fulfilled using Semantic Web technologies.
    </p>

    <h3>
      <a name="media_images">
    2.3 Media
      </a>
    </h3>
    <p>
      The use case developed in this section is mainly targeted at media
      professionals, and less to the general public. Typical requests
      are characterized by very detailed queries, not only about the
      content of images, but also about the media specific details such as
      camera angle, lens settings etc.
    </p>

    <h4>
      <a name="television_archive">
      Use case: Television Archive
      </a>
    </h4>

    <p>
      Audiovisual archive centers are used to manage very large
      multimedia databases. For instance, INA, the French Audiovisual
      National Institute, has been archiving TV documents for 50 years
      and radio documents for 65 years and stores more than 1 million
      hours of broadcast programs. The images and sound archives kept
      at INA are either intended for professional use (journalists,
      film directors, producers, audiovisual and multimedia
      programmers and publishers, in France and worldwide) or
      communicated for research purposes (for a public of students,
      research workers, teachers and writers). In order to allow an
      efficient access to the data stored, most of the parts of these
      video documents are described and indexed by their content. The
      global multimedia information system should then be fine-grain
      enough detailed to support some very complex and precise
      queries. For example, a journalist or a film director client
      might ask for an excerpt of a previously broadcasted program
      showing the first goal of a given football player in its
      national team, scored with its head. The query could
      additionally contain some more technical requirements such that
      the goal action should be available according to both the front
      camera view and the reverse angle camera view. Finally, the
      client might or might not remember some general information
      about this football game, such that the date, the place and the
      final score. <a href="#solution_TVarchive">Section 5.3</a>
      gives a possible solution for this use case using Semantic Web technologies.
    </p>

    <h3>
      <a name="scientific_images">
    2.4 Scientific Images
      </a>
    </h3>
    <p>
      This section presents two use cases from the scientific domain.
      Typically here, images are annotated using large and complex
      ontologies.
    </p>

    <h4>
      <a name="large_collection">
      Use Case: Large-scale Image Collections at NASA
      </a>
    </h4>

    <p>
      Many organizations maintain extremely large-scale image
      collections. The National Aeronautics and Space Administration
      (NASA), for example,   has hundreds of thousands of
      images, stored in different formats, levels of availability and
      resolution, and with associated descriptive information at
      various levels of detail and formality. Such an organization
      also generates thousands of images on an ongoing basis that are
      collected and cataloged. Thus, a mechanism is needed to catalog
      all the different types of image content across various
      domains. Information about both the image itself (e.g., its
      creation date, dpi, source) and about the specific content of
      the image is required. Additionally, the associated metadata
      must be maintainable and extensible so that associated
      relationships between images and data can evolve
      cumulatively. Lastly, management functionality should provide
      mechanisms flexible enough to enforce restriction based on
      content type, ownership, authorization, etc.
      <a href="#solution_NASA">Section 5.4</a>
      gives an example solution for this use case.
    </p>

    <h4>
      <a name="medical_images">
      Use Case: Bio-Medical Images
      </a>
    </h4>

    <p>
    </p>

    <h2>
      <a name="vocabularies">
    3. Vocabularies for Image Annotation
      </a>
    </h2>

    <p>
      Choosing which vocabularies to use for annotating image is a key
      decision in an annotation project.  Typically, one needs more
      than a single vocabulary to cover the different relevant aspects
      of the images.  A separate document named
      <a href="http://www.w3.org/2001/sw/BestPractices/MM/resources/Vocabularies.html">Vocabularies
      Overview</a> discusses a number of individual vocabularies that
      are relevant for images annotation.  The remainder of this
      section discusses more general issues.
    </p>

    <p>
      Many of the relevant vocabularies have been developed prior to
      the Semantic Web, and <a
      href="http://www.w3.org/2001/sw/BestPractices/MM/resources/Vocabularies.html">Vocabularies
      Overview</a> lists
      many translations of such vocabularies to RDF or OWL.  Most
      notably, the key International Standard in this area, the <a
      href="#MPEG-7">Multimedia Content Description</a> standard,
      widely known as MPEG-7, is defined using XML Schema.  At the
      time of writing, there is no commonly accepted mapping from the
      XML Schema definitions in the standard to RDF or OWL.  Several
      alternative mappings, however, have been developed so far and
      are discussed in the overview.
    </p>

    <p>
      Another relevant vocabulary is the <a href="#VraCore">VRA
      Core</a>.  Where the <a href="#DublinCore">Dublin Core</a> (DC)
      specifies a small and commonly used vocabulary for on-line
      resources in general, VRA Core defines a similar set targeted
      especially at visual resources, specializing the DC elements.
      Dublin Core and VRA Core both refer to terms in their
      vocabularies as <em>elements</em>, and both use
      <em>qualifiers</em> to refine elements in similar way. All the
      elements of VRA Core have either direct mappings to comparable
      fields in Dublin Core or are defined as specializations of one
      or more DC elements.  Furthermore, both vocabularies are defined
      in a way that abstracts from implementation issues and
      underlying serialization languages. A key difference, however,
      is that for Dublin Core, there exists a commonly accepted
      mapping to RDF, along with the associated schema. At the time of
      writing, this is not the case for VRA Core, and the overview
      discusses the pros and cons of the alternative mappings.
    </p>

    <p>
      Many annotations on the Semantic Web are about an entire
      resource. For example, a <tt>&lt;dc:title&gt;</tt> property
      applies to the entire document. For images and other multimedia
      documents, one often needs to annotate a specific part of a
      resource (for example, a region in an image). Sharing the
      metadata dealing with the localization of some specific part of
      multimedia content is important since it allows to have multiple
      annotations (potentially from multiple users) referring to the
      same content.
    </p>

<!--
    <p>
      [TO DO: Discuss and give examples of two possible solutions:]
    </p>
-->

    <ol>
      <li>
    Ideally, the target image already specifies this specific
    part, using a name that is addressable in the URI fragment
    identifier (this can be done, for example, in SVG).
      </li>
      <li>
    Otherwise the region needs to be described in the metadata itself, as
    it is done in MPEG-7.
      </li>
    </ol>

<!--
  <p>
    [TO DO: Add concluding remarks / too abrupt currently]
  </p>
-->
    <h2>
      <a name="tools">
    4. Available Tools for Semantic Image Annotation
      </a>
    </h2>

    <p>
    Among the numerous tools used for image archiving and description, some of them
    may be used for semantic annotation. The aim of this section is to
    identify some key characteristics of semantic image annotation tools, so as to provide
    some guidelines for their proper use. Using these characteristics as criteria, users of these
    tools could choose the most appropriate for a specific application.
    </p>

<!--
    <p>
    TO DO: clarify here that we would like to discuss what are the abilities of the tools:
    can they handle different type of content ? do they allow fine-grained annotations ? etc ...
    Obviously, some of these characteritics are intrinsically related to the images themselves,
    or to what users need to do with them, but at the end, we should emphasize that the main
    bottleneck will be what the tools can actually do (not much currently :-( !

    MIKE: Fair point. It would be good to say something about this to avoid confusion.
    </p>
-->

    <p>
    <strong>Type of Content.</strong> A tool can annotate different type of content.
    Usually, the raw content is an image, whose format can be jpg, png, tif, etc. but there
    are also tools that can annotate videos as well.
    </p>

    <p>
    <strong>Type of Metadata.</strong> An annotation can be targeted for different use.
    Following the categorization
    provided by <a href="http://sunsite.berkeley.edu/moa2/wp-v2.html">The Making of
    America II project</a>, the metadata can be <em>descriptive</em> (for description
    and identification of information), <em>structural</em>
    (for navigation and presentation), or <em>administrative</em> (for management and
    processing). Most of the tools can be used in order to provide
    descriptive metadata and for some of them, the user can also provide structural and
    administrative information.
    </p>

    <p>
    <strong>Format of Metadata.</strong> An annotation can be expressed in different format.
    This format is important since it should ensure
    interoperability with other (semantic web) applications. MPEG-7 is often used as
    the metadata format for exchanging automatic analysis results whereas OWL and RDF are
    better appropriate in the Semantic Web world.
    </p>

    <p>
    <strong>Annotation level.</strong> Some tools give to the user the
    opportunity to annotate an image using vocabularies while others allow free text
    annotation only. When ontologies are used (in RDF or OWL format),
    the annotation level is considered to be controlled since the semantics is generally
    provided in a more formal way, whereas if they are not, the annotation level is
    considered to be free.
    </p>

    <p>
    <strong>Client-side Requirement.</strong> This characteristic refers to
    whether users can use a Web browser to access the service(s) or need to install
    a stand-alone application.
    </p>

    <p>
    <strong>License Conditions.</strong> Some of the tools are open source while some
    others are not. It is important for the user and for potential researchers and
    developers in the area of multimedia annotation to know this issue
    before choosing a particular tool.
    </p>

    <p>
    <strong>Collaborative or individual.</strong> This characteristic refers
    to the possible usage of the tool as an annotation framework for web-shared image
    databases or as an individual user multimedia content annotation tool.
    </p>

    <p>
    <strong>Granularity.</strong> Granularity specifies whether annotation is
    segment based or file based. This is an important characteristic since in some applications,
    it could be crucial to provide the structure of the image. For example, it is
    useful to provide annotations for different areas of the image, describing several cues of
    information (like a textual part or sub-images) or defining and describing different objects
    visualized in the image (e.g. people).
    </p>

    <p>
    <strong>Threaded or unthreaded.</strong> This characteristic refers to the
    ability of the tool to respond or add to a previous annotation and to stagger/structure
    the presentation of annotations to reflect this.
    </p>

    <p>
    <strong>Access control.</strong> This refers to the
    access provided for different users to the metadata. For example, it is important to
    distinguish between users that have simple access (just view) and users that have full
    access (view or change).
    </p>

    <p>
    Concluding, the appropriateness of a tool depends on the nature of annotation that the user
    requires and cannot be predetermined. A separate web page is maintained with
    <a href="http://www.w3.org/2001/sw/BestPractices/MM/resources/Tools.html">Semantic Web
    Image Annotation Tools</a>, and categorizes most of the annotation tools found in the Internet,
    according to the characteristics described above. Any comments, suggestions or new tools
    annoucements will be added to this separate document. The tools can be used for different
    types of annotations, depending on the use cases, as shown in the following section.
    </p>

    <!-- EXAMPLES -->
  <h2>
   <a name="examples">
    5. Example Solutions to the Use Cases
   </a>
  </h2>
  <p>
   This section describes possible scenarios for how
   Semantic Web technology could be used for supporting the
   use cases presented in <a href="#use_cases">Section 2</a>.
   These scenarios are provided purely as illustrative examples and do not imply
   endorsement by the W3C membership or the Semantic Web Best
   Practices and Deployment Working Group.
  </p>

  <h3>
   <a name="solution_personal">
    5.1 Use Case: Management of Personal Digital Photo Collections
   </a>
  </h3>

  <div style="float: right; width: 300px; border: 1px solid gray; padding: 1%; margin: 1%">
   <a href="Personal.jpg">
    <img style="width: 300px;"
     src="Personal.jpg"
     alt="A photo from a personal collection"/></a>
   <br/>
   A photo from a personal collection
  </div>

  <h4 id="personal_solution">Possible Semantic Web-based solution</h4>
  <p>
   The solution of the use case described in <a href="#world_images">Section 2.1</a> requires
   the use of multiple
   vocabularies. The potential domain of a photo from a personal
   digital collection is very wide, and may include sports,
   entertainment, sightseeing etc. In order to solve this use case the
   information that a user needs to know about the image has to be
   taken into account for a appropriate selection of vocabularies.  The
   use case requires creating semantic labels and associate them with
   the photo.  Semantic labels may refer to both media and content
   type annotations.  The examples cover three different approaches:
   Manual, Semi-Automatic, and Automatic. Each approach has advantages
   and disadvantages and each one requires different solutions.
  </p>


  <h5>Manual Annotation</h5>
  <p>
   Manual annotation potentially offers the most accurate information,
   but it is the most time demanding and thus expensive.  In manual
   annotation, there is typically no need for creating comprehensive
   annotations based on media features (e.g. low-level image
   characteristics also known as <em>visual descriptors</em>) since
   most users are not interested in querying the image database using
   low-level features such as shape, texture, color histograms etc.
   However, for most applications, some minimal media type information
   is needed such as the type of the image (i.e. jpeg, tiff etc.) or
   the resolution of the image. In addition, provenance information
   regarding the date created, the creator, the thematic category
   etc. is also common. VRA <a href="#VraRDF">[VRA in RDF/OWL]</a> can
   be used to describe the above information.
  </p>
  <p>
   Regarding the actual content of the image, various vocabularies can
   be used depending on the respective thematic category. The example
   shows a photo that has content from the beach holidays thematic
   category.  For this reason, a beach ontology
   and the PhotoStuff image annotator <a href="#PS">[PhotoStuff]</a>
   can be used to describe the image content.
  </p>


  <h5>Semi-Automatic Annotation</h5>
  <p>
   Semi-Automatic Annotation assists the manual annotation to extract
   higher-level, semantic labels (or vice versa).  Image analysis
   tools such as image segmentation and object recognition tools are
   based on lower level aspects of the media. As a result, a more
   extensive set of lower level media type descriptors is needed in
   this approach.  The current trend in the multimedia community is
   that the combination of image analysis tools with
   multimedia-specific and domain-specific vocabularies is shifting the
   image analysis, recognition and retrieval processes to a more
   semantic level.
  </p>

  <p>
   Using the above holiday beach example, in order to
   semi-automatically annotate the image, low-level image concepts and
   relations are needed (color, shape, texture etc.). The MPEG-7
   visual part <a href="#MPEG-7">[MPEG-7]</a> is an appropriate
   framework for the representation of such features. For this reason,
   a Visual Descriptor Ontology (VDO) <a href="#VDO">[VDO]</a> in
   combination with the beach domain ontology can be used to assign
   visual descriptors to domain concepts in order to be automatically
   recognized and thus annotated.  For example, the M-OntoMat
   Annotizer can be used to manually segment objects that have a
   semantic meaning, then extract the respective visual descriptors
   and store them as prototype instances in a predefined domain
   ontology (beach ontology). In addition, reasoning support is also
   required in the semi-automatic process.  Using reasoning tools,
   higher level concepts and events can be recognized in the image.
   Multimedia reasoning tools require spatio-temporal knowledge about
   the objects of the image (e.g. a person consists of a body, two
   hands, two legs and a head; or: the sky is over the sea etc.). An
   example of visual descriptors in association with domain concepts
   using M-OntoMat Annotizer is shown in the RDF graph below (<a
   href="#figure1">Figure 1</a>). The RDF code can be found <a
   href="PersonalContent_M_Ontomat.rdf">here</a>.
  </p>

  <div style="text-align: center">
   <img alt="An RDF Graph Describing the association of MPEG-7 Visual descriptors"
    style="width: 80%;"
    src="PersonalRDFDiagram.png"/>
   <br/>
   <a id="figure1" name="figure1">
    Figure 1: An RDF Graph Describing the association of MPEG-7 Visual
    descriptors with the domain concept "sand"</a>
  </div>


  <h5>Automatic Annotation</h5>
  <p>
   Automatic Annotation means that no user involvement is needed, and
   thus is time and cost effective. However, even with perfect image
   segmentation, person detection and object recognition, a tool will
   not recognize events such as &quot;Katerina's' holidays in
   Thailand&quot;. In the beach holiday example, more vocabularies are
   needed such as a context ontology for acquiring the context of the
   image (e.g. automatically detect that the image is about holidays
   in beaches and not in mountains) in
   order to automatically annotating the image. Also, automation is
   needed in creating the prototype instances using the VDO, the
   domain ontologies and the M-OntoMat Annotizer in order to automatically
   segment regions that may have semantic meaning and then extract and store
   the visual descriptors. Such an advanced approach is beyond the
   scope of this deliverable.
  </p>


<h4 id="personalconclusion">Conclusion and discussion</h4>
  <p>
   The example solution shows that even the manual annotation
   is non-trivial. It is difficult to provide a unified way to
   annotate personal photos. The context of the photo indicates which ontology
   must be used in the annotation process. In the above example, a beach
   domain ontology is used since the context of the photo is summer holidays. Apart from domain
   specific ontologies, media type ontologies and a photo annotation tool are required to
   complete the annotation.
  </p>
  <p>
   In the case of semi-automatic annotation, there are still many open research and technical
   issues.  Even with perfect image analysis tools, a system cannot
   recognise events that may have semantic meaning. This problem is due to the gap that exists
   between low-level image analysis tools and high-level image annotations.
  </p>

  <h3>
   <a name="solution_culture">
    5.2 Use Case: Cultural Heritage
   </a>
  </h3>

  <div style="float: right; width: 30%; border: 1px solid gray; padding: 1%; margin: 1%">
   <img style="width: 100%;;"
    src="http://www.artchive.com/artchive/m/monet/adresse.jpg"
    alt="Image of Monet's painting 'Garden at Sainte-Adresse'"/>

   Claude Monet, Garden at Sainte-Adresse.<br/>

   Image courtesy of <a href="http://www.artchive.com">Mark
    Harden</a>, used with permission.
  </div>

  <h4 id="ec_solution">Possible Semantic Web-based solution</h4>

  <p>
   Many of the requirements of the use case described in <a href="#culture_images">Section 2.2</a>
   can be met by using the vocabulary developed by the <a href="#VraCore">VRA</a> in
   combination with domain-specific vocabularies such as Getty's AAT
   and ULAN.

   In this section, we provide as an example a set of RDF annotations
   of a painting by Claude Monet, which is in English known as "Garden
   at Sainte-Adresse".  It is part of the collection of the
   Metropolitan Museum of Art in New York.  The corresponding RDF file
   is <a href="eculture-use-case.rdf">available as a
   separate document</a>.  No special annotation tools where used to
   create the annotations.  We assume that cultural heritage
   organizations that need to publish similar metadata will do so by
   exporting existing information from their collection database to
   RDF.  Below, we discuss the different annotations used in this
   file.
  </p>


  <h4 id="housekeeping">House keeping</h4>
    <p>
   The file starts as a typical RDF/XML file, by defining the XML
   version and encoding and defining entities for the RDF and VRA
   namespaces that will be used later.  Note that we use the <a
    href="#VraRDF">RDF/OWL schema of VRA Core</a> developed by Mark
   van Assem.
  </p>

  <div class="exampleInner" style="clear: both">
<pre>
&lt;?xml version='1.0' encoding='ISO-8859-1'?&gt;
&lt;!DOCTYPE rdf:RDF [
    &lt;!ENTITY rdf        "http://www.w3.org/1999/02/22-rdf-syntax-ns#"&gt;
    &lt;!ENTITY vra        "http://www.vraweb.org/vracore/vracore3#"&gt;
      </pre>
    </div>

  <h5 id="work_or_image">Work versus Image</h5>
  <p>
   The example includes annotations about two different images of the
   same painting.  An important distinction made by VRA vocabulary is
   the distinction between annotations describing a work of art itself
   and annotations describing (digital) images of that work. This
   example also uses this distinction.  In RDF, to say something about
   a resource, that resource needs to have a URI. We will thus not
   only need the URIs of the two images, but also a URI for the
   painting itself:
  </p>

  <div class="exampleInner">
<pre>
    &lt;!ENTITY image1    "http://www.metmuseum.org/Works_Of_Art/images/ep/images/ep67.241.L.jpg"&gt;
    &lt;!ENTITY image2    "http://www.artchive.com/artchive/m/monet/adresse.jpg"&gt;
    &lt;!ENTITY painting  "http://thing-described-by.org/?http://www.metmuseum.org/Works_Of_Art/images/ep/images/ep67.241.L.jpg"&gt;
]&gt;
   </pre>
  </div>

  <h5 id="uri_conventions">URI and ID conventions</h5>
  <p>
   VRA Core does not specify how works, images or annotation records
   should be identified.  For the two images, we have chosen for the
   most straightforward solution and use the URI of the image as the
   identifying URI.  We did not have, however, a similar URI that
   identifies the painting itself. We could not reuse the URI of one
   of the images.  This is not only conceptually wrong, but would also
   lead to technical errors: it would make the existing instance of
   <tt>vra:Image</tt> also an instance of the <tt>vra:Work</tt> class,
   while this is not allowed by the schema.
  </p>
  <p>
   In the example, we have decided to `mint' the URI of the painting
   by arbitrary selecting the URI of one of the images, and prefixing
   it by <tt><a href="http://thing-described-by.org/">
   http://thing-described-by.org/?</a></tt>.  This creates a new URI
   that is distinct from the image itself, but when a the browser
   resolves it, it will be redirected to the image URI by the
   <tt>thing-described-by.org</tt> web server (one could argue if the
   use of an http-based URI is actually appropriate here.  See <a
   href="#HTTP-URI">What do HTTP URIs Identify?</a> and <a
   href="#httpRange-14">[httpRange-14]</a> for more details on this
   discussion).
  </p>
  <p>
   Warning: The annotations described below also contain a
   <tt>vra:idNumber.currentRepository</tt> element, that defines the
   identifier used <em>locally</em> in the museum's repositories.
   These local identifiers should not be confused with the globally
   unique identifier that is provided by the URI.
  </p>

  <h5 id="housekeeping2">More housekeeping: starting the RDF block</h5>
  <p>
   The next line opens the RDF block, declares the namespaces using
   the XML entities defined above. Out of courtesy, it uses
   <tt>rdf:seeAlso</tt> to help agents find the VRA schema that is
   used.
  </p>

  <div class="exampleInner">
<pre>
&lt;rdf:RDF  xmlns:rdf="&amp;rdf;" xmlns:vra="&amp;vra;"
  rdf:seeAlso="http://www.w3.org/2001/sw/BestPractices/MM/vracore3.rdfs"
&gt;
   </pre>
  </div>

  <h4 id="work">Description of the work (painting)</h4>
  <p>
   The following lines describe properties of the painting itself: we
   will deal with the properties of the two images later.  First, we
   provide general information about the painting such as the title,
   its creator and the date of creation. For these properties, the VRA
   closely follows the Dublin Core conventions:
  </p>
  <div class="exampleInner">
   <pre>

  &lt;!-- Description of the painting --&gt;
  &lt;vra:Work rdf:about="&amp;painting1;"&gt;

    &lt;!-- General information --&gt;
    &lt;vra.title&gt;Jardin &#224;; Sainte-Adresse&lt;/vra.title&gt;
    &lt;vra:title.translation&gt;Garden at Sainte-Adresse&lt;/vra:title.translation&gt;
    &lt;vra:creator&gt;Monet, Claude&lt;/vra:creator&gt;                            &lt;!-- ULAN ID:500019484 --&gt;
    &lt;vra:creator.role&gt;artist&lt;/vra:creator.role&gt;                         &lt;!-- ULAN ID:31100     --&gt;
    &lt;vra:date.creation&gt;1867&lt;/vra:date.creation&gt;
   </pre>
  </div>

  <h5 id="text_or_controlled">Text fields and controlled vocabularies</h5>
  <p>
   Many values are filled with RDF Literals, of which the value is not
   further constraint by the schema.  But many of these values are
   actually terms from other controlled vocabularies, such as the
   Getty <a href="#refAAT">AAT</a>, <a href="#refULAN">ULAN</a> or a image
   type defined by <a href="#refMIME-2">MIME</a>.  Using controlled
   vocabularies solves many problems associated with free text
   annotations.  For example, ULAN recommends a spelling when an
   artist's name is used for indexing, so for the <tt>vra:creator</tt>
   field we have exactly used this spelling ("Monet, Claude"). The
   ULAN identifiers of the records describing Claude Monet and the
   "artist" class are given in XML comments above.  The use of
   controlled vocabulary can avoid confusion and the need for
   "smushing" different spellings for the same name later.
  </p>

  <p>
   However, using controlled vocabularies does not solve the problem
   of ambiguous terms.  The annotations below use three different uses
   of "oil paint", "oil paintings" and "oil painting (technique)".
   The first refers to the type of paint used on the canvas, the
   second to the type of work (e.g. the work is an oil painting, and
   not an etching) and the last to the painting technique used by
   artist.  All three terms refer to different concepts that are part
   of different branches of the AAT term hierarchy (the AAT
   identifiers of these concepts are mentioned in XML comments).
   However, the use of terms that are so similar for different
   concepts is bound to lead to confusion.  Instead, one could switch
   from using <tt>owl:datatypeProperties</tt> to using
   owl:objectProperties, and replace the literal text by a reference
   to the URI of the concept used.  For example, one could change:

   <br/><tt>&lt;vra:material.medium&gt;oil paint&lt;/vra:material.medium&gt;</tt>
   <br/>to
   <br/><tt>&lt;vra:material.medium rdf:resource="http://www.getty.edu/aat#300015050"/&gt;</tt>
   </p>

  <p>
   This approach, requires, however, that an unambiguous URI-based
   naming scheme is defined for all terms in the target vocabulary
   (and in this case, such a URI-based naming scheme does not yet
   exist for AAT terms).  Additional Semantic Web-based processing is
   also only possible once these vocabularies become available in RDF
   or OWL.
  </p>

  <div class="exampleInner">
   <pre>
    &lt;!-- Technical information --&gt;
    &lt;vra:measurements.dimensions&gt;98.1 x 129.9 cm&lt;/vra:measurements.dimensions&gt;
    &lt;vra:material.support&gt;unprimed canvas&lt;/vra:material.support&gt;        &lt;!-- AAT ID:300238097 --&gt;
    &lt;vra:material.medium&gt;oil paint&lt;/vra:material.medium&gt;                &lt;!-- AAT ID:300015050 --&gt;
    &lt;vra:type&gt;oil paintings&lt;/vra:type&gt;                                  &lt;!-- AAT ID:300033799 --&gt;
    &lt;vra.technique&gt;oil painting (technique)&lt;/vra.technique&gt;             &lt;!-- AAT ID:300178684 --&gt;

    &lt;!-- Associated style etc --&gt;
    &lt;vra:stylePeriod&gt;Impressionist&lt;/vra:stylePeriod&gt;                    &lt;!-- AAT ID:300021503 --&gt;
    &lt;vra:culture&gt;French&lt;/vra:culture&gt;                                   &lt;!-- AAT ID:300111188 --&gt;
   </pre>
  </div>

  <h5 id="subject_matter">Annotating subject matter</h5>
  <p>
   For many applications, it is useful to know what is actually
   depicted by the painting.  One could add annotations of this style
   to an arbitrary level of detail.  To keep the example simple, we
   have chosen to record only the names of the people that are
   depicted on the painting, using the <tt>vra:subject</tt> field.
   Also for simplicity, we have chosen not to annotate specific parts
   or regions of the painting.  This might have been appropriate, for
   example, to identify the associated regions that depict the various
   people in the painting:
  </p>

  <div class="exampleInner">
   <pre>
    &lt;!-- Subject matter: (who/what is depicted by this work --&gt;
    &lt;vra:subject&gt;Jeanne-Marguerite Lecadre (artist's cousin)&lt;/vra:subject&gt;
    &lt;vra:subject&gt;Madame Lecadre (artist's aunt)&lt;/vra:subject&gt;
    &lt;vra:subject&gt;Adolphe Monet (artist's father)&lt;/vra:subject&gt;
   </pre>
  </div>

  <h5 id="provenance">Provenance: annotating the past</h5>
  <p>
   Many of the fields below do not contain information about the
   current situation of the painting, but information about places and
   collections the painting has been in the past. This provides
   provenance information that is important in this domain.
  </p>

  <div class="exampleInner">
   <pre>
    &lt;!-- Provenance --&gt;
    &lt;vra:location.currentSite&gt;Metropolitan Museum of Art, New York&lt;/vra:location.currentSite&gt;
    &lt;vra:location.formerSite&gt;Montpellier&lt;/vra:location.formerSite&gt;
    &lt;vra:location.formerSite&gt;Paris&lt;/vra:location.formerSite&gt;
    &lt;vra:location.formerSite&gt;New York&lt;/vra:location.formerSite&gt;
    &lt;vra:location.formerSite&gt;Bryn Athyn, Pa.&lt;/vra:location.formerSite&gt;
    &lt;vra:location.formerSite&gt;London&lt;/vra:location.formerSite&gt;
    &lt;vra:location.formerRepository&gt;
      Victor Frat, Montpellier (probably before 1870 at least 1879;
      bought from the artist); his widow, Mme Frat, Montpellier (until 1913)
    &lt;/vra:location.formerRepository&gt;
    &lt;vra:location.formerRepository&gt;Durand-Ruel, Paris, 1913&lt;/vra:location.formerRepository&gt;
    &lt;vra:location.formerRepository&gt;Durand-Ruel, New York, 1913&lt;/vra:location.formerRepository&gt;
    &lt;vra:location.formerRepository&gt;
      Reverend Theodore Pitcairn and the Beneficia Foundation, Bryn Athyn, Pa. (1926-1967),
      sale, Christie's, London, December 1, 1967, no. 26 to MMA
    &lt;/vra:location.formerRepository&gt;
    &lt;vra:idNumber.currentRepository&gt;67.241&lt;/vra:idNumber.currentRepository&gt; &lt;!-- MMA ID number --&gt;
   </pre>
  </div>

  <h5 id="copyright">Copyright and origin of metadata</h5>
  <p>
   The remaining properties describe the origin the sources used for
   creating the metadata and a rights management statement. We have
   used the <tt>vra:description</tt> element to provide a link to a
   web page with additional descriptive information:
  </p>
  <div class="exampleInner">
   <pre>
    &lt;!-- extra information, source of this information and copyright issues: --&gt;
    &lt;vra:description&gt;For more information, see http://www.metmuseum.org/Works_Of_Art/viewOne.asp?dep=11&amp;viewmode=1&amp;item=67%2E241&amp;section=description#a&lt;/vra:description&gt;
    &lt;vra:source&gt;Metropolitan Museum of Art, New York&lt;/vra:source&gt;
    &lt;vra:rights&gt;Metropolitan Museum of Art, New York&lt;/vra:rights&gt;
      </pre>
    </div>

  <h4 id="images">Image properties</h4>
  <p>
   Finally, we define the properties that are specific to the two
   images of the painting, which differ in resolution, copyright etc.
   The first set of annotations describe a 500x300 pixel image that is
   located at the website of the Metropolitan itself, while the second
   set describes the properties of a larger resolution (1075 x 778px)
   image at Mark Harden's <a
   href="http://www.artchive.com/">Artchive</a> website.
   Note that VRA Core does not specify how Works and their associated Images
   should be related.  In the example we follow <a href="#VraRDF">Van
   Assem's suggestion</a> and use <tt>vra.relation.depicts</tt> to
   explicitly link the Image to the Work it depicts.
  </p>

  <div class="exampleInner">
   <pre>
  &lt;!-- Description of the first online image of the painting --&gt;
  &lt;vra:Image rdf:about="&amp;image1a;"&gt;
    &lt;vra:type&gt;digital images&lt;/vra:type&gt;                                &lt;!-- AAT ID: 300215302 --&gt;
    &lt;vra:relation.depicts rdf:resource="&amp;painting1;"/&gt;
    &lt;vra.measurements.format&gt;image/jpeg&lt;/vra.measurements.format&gt;                   &lt;!-- MIME --&gt;
    &lt;vra.measurements.resolution&gt;500 x 380px&lt;/vra.measurements.resolution&gt;
    &lt;vra.technique&gt;Scanning&lt;/vra.technique&gt;
    &lt;vra:creator&gt;Anonymous employee of the museum&lt;/vra:creator&gt;
    &lt;vra:idNumber.currentRepository&gt;ep67.241.L.jpg&lt;/vra:idNumber.currentRepository&gt;
    &lt;vra:rights&gt;Metropolitan Museum of Art, New York&lt;/vra:rights&gt;
  &lt;/vra:Image&gt;
   </pre>
  </div>
  <div class="exampleInner">
   <pre>
  &lt;!-- Description of the second online image of the painting --&gt;
  &lt;vra:Image rdf:about="&amp;image1b;"&gt;
    &lt;vra:type&gt;digital images&lt;/vra:type&gt;                                &lt;!-- AAT ID: 300215302 --&gt;
    &lt;vra:relation.depicts rdf:resource="&amp;painting1;"/&gt;
    &lt;vra:creator&gt;Mark Harden&lt;/vra:creator&gt;
    &lt;vra.technique&gt;Scanning&lt;/vra.technique&gt;
    &lt;vra.measurements.format&gt;image/jpeg&lt;/vra.measurements.format&gt;                   &lt;!-- MIME --&gt;
    &lt;vra.measurements.resolution&gt;1075 x 778px&lt;/vra.measurements.resolution&gt;
    &lt;vra:idNumber.currentRepository&gt;adresse.jpg&lt;/vra:idNumber.currentRepository&gt;
    &lt;vra:rights&gt;Mark Harden, The Artchive, http://www.artchive.com/&lt;/vra:rights&gt;
  &lt;/vra:Image&gt;
&lt;/rdf:RDF&gt;
   </pre>
  </div>

  <h4 id="ch_conclusion">Conclusion and discussion</h4>
  <p>
   The example above reveals several technical issues that are still
   open.  For example, the way the URI for the painting was minted is
   rather arbitrary.  Preferably, there would have been a commonly
   accepted URI scheme for paintings (c.f. the <a href="#lsid">LSID</a>
   scheme used to identify concepts from the life sciences).  At the
   time of writing, the VRA, AAT and ULAN vocabulary used have
   currently no commonly agreed upon RDF or OWL representation, which
   reduces the interoperability of the chosen approach.  Tool support
   is another issue.  While some major database vendors already start
   to support RDF, generating the type of RDF as shown here from
   existing collection databases will in many cases require non
   trivial custom conversion software.
  </p>
  <p>
   From a modeling point of view, subject matter annotations are
   always non-trivial.  As stated above, it is hard to give general
   guidelines about what should be annotated and to what depth, as
   this can be very application dependent.  Note that in the example,
   we annotated the persons that appear in the painting, and that we
   modeled this information as properties of the painting URI, not of
   the two image URIs. But if we slightly modify our use case and
   assume one normal image and one X-ray image that reveals an older
   painting under this one, it might make more sense to model more
   specific subject matter annotations as properties of the specific
   images.
  </p>

  <p>
   Nevertheless, the example shows that a large part of issues
   described by the use case can be solved using current Semantic Web
   technology.  It shows how RDF can be used to use existing
   vocabularies to annotate various aspects of paintings and the
   images that depict them.
  </p>

  <h3>
   <a name="solution_TVarchive">
    5.3 Use Case: Television News Archive
   </a>
  </h3>

  <h4 id="archive_solution">Possible Semantic Web-based solution</h4>

  <p>
   The use case described in <a href="#media_images">Section 2.3</a>
   is typically one that requires the use of multiple
   vocabularies. Let us imagine that the image to be described is about
   a refused goal of a given soccer player (e.g. J.A Boumsong) for
   an active offside position during a particular game (e.g. Auxerre-Metz).
   First, the image can be extracted from a weekly sports
   magazine broadcasted on a TV channel. This program may be fully
   described using the vocabulary developed by the <a href="#TVA">
   [TV Anytime forum]</a>. Second, this image shows the player
   Jean-Alain Boumsong scoring with his head during the game
   Auxerre-Metz.  The context of this football game could be described
   using the <a href="#MPEG-7">[MPEG-7]</a> vocabulary while the
   action itself might be described by a soccer ontology such as the
   one developed by <a href="#Tsinaraki">[Tsinaraki]</a>. Finally, a
   soccer fan may notice that this goal was actually refused for an
   active offside position of another player. On the image, a circle
   could highlight this player badly positioned. Again, the description
   could merge MPEG-7 vocabulary for delimiting the relevant image
   region and a domain specific ontology for describing the action
   itself.

   In the following, we provide as an example a set of RDF annotations
   illustrating these three levels of description as well as the
   vocabularies involved.
  </p>

  <h5>The image context</h5>
  <p>
   Let us consider that the image comes from a weekly sports magazine named <a
   href="http://sport.france2.fr/stade2/">Stade 2</a> broadcasted on
   March, 17th 2002 on the French public channel <a
   href="http://www.france2.fr/">France 2</a>. This context can be
   represented using the TV Anytime vocabulary which allows for a TV
   (or radio) broadcaster to publish its program listings on the web
   or in an electronic program guide. Therefore, this vocabulary
   provides the necessary concepts and relations for cataloging the
   programs, giving their intended audience, format and genre, or some
   parental guidance. The vocabulary contains also the vocabulary for
   describing afterwards the real audience and the peak viewing times
   which are of crucial importance for the broadcasters in order to
   adapt their advertisement rates.
  </p>

  <div  style="clear: both;" class="exampleOuter">
   <div class="c1">
    <a id="exampleTV" name="exampleTV">RDF description of the program from which the image comes from</a>
   </div>
   <div class="exampleInner">
    <pre>
&lt;?xml version='1.0' encoding='ISO-8859-1'?&gt;
&lt;!DOCTYPE rdf:RDF [
    &lt;!ENTITY rdf        "http://www.w3.org/1999/02/22-rdf-syntax-ns#"&gt;
    &lt;!ENTITY xsd        "http://www.w3.org/2001/XMLSchema#"&gt;
]&gt;

&lt;rdf:RDF
  xmlns:rdf="&amp;rdf;"
  xmlns:xsd="&amp;xsd;"
  xmlns:tva="urn:tva:metadata:2002"
&gt;

  &lt;tva:Program rdf:about="program1"&gt;
    &lt;tva:hasTitle&gt;Stade 2&lt;/tva:hasTitle&gt;
    &lt;tva:hasSynopsis&gt;Weekly Sports Magazine broadcasted every Sunday&lt;/tva:hasSynopsis&gt;
    &lt;tva:Genre rdf:resource="urn:tva:metadata:cs:IntentionCS:2002:Entertainment"/&gt;
    &lt;tva:Genre rdf:resource="urn:tva:metadata:cs:FormatCS:2002:Magazine"/&gt;
    &lt;tva:Genre rdf:resource="urn:tva:metadata:cs:ContentCS:2002:Sports"/&gt;
    &lt;tva:ReleaseInformation&gt;
      &lt;rdf:Description&gt;
        &lt;tva:ReleaseDate xsd:date="2002-03-17"/&gt;
        &lt;tva:ReleaseLocation&gt;fr&lt;/tva:ReleaseLocation&gt;
      &lt;/rdf:Description&gt;
    &lt;/tva:ReleaseInformation&gt;
  &lt;/tva:Program&gt;

&lt;/rdf:RDF&gt;
    </pre>
   </div>
  </div>

  <h5>The description of the action</h5>
  <p>
   To be done.
  </p>

  <h5>The description of particular region</h5>
  <p>
   Discuss the pros and cons of having either 2 separate files (one
   expressing the localization of the region and one representing the
   content annotation) or 1 RDF file having both description.
  </p>

  <h5>The annotation link</h5>
  <p>
   Discuss the various annotation links provided by MPEG-7 (annotates, depicts, exemplifies, etc).
  </p>


  <h3>
   <a name="solution_NASA">
    5.4 Use Case: large-scale image collections at NASA
   </a>
  </h3>

  <div style="float: right; width: 317px; height:540px; border: 1px solid gray; padding: 1%; margin: 1%">
   <a href="http://grin.hq.nasa.gov/IMAGES/SMALL/GPN-2000-001171.jpg">
    <img style="width: 317px;height:450px;"
     src="http://grin.hq.nasa.gov/IMAGES/SMALL/GPN-2000-001171.jpg"
     alt="Apollo 7 Saturn rocket launch"/></a>
   <br/>
   Apollo 7 Saturn rocket launch -
   October, 10th 1968. Image courtesy of NASA, available at <a href="http://grin.hq.nasa.gov/">GRIN</a>,
   used with permission.
  </div>

  <h4 id="large-collection-solution">Possible Semantic Web-based solution</h4>
  <p>
    One possible solution for the requirements expressed in the use case description
    in <a href="#scientific_images">Section 2.4</a>
    is an annotation environment that enables users to annotate information
    about images and/or their regions using concepts in ontologies
    (OWL and/or RDFS). More specifically, subject matter experts will
    be able to assert metadata elements about images and their
    specific content. Multimedia related ontologies can be used to
    localize and represent regions within particular images. These
    regions can then be related to the image via a
    depiction/annotation property. This functionality can be provided,
    for example, by the <a
    href="http://www.mindswap.org/2005/owl/digital-media">MINDSWAP
    digital-media ontology</a> (to represent images, image regions,
    etc.), in conjunction with <a
    href="http://xmlns.com/foaf/0.1/">FOAF</a> (to assert image
    depictions). Additionally, in order to represent the low level
    image features of regions, the <a
    href="http://www.acemedia.org/aceMedia/reference/resource/index.html">aceMedia
    Visual Descriptor Ontology</a> can be used.
    </p>

  <h5>Domain Specific Ontologies</h5>

  <p>
   In order to describe the content of such images, a mechanism to
   represent the domain specific content depicted within them is
   needed. For this use case, domain ontologies that define space
   specific concepts and relations can be used. Such ontologies are
   freely available and include, but are not limited to the following:
  </p>
  <ul>
   <li> <a href="http://semspace.mindswap.org/2004/ontologies/ShuttleMission-ont.owl">Shuttle related (OWL)</a></li>
   <li> <a href="http://semspace.mindswap.org/2004/ontologies/ShuttleMission-ont.rdfs">Shuttle related (RDFS)</a></li>
   <li> <a href="http://semspace.mindswap.org/2004/ontologies/System-ont.owl">Space vehicle system related (OWL)</a></li>
   <li> <a href="http://semspace.mindswap.org/2004/ontologies/System-ont.rdfs">Space vehicle system related (RDFS)</a></li>
  </ul>

  <h5>Visual Ontologies</h5>
  <p>
   As discussed above, this scenario requires the ability to state
   that images (and possibly their regions) depict certain things. For
   example, consider a picture of the <a
   href="http://grin.hq.nasa.gov/IMAGES/SMALL/GPN-2000-001171.jpg">Apollo
   7 Saturn rocket launch</a>. One would want to make assertions that
   include that the image <i>depicts</i> the Apollo 7 launch, the
   Apollo 7 Saturn IB space vehicle is depicted in a rectangular
   <i>region</i> around the rocket, the image <i>creator</i> is NASA,
   etc. One possible way to accomplish this is to use a combination of
   various multimedia related ontologies, including <a
   href="http://xmlns.com/foaf/0.1/">FOAF</a> and the <a
   href="http://www.mindswap.org/2005/owl/digital-media">MINDSWAP
   digital-media ontology</a>. More specifically, image depictions can
   be asserted via a <i>depiction</i> property (a sub-property of
   foaf:depiction) defined in the MINDSWAP Digital Media
   ontology. Thus, images can be semantically linked to instances
   defined on the Web. Image regions can defined via an
   <i>ImagePart</i> concept (also defined in the MINDSWAP Digital
   Media ontology). Additionally, regions can be given a bounding box
   by using a property named <i>svgOutline</i>, allowing localizing of
   image parts. Essentially SVG outlines (SVG XML literals) of the
   regions can be specified using this property. Using the <a
   href="http://dublincore.org/schemas/rdfs//">Dublin Core
   standard</a> and the <a href="http://www.w3.org/2003/12/exif/">EXIF
   Schema </a> more general annotations about the image can be stated
   as well, including its creator, size, etc. A subset of these sample
   annotations are shown in an RDF graph below in <a
   href="#figure2">Figure 2</a>.
  </p>

  <div style="text-align: center">
   <img alt="RDF Graph Describing the Apollo 7 Launch Image"
    style="width: 100%;"
    src="nasaRDFDiagram.png"/>
   <br/>
   <a id="figure2" name="figure2">Figure 2: An RDF Graph
    Describing the Apollo 7 Launch Image</a>
  </div>

  <p>
   <a href="#figure2">Figure 2</a> illustrates how the approach links metadata to the image:</p>

  <ul>
   <li>image content, e.g., Apollo 7 Launch, is identified by
    <code>http://www.mindswap.org/2005/owl/digital-media#depicts</code></li>

   <li>image subparts are identified by the property
    <code>http://www.mindswap.org/2005/owl/digital-media#hasRegion</code></li>

   <li>image regions are localized using
    <code>http://www.mindswap.org/2005/owl/digital-media#svgOutline</code> and an SVG snippet</li>

  </ul>


  <p>
   Additionally, the entire annotations of the Apollo 7 launch are shown below in RDF/XML.<br/><br/>
  </p>
  <div class="c1">
   <a id="exampleApollo7" name="exampleApollo7">RDF/XML annotations of Apollo 7 launch </a>
  </div>

  <div class="exampleInner">
   <pre>
&lt;rdf:RDF
    xmlns:j.0="http://www.w3.org/2003/12/exif/ns#"
    xmlns:j.1="http://www.mindswap.org/2005/owl/digital-media#"
    xmlns:rdf="http://www.w3.org/1999/02/22-rdf-syntax-ns#"
    xmlns:j.2="http://semspace.mindswap.org/2004/ontologies/System-ont.owl#"
    xmlns:rdfs="http://www.w3.org/2000/01/rdf-schema#"
    xmlns:owl="http://www.w3.org/2002/07/owl#"
    xmlns:dc="http://purl.org/dc/elements/1.1/"
    xmlns:j.3="http://semspace.mindswap.org/2004/ontologies/ShuttleMission-ont.owl#"
    xml:base="http://example.org/NASA-Use-Case" &gt;

  &lt;rdf:Description rdf:about="A0"&gt;
    &lt;j.1:depicts rdf:resource="#Saturn_1B"/&gt;
    &lt;rdf:type rdf:resource="http://www.mindswap.org/~glapizco/technical.owl#ImagePart"/&gt;
    &lt;rdfs:label&gt;region2407&lt;/rdfs:label&gt;
    &lt;j.1:regionOf rdf:resource="http://grin.hq.nasa.gov/IMAGES/SMALL/GPN-2000-001171.jpg"/&gt;
    &lt;j.1:svgOutline&gt;
     &lt;svg xml:space="preserve" width="451" heigth="640" viewBox="0 0 451 640"&gt;
      &lt;image xlink:href="http://grin.hq.nasa.gov/IMAGES/SMALL/GPN-2000-001171.jpg" x="0" y="0"  width="451" height="640" /&gt;
      &lt;rect x="242.0" y="79.0" width="46.0" height="236.0" style="fill:none; stroke:yellow; stroke-width:1pt;"/&gt;
     &lt;/svg&gt;
    &lt;/j.1:svgOutline&gt;
  &lt;/rdf:Description&gt;

  &lt;rdf:Description rdf:about="http://grin.hq.nasa.gov/IMAGES/SMALL/GPN-2000-001171.jpg"&gt;
    &lt;j.0:imageLength&gt;640&lt;/j.0:imageLength&gt;
    &lt;dc:date&gt;10/11/1968&lt;/dc:date&gt;
    &lt;dc:description&gt;Taken at Kennedy Space Center in Florida&lt;/dc:description&gt;
    &lt;j.1:depicts rdf:resource="#Apollo_7_Launch"/&gt;
    &lt;j.1:hasRegion rdf:nodeID="A0"/&gt;
    &lt;dc:creator&gt;NASA&lt;/dc:creator&gt;
    &lt;rdf:type rdf:resource="http://www.mindswap.org/~glapizco/technical.owl#Image"/&gt;
    &lt;j.0:imageWidth&gt;451&lt;/j.0:imageWidth&gt;
  &lt;/rdf:Description&gt;

  &lt;rdf:Description rdf:about="#Apollo_7_Launch"&gt;
    &lt;j.3:launchDate&gt;10/11/1968&lt;/j.3:launchDate&gt;
    &lt;j.3:codeName&gt;Apollo 7 Launch&lt;/j.3:codeName&gt;
    &lt;j.3:has_shuttle rdf:resource="#Saturn_1B"/&gt;

    &lt;rdfs:label&gt;Apollo 7 Launch&lt;/rdfs:label&gt;
    &lt;j.1:depiction rdf:resource="http://grin.hq.nasa.gov/IMAGES/SMALL/GPN-2000-001171.jpg"/&gt;
    &lt;rdf:type rdf:resource="http://semspace.mindswap.org/2004/ontologies/ShuttleMission-ont.owl#Launch"/&gt;
  &lt;/rdf:Description&gt;

  &lt;rdf:Description rdf:about="#Saturn_1B"&gt;
    &lt;rdfs:label&gt;Saturn_1B&lt;/rdfs:label&gt;
    &lt;j.1:depiction rdf:nodeID="A1"/&gt;
    &lt;rdfs:label&gt;Saturn 1B&lt;/rdfs:label&gt;
    &lt;rdf:type rdf:resource="http://semspace.mindswap.org/2004/ontologies/System-ont.owl#ShuttleName"/&gt;
    &lt;j.1:depiction rdf:nodeID="A0"/&gt;
  &lt;/rdf:Description&gt;

&lt;/rdf:RDF&gt;
   </pre>
  </div>


  <p>
   In order to represent the low level features of images, the <a
   href="http://www.acemedia.org/aceMedia/reference/resource/index.html">aceMedia
   Visual Descriptor Ontology</a> can be used. This ontology contains
   representations of MPEG-7 visual descriptors and models Concepts
   and Properties that describe visual characteristics of objects. For
   example, the dominant color descriptor can be used to describe the
   number and value of dominant colors that are present in a region of
   interest and the percentage of pixels that each associated color
   value has.
  </p>


  <h5>Available Annotation Tools</h5>
  <p>
   Existing toolkits, such as <a href="#PS">[PhotoStuff]</a> and <a
   href="#mOnto">[M-OntoMat-Annotizer]</a>, currently provide
   graphical environments to accomplish the annotation tasks mentioned
   above. Using such tools, users can load images, create regions
   around parts of the image, automatically extract low-level features
   of selected regions (via M-OntoMat-Annotizer), assert statements
   about the selected regions, etc. Additionally, the resulting
   annotations can be exported as RDF/XML (as shown above), thus
   allowing them be shared, indexed, and used by advanced
   annotation-based browsing (and searchable) environments.
   </p>

  <!-- ===================================================================== -->
  <h2>
   <a name="conclusions" id="conclusions">
    6. Conclusions
   </a>
  </h2>
  <p>
   Current Semantic Web technologies are sufficiently generic to support
   annotation of a wide variety of Web resources, including image
   resources. This document provides examples of the use of Semantic
   Web languages and tools for image annotation, based on use cases
   for a wide variety of domains. It also briefly surveys some
   currently available vocabularies and tools that can be used to
   semantically annotate images so that machine can better process them.
   The use of Semantic Web technologies have significant advantages in applications areas in
   which the interoperability of heterogeneous metadata is important
   and in areas that require an explicitly defined and formal
   semantics of the metadata in order to perform reasoning tasks.
  </p>

  <p>
   Still, many things need to be improved. Commonly accepted, widely
   used vocabularies for image annotation are still missing.  Having
   such vocabularies would help in sharing metadata across
   applications and across multiple domains. Especially, a standard
   means to address subregions withing an image is still missing.  In
   addition, tool support needs to improve dramatically before
   Semantic Web-based image annotation can be applied on an industrial
   scale: support needs to be integrated in the entire production and
   distribution chain. Finally, many existing approaches for image
   metadata are not based on Semantic Web technologies, and work is
   required to make these approaches interoperable with the Semantic Web.
  </p>

  <!-- ===================================================================== -->
  <h2>
   <a name="references" id="references">
    References</a>
  </h2>

  <dl>

   <dt>
    <a id="refAAT" name="refAAT">[AAT]</a>
   </dt>
   <dd><span class="title">Art and Architecture Thesaurus</span>.
    The J.  Paul Getty Trust, 2004.
    (See <a href="http://www.getty.edu/research/conducting_research/vocabularies/aat/">
     http://www.getty.edu/research/conducting_research/vocabularies/aat/</a>)
   </dd>

   <dt>
    <a id="DublinCore" name="DublinCore">[Dublin Core]</a>
   </dt>
   <dd>
    The Dublin Core Metadata Initiative,
    <a href="http://dublincore.org/documents/dces/">Dublin Core Metadata Element Set, Version 1.1: Reference Description</a>.
   </dd>

   <dt>
    <a id="httpRange-14" name="httpRange-14">
     [httpRange-14]</a></dt>
   <dd>
    TAG's issue list, issue 14, see
    <a href="http://www.w3.org/2001/tag/issues.html?type=1#httpRange-14">
     http://www.w3.org/2001/tag/issues.html?type=1#httpRange-14</a>
   </dd>

   <dt>
    <a id="HTTP-URI" name="HTTP-URI">
     [HTTP-URI]</a></dt>
   <dd>
    Tim Berners-Lee, What do HTTP URIs Identify?  Available at <a
    href="http://www.w3.org/DesignIssues/HTTP-URI">http://www.w3.org/DesignIssues/HTTP-URI</a>
   </dd>

   <dt>
    <a id="Hunter01" name="Hunter01">[Hunter, 2001]</a>
   </dt>
   <dd>
    J. Hunter.
    <!-- official link broken? a href="http://www.semanticweb.org/SWWS/program/full/paper59.pdf" -->
    <a href="http://archive.dstc.edu.au/RDU/staff/jane-hunter/swws.pdf">Adding
    Multimedia to the Semantic Web &#8212; Building an MPEG-7
    Ontology</a>. In <i><a
    href="http://www.semanticweb.org/SWWS/">International Semantic Web
    Working Symposium (SWWS 2001)</a></i>, Stanford University,
    California, USA, July 30 - August 1, 2001.
      </dd>

   <dt>
    <a id="lsid" name="lsid">
     [LSID]
    </a>
   </dt>
   <dd>
    Life Sciences Identifier specification, <a
     href="http://www.omg.org/cgi-bin/doc?dtc/04-05-01">
     http://www.omg.org/cgi-bin/doc?dtc/04-05-01</a>.
   </dd>

   <dt>
    <a name="refMIME-2" id="refMIME-2">[MIME-2]</a>
   </dt>
   <dd>
    <a href="ftp://ftp.isi.edu/in-notes/rfc2046.txt">
     RFC 2046: Multipurpose Internet Mail Extensions (MIME) Part Two:
     Media Types
    </a>
    .  N. Freed, N. Borenstein, November 1996.  Available at
    <a href="ftp://ftp.isi.edu/in-notes/rfc2046.txt">ftp://ftp.isi.edu/in-notes/rfc2046.txt</a>
   </dd>

   <dt>
    <a id="mOnto" name="mOnto">[M-OntoMat-Annotizer]</a>
   </dt>
   <dd>
    M-OntoMat-AnnotizerProject Homepage at <a
    href="http://www.acemedia.org/aceMedia/results/software/m-ontomat-annotizer.html">
    http://www.acemedia.org/aceMedia/results/software/m-ontomat-annotizer.html</a>
   </dd>

   <dt>
    <a id="MPEG-7" name="MPEG-7">[MPEG-7]</a>
   </dt>
   <dd>
    Information Technology - Multimedia Content Description Interface (MPEG-7).
    Standard No. ISO/IEC 15938:2001, International Organization for Standardization(ISO), 2001.
   </dd>

   <dt>
    <a id="Ossenbruggen04" name="Ossenbruggen04">[Ossenbruggen, 2004]</a>
   </dt>
   <dd>
    J. van Ossenbruggen, F. Nack, and L. Hardman. That Obscure Object of Desire: Multimedia Metadata on the Web (Part I). In:
    IEEE Multimedia 11(4), pp. 38-48 October-December 2004.
   </dd>

   <dt>
    <a id="Ossenbruggen05" name="Ossenbruggen05">[Ossenbruggen, 2005]</a>
   </dt>
   <dd>
    F. Nack, J. van Ossenbruggen, and L. Hardman. That Obscure Object of Desire: Multimedia Metadata on the Web (Part II). In:
    IEEE Multimedia 12(1), pp. 54-63 January-March 2005.
   </dd>

   <dt>
    <a name="OWL-Guide" id="OWL-Guide">[OWL Guide]</a>
   </dt>
   <dd>
     <cite>
       <a href="http://www.w3.org/TR/2004/REC-owl-guide-20040210/">
        OWL Web Ontology Language Guide</a></cite>, Michael K.
        Smith, Chris Welty, and Deborah L. McGuinness, Editors, W3C
        Recommendation, 10 February 2004,
        http://www.w3.org/TR/2004/REC-owl-guide-20040210/ .
        <a href="http://www.w3.org/TR/owl-guide/">Latest
        version</a> available at http://www.w3.org/TR/owl-guide/ .
   </dd>

   <dt>
    <a name="OWL" id="OWL">[OWL Semantics and Abstract Syntax]</a></dt>
   <dd>
    <cite>
     <a href=
      "http://www.w3.org/TR/2004/REC-owl-semantics-20040210/">OWL Web
     Ontology Language Semantics and Abstract Syntax</a></cite>, Peter
     F. Patel-Schneider, Patrick Hayes, and Ian Horrocks, Editors, W3C
     Recommendation 10 February 2004,
     http://www.w3.org/TR/2004/REC-owl-semantics-20040210/ . <a href=
     "http://www.w3.org/TR/owl-semantics/">Latest version</a>
     available at http://www.w3.org/TR/owl-semantics/ .</dd>

   <dt>
    <a id="PS" name="PS">[PhotoStuff]</a>
   </dt>
   <dd>
    PhotoStuff Project Homepage at <a
    href="http://www.mindswap.org/2003/PhotoStuff/">http://www.mindswap.org/2003/PhotoStuff/</a>
   </dd>

   <dt><a id="RDF-Primer" name="RDF-Primer">[RDF Primer]</a></dt>
   <dd>
     <cite><a href="http://www.w3.org/TR/2004/REC-rdf-primer-20040210/">RDF
   Primer</a></cite>, F. Manola, E. Miller, Editors, W3C Recommendation, 10 February 2004.  <a href="http://www.w3.org/TR/2004/REC-rdf-primer-20040210/">This
   version</a> is
   http://www.w3.org/TR/2004/REC-rdf-primer-20040210/. The <a href="http://www.w3.org/TR/rdf-primer/">latest version</a> is at
   http://www.w3.org/TR/rdf-primer/.
   </dd>

   <dt><a id="RDF" name="RDF"></a>[RDF Syntax]</dt>
   <dd>
    <cite>
     <a href="http://www.w3.org/TR/2004/REC-rdf-syntax-grammar-20040210/">
      RDF/XML Syntax Specification (Revised)</a></cite>, Dave Beckett,
      Editor, W3C Recommendation, 10 February 2004,
      http://www.w3.org/TR/2004/REC-rdf-syntax-grammar-20040210/ .  <a
      href="http://www.w3.org/TR/rdf-syntax-grammar/">Latest
      version</a> available at
      http://www.w3.org/TR/rdf-syntax-grammar/ .</dd>


   <dt>
    <a id="Stamou05" name="Stamou05">[Stamou, 2005]</a>
   </dt>
   <dd>
    G. Stamou and S. Kollias (eds). Multimedia Content and the
    Semantic Web: Methods, Standards and Tools.  John Wiley &amp; Sons
    Ltd, 2005.
   </dd>

   <dt>
    <a id="Troncy03" name="Troncy03">[Troncy, 2003]</a>
   </dt>
   <dd>
    R. Troncy. <a
    href="http://springerlink.metapress.com/openurl.asp?genre=article&amp;issn=0302-9743&amp;volume=2870&amp;spage=566">
    Integrating Structure and Semantics into Audio-visual
    Documents</a>. In <i><a
    href="http://iswc2003.semanticweb.org/">Second International
    Semantic Web Conference (ISWC 2003)</a></i>, pages 566 &#8211;
    581, Sanibel Island, Florida, USA, October 20-23,
    2003. Springer-Verlag Heidelberg.
   </dd>

   <dt>
    <a id="Tsinaraki" name="Tsinaraki">
     [Tsinaraki]</a></dt>
   <dd>
    Tsinaraki, C.: OWL soccer ontology available at
    <a href="http://elikonas.ced.tuc.gr/ontologies/soccer.zip">http://elikonas.ced.tuc.gr/ontologies/soccer.zip</a>.
   </dd>

   <dt>
    <a id="TVA" name="TVA">[TV Anytime]</a>
   </dt>
   <dd>
    TV Anytime Forum,
    <a href="http://www.tv-anytime.org/">
     http://www.tv-anytime.org/
    </a>
   </dd>

   <dt>
    <a id="refULAN" name="refULAN">[ULAN]</a>
   </dt>
   <dd><span class="title">Union List of Artist Names</span>.
    The J.  Paul Getty Trust, 2004.
    (See <a href="http://www.getty.edu/research/conducting_research/vocabularies/ulan/">
     http://www.getty.edu/research/conducting_research/vocabularies/ulan/</a>)
   </dd>

   <dt>
    <a id="VDO" name="VDO">[VDO]</a>
   </dt>
   <dd>
    aceMedia Visual Descriptor Ontology, available from <a
    href="http://www.acemedia.org/aceMedia/reference/resource/index.html">
    http://www.acemedia.org/aceMedia/reference/resource/index.html</a>
   </dd>


   <dt>
    <a id="VraCore" name="VraCore">[VRA Core]</a>
   </dt>
   <dd>
    Visual Resources Association Data Standards Committee,
    <a href="http://www.vraweb.org/vracore3.htm">
     VRA Core Categories, Version 3.0</a>.  Available at:
    <a href="http://www.vraweb.org/vracore3.htm">
     http://www.vraweb.org/vracore3.htm</a>.
   </dd>


   <dt>
    <a id="VraRDF" name="VraRDF">[VRA in RDF/OWL]</a>
   </dt>

   <dd>
    Mark van Assem. <a href="http://www.w3.org/2001/sw/BestPractices/MM/vra-conversion.html">
     http://www.w3.org/2001/sw/BestPractices/MM/vra-conversion.html</a>
    describes the RDFS schema of VRA Core 3.0 used in
    <a href="#solution_culture">section 5.2</a>.
   </dd>

    <dt>
     <a id="XML-NS" name="XML-NS">[XML NS]</a>
    </dt>
    <dd>
     <cite>
     <a href="http://www.w3.org/TR/1999/REC-xml-names-19990114/">Namespaces
     in XML</a></cite>, Bray T., Hollander D., Layman A.
     (Editors), World Wide Web Consortium, 14 January 1999. <a href="http://www.w3.org/TR/1999/REC-xml-names-19990114/">This
     version</a> is http://www.w3.org/TR/1999/REC-xml-names-19990114/. The <a href="http://www.w3.org/TR/REC-xml-names/">latest version</a>
     is http://www.w3.org/TR/REC-xml-names/.
    </dd>

  </dl>

  <!-- ======================================================================== -->

  <h2>
   <a id="acknowledgments" name="acknowledgments">Acknowledgments</a>
  </h2>

  <p>
   The editors would like to thank
   John Smith (IBM T. J. Watson Research Center), Chris Catton (University of Oxford)
   and the following Working Group members
   for their feedback on earlier versions of this document:
   Mark van Assem,
   Jeremy Caroll,
   Jane Hunter,
   Libby Miller,
   Guus Schreiber and
   Michael Uschold.
  </p>

  <p>
   This document is a product of the Multimedia Annotation on the
   Semantic Web Task Force of the Semantic Web Best Practices and
   Deployment Working Group.
  </p>

 </body>
</html>