index.html 125 KB

Raw Blame History Permalink

<?xml version="1.0" encoding="UTF-8"?>
<!--OFFLINE
<!DOCTYPE html SYSTEM "DTD/xhtml1-strict.dtd">
OFFLINE-->
<!DOCTYPE html PUBLIC "-//W3C//DTD XHTML 1.0 Transitional//EN"
      "http://www.w3.org/TR/xhtml1/DTD/xhtml1-transitional.dtd">
<html xmlns="http://www.w3.org/1999/xhtml" xml:lang="en" lang="en">
<head>
  <title>Emotion Markup Language (EmotionML) 1.0</title>
  <meta http-equiv="content-type" content="text/html; charset=UTF-8" />
  <style type="text/css" xml:space="preserve">
/*<![CDATA[*/
    .sample { border: thin black solid }
    .boilerplate, .boilerplate-nocode { font-size: small; background: #ddd }
    div.boilerplate, p.boilerplate, blockquote.boilerplate, div.boilerplate-nocode, p.boilerplate-nocode, blockquote.boilerplate-nocode { margin: 1em; border: thin black solid; padding: .25em }
    div.source { margin: 1em }
    .notetoeditor { color: red; background: white}
    .comment { color: red; background: white}

div.issue {
  border-width: thin;
  border-style: solid;
  border-color: maroon;
  background-color: #FFEECC;
  color: maroon;
  width: 95%; padding: 0.5em; }

div.issue h4 {
  margin-top: 0;
  font-weight: bold;
  color: maroon;
}

pre {
   font-family: monospace;
   font-weight: bold;
   white-space: pre;
   background-color: rgb(204,204,255);
   padding: 0.5em;
   border: none;
   margin-left: 0;
   width: 90%;
}

pre.xml {
   font-family: "Lucida Sans Unicode", monospace;
   background-color: rgb(204,204,255);
   border: solid black thin;
}
code {
    color: green;
    font-family: monospace;
    font-weight: bold;
}

table.defn th { background-color: rgb(220,220,255);
  border-style: solid; border-color: black; border-width: thin }
table.defn td { background-color: rgb(230,230,255);
  border-style: solid; border-color: black; border-width: thin }


    /*]]>*/

  </style>
  <link rel="stylesheet" type="text/css"
  href="http://www.w3.org/StyleSheets/TR/W3C-WD" />
</head>

<body>

<div class="head">
<p><a href="http://www.w3.org/"><img alt="W3C" height="48"
src="http://www.w3.org/Icons/w3c_home" width="72" /></a> </p>

<h1 style="clear:both" id="title">Emotion Markup Language (EmotionML) 1.0 </h1>

<h2 id="W3C-doctype">W3C Working Draft 7 April 2011 </h2>
<dl>
  <dt>This version:</dt>
    <dd><a
      href="http://www.w3.org/TR/2011/WD-emotionml-20110407/">http://www.w3.org/TR/2011/WD-emotionml-20110407/</a>
    </dd>
  <dt>Latest version:</dt>
    <dd><a
      href="http://www.w3.org/TR/emotionml/">http://www.w3.org/TR/emotionml/</a>
    </dd>
  <dt>Previous version:</dt>
    <dd><a
      href="http://www.w3.org/TR/2010/WD-emotionml-20100729/">http://www.w3.org/TR/2010/WD-emotionml-20100729/</a>
    </dd>
</dl>
<dl>
  <dt>Editor:</dt>
    <dd>Marc Schröder (DFKI GmbH)</dd>
  <dt>Authors: </dt>
    <dd><em>(in alphabetic order)</em></dd>
    <dd>Paolo Baggia (Loquendo, S.p.A.)</dd>
    <dd>Felix Burkhardt (Deutsche Telekom AG)</dd>
    <dd>Catherine Pelachaud (Telecom ParisTech)</dd>
    <dd>Christian Peter (Fraunhofer Gesellschaft)</dd>
    <dd>Enrico Zovato (Loquendo, S.p.A.)</dd>
</dl>

<p class="copyright"><a href="http://www.w3.org/Consortium/Legal/ipr-notice#Copyright">Copyright</a> &copy; 2011 <a href="http://www.w3.org/"><acronym title="World Wide Web Consortium">W3C</acronym></a><sup>&reg;</sup> (<a href="http://www.csail.mit.edu/"><acronym title="Massachusetts Institute of Technology">MIT</acronym></a>, <a href="http://www.ercim.eu/"><acronym title="European Research Consortium for Informatics and Mathematics">ERCIM</acronym></a>, <a href="http://www.keio.ac.jp/">Keio</a>), All Rights Reserved. W3C <a href="http://www.w3.org/Consortium/Legal/ipr-notice#Legal_Disclaimer">liability</a>, <a href="http://www.w3.org/Consortium/Legal/ipr-notice#W3C_Trademarks">trademark</a> and <a href="http://www.w3.org/Consortium/Legal/copyright-documents">document use</a> rules apply.</p>
<hr />
</div>

<h2 id="abstract">Abstract</h2>

<p>As the web is becoming ubiquitous, interactive, and multimodal, technology
needs to deal increasingly with human factors, including emotions. The
specification of Emotion Markup Language 1.0 aims to strike a balance between
practical applicability and scientific well-foundedness. The language is
conceived as a "plug-in" language suitable for use in three different areas:
(1) manual annotation of data; (2) automatic recognition of emotion-related
states from user behavior; and (3) generation of emotion-related system
behavior.</p>

<h2 id="status">Status of this document</h2>

<p><em>This section describes the status of this document at the time of its
publication. Other documents may supersede this document. A list of current W3C
publications and the latest revision of this technical report can be found in
the <a href="http://www.w3.org/TR/">W3C technical reports index</a> at
http://www.w3.org/TR/.</em></p>

<p>This is a Last Call Working Draft of "Emotion Markup Language 1.0",
published on 7 April 2011. The W3C Membership and other interested parties are
invited to review the document and send comments to <a
href="mailto:www-multimodal@w3.org">www-multimodal@w3.org</a> (with <a
href="http://lists.w3.org/Archives/Public/www-multimodal/">public archive</a>)
until 7 June 2011.</p>

<p>This Last Call Working Draft has addressed all open issues from the previous
working draft, as well as the issues which were raised at the W3C workshop on
EmotionML. The changes compared to the previous Working Draft of 29 July 2010
are listed in <a href="#changelog">Appendix A</a>.</p>

<p>This document was developed by the <a
href="http://www.w3.org/2002/mmi/">Multimodal Interaction Working Group</a>.
The Working Group expects to advance this Working Draft to Recommendation
Status. </p>

<p>Publication as a Working Draft does not imply endorsement by the W3C
Membership. This is a draft document and may be updated, replaced or obsoleted
by other documents at any time. It is inappropriate to cite this document as
other than work in progress. </p>

<p>This document was produced by a group operating under the <a
href="http://www.w3.org/Consortium/Patent-Policy-20040205/">5 February 2004 W3C
Patent Policy</a>. W3C maintains a <a rel="disclosure"
href="http://www.w3.org/2004/01/pp-impl/34607/status">public list of any patent
disclosures</a> made in connection with the deliverables of the group; that
page also includes instructions for disclosing a patent. An individual who has
actual knowledge of a patent which the individual believes contains <a
href="http://www.w3.org/Consortium/Patent-Policy-20040205/#def-essential">Essential
Claim(s)</a> must disclose the information in accordance with <a
href="http://www.w3.org/Consortium/Patent-Policy-20040205/#sec-Disclosure">section
6 of the W3C Patent Policy</a>. </p>

<h2 id="conv">Conventions of this document</h2>

<p>The key words "MUST", "MUST NOT", "REQUIRED", "SHALL", "SHALL NOT",
"SHOULD", "SHOULD NOT", "RECOMMENDED", "MAY", and "OPTIONAL" in this document
are to be interpreted as described in [<a href="#ref-rfc2119">RFC2119</a>]. </p>

<p>The sections in the main body of this document are normative unless
otherwise specified. The appendices in this document are informative unless
otherwise indicated explicitly.</p>

<h2 id="toc">Table of Contents</h2>

<div class="toc">
<ul>
  <li><a href="#s1">1 Introduction</a>
    <ul>
      <li><a href="#s1.1">1.1 Reasons for defining an Emotion Markup
        Language</a></li>
      <li><a href="#s1.2">1.2 The challenge of defining a generally usable
        Emotion Markup Language</a></li>
      <li><a href="#s1.3">1.3 Glossary of terms</a></li>
    </ul>
  </li>
  <li><a href="#s2">2 Elements of Emotion Markup</a>
    <ul>
      <li><a href="#s2.1">2.1 Document structure</a>
        <ul>
          <li><a href="#s2.1.1">2.1.1 Document root: The
            <code>&lt;emotionml&gt;</code> element</a></li>
          <li><a href="#s2.1.2">2.1.2 A single emotion annotation: The
            <code>&lt;emotion&gt;</code> element</a></li>
        </ul>
      </li>
      <li><a href="#s2.2">2.2 Representations of emotions and related
        states</a>
        <ul>
          <li><a href="#s2.2.1">2.2.1 The <code>&lt;category&gt;</code>
            element</a></li>
          <li><a href="#s2.2.2">2.2.2 The <code>&lt;dimension&gt;</code>
            element</a></li>
          <li><a href="#s2.2.3">2.2.3 The <code>&lt;appraisal&gt;</code>
            element</a></li>
          <li><a href="#s2.2.4">2.2.4 The <code>&lt;action-tendency&gt;</code>
            element</a></li>
        </ul>
      </li>
      <li><a href="#s2.3">2.3 Meta-information</a>
        <ul>
          <li><a href="#s2.3.1">2.3.1 The <code>confidence</code>
          attribute</a></li>
          <li><a href="#s2.3.2">2.3.2 The <code>expressed-through</code>
            attribute</a></li>
          <li><a href="#s2.3.3">2.3.3 The <code>&lt;info&gt;</code>
          element</a></li>
        </ul>
      </li>
      <li><a href="#s2.4">2.4 References and time</a>
        <ul>
          <li><a href="#s2.4.1">2.4.1 The <code>&lt;reference&gt;</code>
            element</a></li>
          <li><a href="#s2.4.2">2.4.2 Timestamps</a>
            <ul>
              <li><a href="#s2.4.2.1">2.4.2.1 Absolute time</a></li>
              <li><a href="#s2.4.2.2">2.4.2.2 Duration</a></li>
              <li><a href="#s2.4.2.3">2.4.2.3 Relative time</a></li>
              <li><a href="#s2.4.2.4">2.4.2.4 Timing in media</a></li>
            </ul>
          </li>
        </ul>
      </li>
      <li><a href="#s2.5">2.5 Scale values</a>
        <ul>
          <li><a href="#s2.5.1">2.5.1 The <code>value</code> attribute</a></li>
          <li><a href="#s2.5.2">2.5.2 The <code>&lt;trace&gt;</code>
          element</a></li>
        </ul>
      </li>
    </ul>
  </li>
  <li><a href="#s3">3 Defining vocabularies for representing emotions</a>
    <ul>
      <li><a href="#s3.1">3.1 Mechanism for defining vocabularies</a>
        <ul>
          <li><a href="#s3.1.1">3.1.1 The <code>&lt;vocabulary&gt;</code>
            element</a></li>
          <li><a href="#s3.1.2">3.1.2 The <code>&lt;item&gt;</code>
          element</a></li>
        </ul>
      </li>
    </ul>
  </li>
  <li><a href="#s4">4 Conformance</a>
    <ul>
      <li><a href="#s4.1">4.1 EmotionML namespace</a></li>
      <li><a href="#s4.2">4.2 Use with other namespaces</a></li>
      <li><a href="#s4.3">4.3 Schema validation and processor validation of
        EmotionML documents</a></li>
    </ul>
  </li>
  <li><a href="#s5">5 Examples</a>
    <ul>
      <li><a href="#s5.1">5.1 Examples of emotion annotation</a>
        <ul>
          <li><a href="#s5.1.1">5.1.1 Manual annotation of emotional
            material</a>
            <ul>
              <li><a href="#example_annotation_images">Annotation of static
                images</a></li>
              <li><a href="#example_annotation_videos">Annotation of
              videos</a></li>
            </ul>
          </li>
          <li><a href="#s5.1.2">5.1.2 Automatic recognition of emotions</a></li>
          <li><a href="#s5.1.3">5.1.3 Generation of emotion-related system
            behavior</a>
            <ul>
              <li><a href="#example_generation_mpeg4">Generation of facial
                expressions in an MPEG-4 face model</a></li>
              <li><a href="#example_generation_robot">Generation of robot
                behavior</a></li>
            </ul>
          </li>
        </ul>
      </li>
      <li><a href="#s5.2">5.2 Examples of possible use with other markup
        languages</a>
        <ul>
          <li><a href="#s5.2.1">5.2.1 Use with EMMA</a></li>
          <li><a href="#s5.2.2">5.2.2 Use with SSML</a></li>
          <li><a href="#s5.2.3">5.2.3 Use with SMIL</a></li>
        </ul>
      </li>
    </ul>
  </li>
  <li><a href="#s6">6 References</a>
    <ul>
      <li><a href="#s6.1">6.1 Normative references</a></li>
      <li><a href="#s6.2">6.2 Informative references</a></li>
    </ul>
  </li>
  <li><a href="#s7">7 Acknowledgments</a></li>
  <li><a href="#changelog">Appendix A: Changes</a></li>
</ul>
</div>

<h2 id="s1">1 Introduction</h2>

<p><i>This section is informative.</i></p>

<p>Human emotions are increasingly understood to be a crucial aspect in
human-machine interactive systems. Especially for non-expert end users,
reactions to complex intelligent systems resemble social interactions,
involving feelings such as frustration, impatience, or helplessness if things
go wrong. Furthermore, technology is increasingly used to observe
human-to-human interactions, such as customer frustration monitoring in call
center applications. Dealing with these kinds of states in technological
systems requires a suitable representation, which should make the concepts and
descriptions developed in the affective sciences available for use in
technological contexts.</p>

<p>This report specifies Emotion Markup Language (EmotionML) 1.0, a markup
language designed to be usable in a broad variety of technological contexts
while reflecting concepts from the affective sciences. </p>

<h3 id="s1.1">1.1 Reasons for defining an Emotion Markup Language</h3>

<p>As for any standard format, the first and main goal of an EmotionML is
twofold: to allow a technological component to represent and process data, and
to enable interoperability between different technological components
processing the data.</p>

<p>Use cases for EmotionML can be grouped into three broad types:</p>
<ol>
  <li>Manual annotation of material involving emotionality, such as annotation
    of videos, of speech recordings, of faces, of texts, etc;</li>
  <li>Automatic recognition of emotions from sensors, including physiological
    sensors, speech recordings, facial expressions, etc., as well as from
    multi-modal combinations of sensors;</li>
  <li>Generation of emotion-related system responses, which may involve
    reasoning about the emotional implications of events, emotional prosody in
    synthetic speech, facial expressions and gestures of embodied agents or
    robots, the choice of music and colors of lighting in a room, etc.</li>
</ol>

<p>Interactive systems are likely to involve both analysis and generation of
emotion-related behavior; furthermore, systems are likely to benefit from data
that was manually annotated, be it as training data or for rule-based
modeling. Therefore, it is desirable to propose a single EmotionML that can be
used in all three contexts.</p>

<p>Concrete examples of existing technology that could apply EmotionML
include:</p>
<ul>
  <li>Opinion mining / sentiment analysis in Web 2.0, to automatically track
    customer's attitude regarding a product across blogs;</li>
  <li>Affective monitoring, such as ambient assisted living applications for
    the elderly, fear detection for surveillance purposes, or using wearable
    sensors to test customer satisfaction;</li>
  <li>Character design and control for games and virtual worlds;</li>
  <li>Social robots, such as guide robots engaging with visitors;</li>
  <li>Expressive speech synthesis, generating synthetic speech with different
    emotions, such as happy or sad, friendly or apologetic;</li>
  <li>Emotion recognition (e.g., for spotting angry customers in speech dialog
    systems);</li>
  <li>Support for people with disabilities, such as educational programs for
    people with autism.</li>
</ul>

<p>The Emotion Incubator Group has listed <a
href="http://www.w3.org/2005/Incubator/emotion/XGR-emotion/#AppendixUseCases">39
individual use cases for an EmotionML</a>.</p>

<p></p>

<p>A second reason for defining an EmotionML is the observation that ad hoc
attempts to deal with emotions and related states often lead people to make the
same mistakes that others have made before. The most typical mistake is to
model emotions as a small number of intense states such as anger, fear, joy,
and sadness; this choice is often made irrespective of the question whether
these states are the most appropriate for the intended application. Crucially,
the available alternatives that have been developed in the affective science
literature are not sufficiently known, resulting in dead-end situations after
the initial steps of work. Careful consideration of states to study and of
representations for describing them can help avoid such situations.</p>

<p>EmotionML makes scientific concepts of emotions practically applicable. This
can help potential users to identify the suitable representations for their
respective applications.</p>

<h3 id="s1.2">1.2 The challenge of defining a generally usable Emotion Markup
Language</h3>

<p>Any attempt to standardize the description of emotions using a finite set of
fixed descriptors is doomed to failure: even scientists cannot agree on the
number of relevant emotions, or on the names that should be given to them. Even
more basically, the list of emotion-related states that should be distinguished
varies depending on the application domain and the aspect of emotions to be
focused. Basically, the vocabulary needed depends on the context of use. On the
other hand, the basic structure of concepts is less controversial: it is
generally agreed that emotions involve triggers, appraisals, feelings,
expressive behavior including physiological changes, and action tendencies;
emotions in their entirety can be described in terms of categories or a small
number of dimensions; emotions have an intensity, and so on. For details, see
<a
href="http://www.w3.org/2005/Incubator/emotion/XGR-emotion/#ScientificDescriptions">Scientific
Descriptions of Emotions</a> in the Final Report of the <a
href="#ref-emotion-xg">Emotion Incubator Group</a>.</p>

<p>Given this lack of agreement on descriptors in the field, the only practical
way of defining an EmotionML is the definition of possible structural elements
and their valid child elements and attributes, but to allow users to "plug in"
vocabularies that they consider appropriate for their work. A separate W3C
Working Draft complements this specification to provide a central repository of
[<a href="#ref-emotion-voc">Vocabularies for EmotionML</a>] which can serve as
a starting point; where the vocabularies listed there seem inappropriate, users
can create their custom vocabularies.</p>

<p>An additional challenge lies in the aim to provide a generally usable
markup, as the requirements arising from the three different use cases
(annotation, recognition, and generation) are rather different. Whereas manual
annotation tends to require all the fine-grained distinctions considered in the
scientific literature, automatic recognition systems can usually distinguish
only a very small number of different states.</p>

<p>For the reasons outlined here, it is clear that there is an inevitable
tension between flexibility and interoperability, which need to be weighed in
the formulation of an EmotionML. The guiding principle in the following
specification has been to provide a choice only where it is needed, and to
propose reasonable default options for every choice.</p>

<h3 id="s1.3">1.3 Glossary of terms</h3>

<p>The terms related to emotions are not used consistently, neither in common
use nor in the scientific literature. The following glossary describes the
intended meaning of terms in this document.</p>

<p></p>
<dl>
  <dt id="term-action-tendency">Action tendency</dt>
    <dd>Emotions have a strong influence on the motivational state of a
      subject. Emotion theory associates emotions to a small set of so-called
      action tendencies, e.g. avoidance (relates to fear), rejecting (disgust)
      etc. Action tendencies can be viewed as a link between the outcome of an
      <a href="#term-appraisal">appraisal</a> process and actual actions. </dd>
  <dt id="term-affect">Affect / Affective state</dt>
    <dd>In the scientific literature, the term "affect" is often used as a
      general term covering a range of phenomena called "affective states",
      including emotions, moods, attitudes, etc. Proponents of the term
      consider it to be more generic than "emotion", in the sense that it
      covers both acute and long-term, specific and unspecific states. In this
      report, the term "affect" is avoided so that the scope of the intended
      markup language is more easily accessible to the non-expert; the term
      "affective state" is used interchangeably with "<a
      href="#term-emotion-related-state">emotion-related state</a>".</dd>
  <dt id="term-appraisal">Appraisal</dt>
    <dd>The term "appraisal" is used in the scientific literature to describe
      the evaluation process leading to an emotional response. Triggered by an
      "emotion-eliciting event", an individual carries out an automatic,
      subjective assessment of the event, in order to determine the relevance
      of the event to the individual. This assessment is carried out along a
      number of "appraisal dimensions" such as the novelty, pleasantness or
      goal conduciveness of the event. </dd>
  <dt id="term-attitude">Attitude</dt>
    <dd>In psychology, "attitude" is related to the global evaluation of an
      object, such as a person, an object, oneself, etc. Attitude is considered
      to include an emotional component, as well as cognition and behavior.
      However, the term "attitude" is sometimes used with slightly different
      meanings, such as speaking style ("he said it with a certain attitude")
      or more generally personal lifestyle ("she has quite an attitude").
      Because of this ambiguity, this specification avoids the term.</dd>
  <dt id="term-emotion">Emotion</dt>
    <dd>In this report, the term "emotion" is used in a very broad sense,
      covering both intense and weak states, short and long term, with and
      without event focus. This meaning is intended to reflect the
      understanding of the term "emotion" by the general public. In the
      scientific literature on emotion theories, the term "emotion" or
      "fullblown emotion" refers to intense states with a strong focus on
      current events, often in the context of the survival-benefiting function
      of behavioral responses such as "fight or flight". This reading of the
      term seems inappropriate for the vast majority of human-machine
      interaction contexts, in which more subtle states dominate; therefore,
      where this reading is intended, the term "<a
      href="#term-fullblown-emotion">fullblown emotion</a>" is used in this
      report.</dd>
  <dt id="term-emotion-related-state">Emotion-related state</dt>
    <dd>A cover term for the broad range of phenomena intended to be covered by
      this specification. In the scientific literature, several kinds of
      emotion-related or affective states are distinguished, see <a
      href="http://www.w3.org/2005/Incubator/emotion/XGR-emotion/#EmotionRelatedStates">Emotions
      and related states</a> in the final report of the <a
      href="#ref-emotion-xg">Emotion Incubator Group</a>.</dd>
  <dt id="term-emotion-dimensions">Emotion dimensions</dt>
    <dd>A small number of continuous scales describing the most basic
      properties of an emotion. Often three dimensions are used: valence
      (sometimes named pleasure), arousal (or activity/activation), and potency
      (sometimes called control, power or dominance). However, sometimes two,
      or more than three dimensions are used.</dd>
  <dt id="term-fullblown-emotion">Fullblown emotion</dt>
    <dd>Intense states with a strong focus on current events, often in the
      context of the survival-benefiting function of behavioral responses such
      as "fight or flight".</dd>
</dl>

<div class="comment">
</div>

<h2 id="s2">2 Elements of Emotion Markup</h2>

<p>The following sections describe the syntax of the main elements of
EmotionML.</p>

<h3 id="s2.1">2.1 Document structure</h3>

<h4 id="s2.1.1">2.1.1 Document root: The <code>&lt;emotionml&gt;</code>
element</h4>

<table class="defn" cellspacing="0" cellpadding="5" width="98%"
summary="property definition">
  <tbody>
    <tr>
      <th>Annotation</th>
      <td><code>&lt;emotionml&gt;</code></td>
    </tr>
    <tr>
      <th>Definition</th>
      <td>The root element of an EmotionML document.</td>
    </tr>
    <tr>
      <th>Children</th>
      <td>The element MAY contain one or more <code>&lt;emotion&gt;</code>
        elements. It MAY contain a single <a
        href="#s2.3.3"><code>&lt;info&gt;</code></a> element. It MAY contain
        one or more <a href="#s3.1.1"><code>&lt;vocabulary&gt;</code></a>
        elements.</td>
    </tr>
    <tr>
      <th>Attributes</th>
      <td><ul>
          <li><strong>Required</strong>:
            <ul>
              <li>Namespace declaration for EmotionML, see <a
                href="#s4.1">EmotionML namespace</a>.</li>
              <li><code>version</code> indicates the version of the
                specification to be used for the document and MUST have the
                value "1.0".</li>
            </ul>
          </li>
          <li><strong>Optional</strong>:
            <ul>
              <li><code>category-set</code>, <code>dimension-set</code>,
                <code>appraisal-set</code> and <code>action-tendency-set</code>
                indicate default emotion vocabularies used in an EmotionML
                document. Any of these attributes used at document level makes
                optional the use of the same attribute in any
                <code>&lt;emotion&gt;</code> elements within the document; the
                document-level attribute determines the emotion vocabularies
                used for any <code>&lt;emotion&gt;</code> elements for which
                the respective attribute is not locally specified. The
                attributes are of type <code>xsd:anyURI</code> and MUST point
                to a definition of an emotion vocabulary as specified in <a
                href="#s3">Defining vocabularies for representing
              emotions</a>.</li>
            </ul>
          </li>
        </ul>
      </td>
    </tr>
    <tr>
      <th>Occurrence</th>
      <td>This is the root element -- it cannot occur as a child of any other
        EmotionML element.</td>
    </tr>
  </tbody>
</table>

<p><code>&lt;emotionml&gt;</code> is the root element of a standalone EmotionML
document. It MAY contain a single <code>&lt;info&gt;</code> element, providing
document-level metadata.</p>

<p>The <code>&lt;emotionml&gt;</code> element MUST define the <a
href="#s4.1">EmotionML namespace</a>.</p>

<p>Standalone EmotionML documents usually serve one or both of the following
two purposes:</p>
<ol>
  <li>to wrap a number of <a href="#s2.1.2"><code>&lt;emotion&gt;</code></a>
    elements into a single document;</li>
  <li>to define <a href="#s3">emotion vocabularies</a> for use with <a
    href="#s2.1.2"><code>&lt;emotion&gt;</code></a> annotations in the same or
    other documents.</li>
</ol>

<p>Example:</p>
<pre class="example">&lt;emotionml version="1.0" xmlns="http://www.w3.org/2009/10/emotionml"&gt;
 ...
&lt;/emotionml&gt;</pre>

<p>or</p>
<pre class="example">&lt;em:emotionml version="1.0" xmlns:em="http://www.w3.org/2009/10/emotionml"&gt;
 ...
&lt;/em:emotionml&gt;</pre>

<p></p>

<p>Note: One of the envisaged uses of EmotionML is to be used in the context of
other markup languages. In such cases, there will be no
<code>&lt;emotionml&gt;</code> root element, but <code>&lt;emotion&gt;</code>
elements will be used directly in other markup -- see <a href="#s5.2">Examples
of possible use with other markup languages</a>.</p>

<h4 id="s2.1.2">2.1.2 A single emotion annotation: The
<code>&lt;emotion&gt;</code> element</h4>

<table class="defn" cellspacing="0" cellpadding="5" width="98%"
summary="property definition">
  <tbody>
    <tr>
      <th>Annotation</th>
      <td><code>&lt;emotion&gt;</code></td>
    </tr>
    <tr>
      <th>Definition</th>
      <td>This element represents a single emotion annotation.</td>
    </tr>
    <tr>
      <th>Children</th>
      <td>All children are optional. However, at least one of
        <code>&lt;category&gt;</code>, <code>&lt;dimension&gt;</code>,
        <code>&lt;appraisal&gt;</code>, <code>&lt;action-tendency&gt;</code>
        MUST occur.

        <p>If present, the following child element can occur only once:
        <code>&lt;info&gt;</code>.</p>

        <p>If present, the following child elements may occur one or more
        times: <code>&lt;category&gt;</code>; <code>&lt;dimension&gt;</code>;
        <code>&lt;appraisal&gt;</code>; <code>&lt;action-tendency&gt;</code>;
        <code>&lt;reference&gt;</code>.</p>

        <p>There are no constraints on the combinations of children that are
        allowed. There are no constraints on the order in which children
        occur.</p>
      </td>
    </tr>
    <tr>
      <th>Attributes</th>
      <td><ul>
          <li><strong>Required</strong>:
            <ul>
              <li><code>category-set</code>, <code>dimension-set</code>,
                <code>appraisal-set</code> and <code>action-tendency-set</code>
                indicate the emotion vocabularies used in this
                <code>&lt;emotion&gt;</code>. The attributes are of type
                <code>xsd:anyURI</code> and MUST point to a definition of an
                emotion vocabulary as specified in <a href="#s3">Defining
                vocabularies for representing emotions</a>. The attributes are
                required as follows:
                <ul>
                  <li>if the <code>&lt;emotion&gt;</code> element has a child
                    element <code>&lt;category&gt;</code>, the
                    <code>category-set</code> attribute is required and must
                    point to the definition of a category vocabulary;</li>
                  <li>if the <code>&lt;emotion&gt;</code> element has a child
                    element <code>&lt;dimension&gt;</code>, the
                    <code>dimension-set</code> attribute is required and must
                    point to the definition of a dimension vocabulary;</li>
                  <li>if the <code>&lt;emotion&gt;</code> element has a child
                    element <code>&lt;appraisal&gt;</code>, the
                    <code>appraisal-set</code> attribute is required and must
                    point to the definition of an appraisal vocabulary;</li>
                  <li>if the <code>&lt;emotion&gt;</code> element has a child
                    element <code>&lt;action-tendency&gt;</code>, the
                    <code>action-tendency-set</code> attribute is required and
                    must point to the definition of an action tendency
                    vocabulary.</li>
                </ul>
                <p>The attribute values default to the values of any attributes
                of same name on an enclosing <code>&lt;emotionml&gt;</code>
                element. Any of these attributes used at document level makes
                optional the use of the same attribute in any
                <code>&lt;emotion&gt;</code> elements. </p>
              </li>
              <li><code>version</code> indicates the version of the
                specification to be used for the <code>&lt;emotion&gt;</code>
                and its descendants. Its value defaults to "1.0". </li>
            </ul>
          </li>
          <li><strong>Optional</strong>:
            <ul>
              <li><code>id</code>, a unique identifier for the emotion, of type
                <code>xsd:ID</code>.</li>
              <li><a href="#s2.4.2.1"><code>start</code></a>, <a
                href="#s2.4.2.1"><code>end</code></a>, <a
                href="#s2.4.2.2"><code>duration</code></a>, <a
                href="#s2.4.2.3"><code>time-ref-uri</code></a>, <a
                href="#s2.4.2.3"><code>time-ref-anchor-point</code></a> and <a
                href="#s2.4.2.3"><code>offset-to-start</code></a> provide
                information about the times at which an emotion happened, as
                defined in <a href="#s2.4.2">Timestamps</a>.</li>
              <li><a href="#s2.3.2"><code>expressed-through</code></a>, the
                modality, or list of modalities, through which the emotion is
                expressed. </li>
            </ul>
          </li>
        </ul>
      </td>
    </tr>
    <tr>
      <th>Occurrence</th>
      <td>as a child of <code>&lt;emotionml&gt;</code>, or in any markup using
        EmotionML.</td>
    </tr>
  </tbody>
</table>

<p>The <code>&lt;emotion&gt;</code> element represents an individual emotion
annotation. No matter how simple or complex its substructure is, it represents
a single statement about the emotional content of some annotated item. Where
several statements about the emotion in a certain context are to be made,
several <code>&lt;emotion&gt;</code> elements MUST be used. See <a
href="#s5.1">Examples of emotion annotation</a> for illustrations of this
issue.</p>

<p>An <code>&lt;emotion&gt;</code> element MAY have an <code>id</code>
attribute, allowing for a unique reference to the individual emotion
annotation. Since the <code>&lt;emotion&gt;</code> annotation is an atomic
statement about the emotion, it is inappropriate to refer to individual emotion
representations such as <code>&lt;category&gt;</code>,
<code>&lt;dimension&gt;</code>, <code>&lt;appraisal&gt;</code>,
<code>&lt;action-tendency&gt;</code> or their children directly. For this
reason, these elements do not allow for an <code>id</code> attribute.</p>

<p>Whereas it is possible to use <code>&lt;emotion&gt;</code> elements in a
standalone <code>&lt;emotionml&gt;</code> document, a typical use case is
expected to be embedding an <code>&lt;emotion&gt;</code> into some other markup
-- see <a href="#s5.2">Examples of possible use with other markup
languages</a>.</p>

<h3 id="s2.2">2.2 Representations of emotions and related states</h3>

<h4 id="s2.2.1">2.2.1 The <code>&lt;category&gt;</code> element</h4>

<table class="defn" cellspacing="0" cellpadding="5" width="98%"
summary="property definition">
  <tbody>
    <tr>
      <th>Annotation</th>
      <td><code>&lt;category&gt;</code></td>
    </tr>
    <tr>
      <th>Definition</th>
      <td>Description of an emotion or a related state using a category.</td>
    </tr>
    <tr>
      <th>Children</th>
      <td><a href="#s2.5.2"><code>&lt;trace&gt;</code></a>: A
        <code>&lt;category&gt;</code> MAY contain either a <code>value</code>
        attribute or a <code>&lt;trace&gt;</code> element.</td>
    </tr>
    <tr>
      <th>Attributes</th>
      <td><ul>
          <li><strong>Required</strong>:
            <ul>
              <li><code>name</code>, the name of the category, which must be
                contained in the set of categories identified in the enclosing
                <a href="#s2.1.1"><code>&lt;emotionml&gt;</code></a> or <a
                href="#s2.1.2"><code>&lt;emotion&gt;</code></a> element's
                <code>category-set</code> attribute.</li>
            </ul>
          </li>
          <li><strong>Optional</strong>:
            <ul>
              <li><a href="#s2.5.1"><code>value</code></a>: A
                <code>&lt;category&gt;</code> MAY contain either a
                <code>value</code> attribute or a <code>&lt;trace&gt;</code>
                element.</li>
              <li><a href="#s2.3.1"><code>confidence</code></a>, the
                annotator's confidence that the annotation given for this
                category is correct.</li>
            </ul>
          </li>
        </ul>
      </td>
    </tr>
    <tr>
      <th>Occurrence</th>
      <td>One or more <code>&lt;category&gt;</code> elements MAY occur as a
        child of <code>&lt;emotion&gt;</code>. For any given category name in
        the set, zero or one occurrence is allowed within an
        <code>&lt;emotion&gt;</code> element.</td>
    </tr>
  </tbody>
</table>

<p><code>&lt;category&gt;</code> describes an <a
href="#term-emotion">emotion</a> or a <a
href="#term-emotion-related-state">related state</a> in terms of a category
name, given as the value of the <code>name</code> attribute. The name MUST
belong to a clearly-identified set of category names, which MUST be defined
according to <a href="#s3">Defining vocabularies for representing
emotions</a>.</p>

<p>The set of legal values of the <code>name</code> attribute is indicated in
the <code>category-set</code> attribute of the enclosing <a
href="#s2.1.1"><code>&lt;emotion&gt;</code></a> or <a
href="#s2.1.1"><code>&lt;emotionml&gt;</code></a> element. Different sets can
be used, depending on the requirements of the use case. In particular,
different types of <a href="#term-emotion-related-state">emotion-related</a> /
<a href="#term-affect">affective states</a> can be annotated by using
appropriate value sets. </p>

<p>The intensity of an emotion category MAY be specified as a <a
href="#s2.5">Scale value</a>, either as a static value in the <a
href="#s2.5.1"><code>value</code></a> attribute, or as a dynamic trace over
time using the <a href="#s2.5.2"><code>&lt;trace&gt;</code></a> element. </p>

<p>Examples:</p>

<p>In the following example, the emotion category "satisfaction" is being
annotated; it must be contained in the definition of the emotion category
vocabulary located at http://www.w3.org/TR/emotion-voc/xml#everyday-categories,
which is one of the category vocabularies provided in [<a
href="#ref-emotion-voc">Vocabularies for EmotionML</a>].</p>
<pre class="example">&lt;emotion category-set="http://www.w3.org/TR/emotion-voc/xml#everyday-categories"&gt;
    &lt;category name="satisfaction"/&gt;
&lt;/emotion&gt;</pre>

<p>The following is an annotation of an interpersonal stance "distant" which
must be defined in the custom category set at the URI given in the category-set
attribute: </p>
<pre class="example">&lt;emotion category-set="http://www.example.com/custom/category/interpersonal-stances.xml"&gt;
    &lt;category name="distant"/&gt;
&lt;/emotion&gt;</pre>

<p>In the following example, an emotion is described by several categories,
each being present with different values of intensity. The category set used is
the "big six" set described in [<a href="#ref-emotion-voc">Vocabularies for
EmotionML</a>].</p>
<pre class="example">&lt;emotion category-set="http://www.w3.org/TR/emotion-voc/xml#big6"&gt;
    &lt;category name="sadness" value="0.3"/&gt;
    &lt;category name="anger" value="0.8"/&gt;
    &lt;category name="fear" value="0.3"/&gt;
&lt;/emotion&gt;</pre>

<h4 id="s2.2.2">2.2.2 The <code>&lt;dimension&gt;</code> element</h4>

<table class="defn" cellspacing="0" cellpadding="5" width="98%"
summary="property definition">
  <tbody>
    <tr>
      <th>Annotation</th>
      <td><code>&lt;dimension&gt;</code></td>
    </tr>
    <tr>
      <th>Definition</th>
      <td>One or more <code>&lt;dimension&gt;</code> elements jointly describe
        an emotion or a related state according to an emotion dimension
        vocabulary.</td>
    </tr>
    <tr>
      <th>Children</th>
      <td><a href="#s2.5.2"><code>&lt;trace&gt;</code></a>: A
        <code>&lt;dimension&gt;</code> MUST contain either a <code>value</code>
        attribute or a <code>&lt;trace&gt;</code> element.</td>
    </tr>
    <tr>
      <th>Attributes</th>
      <td><ul>
          <li><strong>Required</strong>:
            <ul>
              <li><code>name</code>, the name of the dimension, which must be
                contained in the set of dimensions identified in the enclosing
                <a href="#s2.1.1"><code>&lt;emotionml&gt;</code></a> or <a
                href="#s2.1.2"><code>&lt;emotion&gt;</code></a> element's
                <code>dimension-set</code> attribute.</li>
              <li><a href="#s2.5.1"><code>value</code></a>: A
                <code>&lt;dimension&gt;</code> MUST contain either a
                <code>value</code> attribute or a <code>&lt;trace&gt;</code>
                element.</li>
            </ul>
          </li>
          <li><strong>Optional</strong>:
            <ul>
              <li><a href="#s2.3.1"><code>confidence</code></a>, the
                annotator's confidence that the annotation given for this
                dimension is correct.</li>
            </ul>
          </li>
        </ul>
      </td>
    </tr>
    <tr>
      <th>Occurrence</th>
      <td><code>&lt;dimension&gt;</code> elements occur as children of
        <code>&lt;emotion&gt;</code>. For any given dimension name in the set,
        zero or one occurrence is allowed within an
        <code>&lt;emotion&gt;</code> element.</td>
    </tr>
  </tbody>
</table>

<p>One or more <code>&lt;dimension&gt;</code> elements jointly describe an <a
href="#term-emotion">emotion</a> or a <a
href="#term-emotion-related-state">related state</a> in terms of a set of <a
href="#term-emotion-dimensions">emotion dimensions</a>. The names of the
emotion dimensions MUST belong to a clearly-identified set of dimension names,
which MUST be defined according to <a href="#s3">Defining vocabularies for
representing emotions</a>.</p>

<p>The set of legal values of the <code>name</code> attribute is indicated in
the <code>dimension-set</code> attribute of the enclosing <a
href="#s2.1.1"><code>&lt;emotionml&gt;</code></a> or <a
href="#s2.1.2"><code>&lt;emotion&gt;</code></a> element. Different sets can be
used, depending on the requirements of the use case.</p>

<p>The position on an emotion dimension MUST be specified as a <a
href="#s2.5">Scale value</a>, either as a static value in the <a
href="#s2.5.1"><code>value</code></a> attribute, or as a dynamic trace over
time using the <a href="#s2.5.2"><code>&lt;trace&gt;</code></a> element. </p>

<p>Examples:</p>

<p>One of the most widespread sets of emotion dimensions used (sometimes by
different names) is the combination of valence, arousal and potency. The
following example is a state of rather low arousal, very positive valence, and
high potency -- in other words, a relaxed, positive state with a feeling of
being in control of the situation. The example uses the
Pleasure-Arousal-Dominance (PAD) vocabulary from [<a
href="#ref-emotion-voc">Vocabularies for EmotionML</a>]:</p>
<pre class="example">&lt;emotion dimension-set="http://www.w3.org/TR/emotion-voc/xml#pad-dimensions"&gt;
    &lt;dimension name="arousal" value="0.3"/&gt;&lt;!-- lower-than-average arousal --&gt;
    &lt;dimension name="pleasure" value="0.9"/&gt;&lt;!-- very high positive valence --&gt;
    &lt;dimension name="dominance" value="0.8"/&gt;&lt;!-- relatively high potency    --&gt;
&lt;/emotion&gt;</pre>

<p>In some use cases, custom sets of application-specific dimensions will be
required. The following example uses a custom set of dimensions, defining a
single dimension "friendliness".</p>
<pre class="example">&lt;emotion dimension-set="http://www.example.com/custom/dimension/friendliness.xml"&gt;
    &lt;dimension name="friendliness" value="0.2"/&gt;&lt;!-- a pretty unfriendly person --&gt;
&lt;/emotion&gt;</pre>

<p>The usual way to represent the intensity of an emotion would be the
<code>value</code> attribute of a <code>&lt;category&gt;</code>. However, if
only the intensity of an emotion is annotated, but not its nature, this can be
done by using an "intensity" dimension. Thus, an emotional state's "strength"
or "intensity" can be described independently from categorical or dimensional
descriptions, as shown by the following example.</p>
<pre class="example">&lt;emotion dimension-set="http://www.w3.org/TR/emotion-voc/xml#intensity-dimension"&gt;
    &lt;dimension name="intensity" value="0.2"/&gt;&lt;!-- not in a strong emotional state --&gt;
&lt;/emotion&gt;</pre>

<p></p>

<h4 id="s2.2.3">2.2.3 The <code>&lt;appraisal&gt;</code> element</h4>

<table class="defn" cellspacing="0" cellpadding="5" width="98%"
summary="property definition">
  <tbody>
    <tr>
      <th>Annotation</th>
      <td><code>&lt;appraisal&gt;</code></td>
    </tr>
    <tr>
      <th>Definition</th>
      <td>One or more <code>&lt;appraisal&gt;</code> elements jointly describe
        an emotion or a related state according to an emotion appraisal
        vocabulary.</td>
    </tr>
    <tr>
      <th>Children</th>
      <td><a href="#s2.5.2"><code>&lt;trace&gt;</code></a>: An
        <code>&lt;appraisal&gt;</code> MAY contain either a <code>value</code>
        attribute or a <code>&lt;trace&gt;</code> element. </td>
    </tr>
    <tr>
      <th>Attributes</th>
      <td><ul>
          <li><strong>Required</strong>:
            <ul>
              <li><code>name</code>, the name of the appraisal, which must be
                contained in the set of appraisals identified in the enclosing
                <a href="#s2.1.1"><code>&lt;emotionml&gt;</code></a> or <a
                href="#s2.1.2"><code>&lt;emotion&gt;</code></a> element's
                <code>appraisal-set</code> attribute.</li>
            </ul>
          </li>
          <li><strong>Optional</strong>:
            <ul>
              <li><a href="#s2.5.1"><code>value</code></a>: An
                <code>&lt;appraisal&gt;</code> MAY contain either a
                <code>value</code> attribute or a <code>&lt;trace&gt;</code>
                element. </li>
              <li><a href="#s2.3.1"><code>confidence</code></a>, the
                annotator's confidence that the annotation given for this
                appraisal is correct.</li>
            </ul>
          </li>
        </ul>
      </td>
    </tr>
    <tr>
      <th>Occurrence</th>
      <td><code>&lt;appraisal&gt;</code> elements occur as children of
        <code>&lt;emotion&gt;</code>. For any given appraisal name in the set,
        zero or one occurrence is allowed within an
        <code>&lt;emotion&gt;</code> element.</td>
    </tr>
  </tbody>
</table>

<p>One or more <code>&lt;appraisal&gt;</code> elements jointly describe an <a
href="#term-emotion">emotion</a> or a <a
href="#term-emotion-related-state">related state</a> in terms of a set of <a
href="#term-appraisal">appraisals</a>. The names of the appraisals MUST belong
to a clearly-identified set of appraisal names, which MUST be defined according
to <a href="#s3">Defining vocabularies for representing emotions</a>.</p>

<p>The set of legal values of the <code>name</code> attribute is indicated in
the <code>appraisal-set</code> attribute of the enclosing
<code>&lt;emotion&gt;</code> element. Different sets can be used, depending on
the requirements of the use case.</p>

<p>The degree to which an appraisal is present MAY be specified as a <a
href="#s2.5">Scale value</a>, either as a static value in the <a
href="#s2.5.1"><code>value</code></a> attribute, or as a dynamic trace over
time using the <a href="#s2.5.2"><code>&lt;trace&gt;</code></a> element. </p>

<p>Examples:</p>

<p>One of the most widespread sets of emotion appraisals used is the appraisals
set proposed by Klaus Scherer, covering aspects of novelty, intrinsic
pleasantness, goal/need significance, coping potential, and norm/self
compatibility. Another very widespread set of emotion appraisals, used in
particular in computational models of emotion, is the OCC set of appraisals (<a
href="#ref-Ortony1988">Ortony et al., 1988</a>), which includes the
consequences of events for oneself or for others, the actions of others and the
perception of objects. Using Scherer's appraisals from [<a
href="#ref-emotion-voc">Vocabularies for EmotionML</a>], the following example
is a state arising from the evaluation of an unpredicted and quite unpleasant
event:</p>
<pre class="example">&lt;emotion appraisal-set="http://www.w3.org/TR/emotion-voc/xml#scherer-appraisals"&gt;
    &lt;appraisal name="suddenness" value="0.8"/&gt;
    &lt;appraisal name="intrinsic-pleasantness" value="0.2"/&gt;
&lt;/emotion&gt;</pre>

<p>In some use cases, custom sets of application-specific appraisals will be
required. The following example uses a custom set of appraisals, defining the
single appraisal "likelihood".</p>
<pre class="example">&lt;emotion appraisal-set="http://www.example.com/custom/appraisal/likelihood.xml"&gt;
    &lt;appraisal name="likelihood" value="0.8"/&gt;&lt;!-- a very predictable event --&gt;
&lt;/emotion&gt;</pre>

<p></p>

<h4 id="s2.2.4">2.2.4 The <code>&lt;action-tendency&gt;</code> element</h4>

<table class="defn" cellspacing="0" cellpadding="5" width="98%"
summary="property definition">
  <tbody>
    <tr>
      <th>Annotation</th>
      <td><code>&lt;action-tendency&gt;</code></td>
    </tr>
    <tr>
      <th>Definition</th>
      <td>One or more <code>&lt;action-tendency&gt;</code> elements jointly
        describe an emotion or a related state according to an emotion action
        tendency vocabulary.</td>
    </tr>
    <tr>
      <th>Children</th>
      <td><a href="#s2.5.2"><code>&lt;trace&gt;</code></a>: An
        <code>&lt;action-tendency&gt;</code> MAY contain either a
        <code>value</code> attribute or a <code>&lt;trace&gt;</code> element.
      </td>
    </tr>
    <tr>
      <th>Attributes</th>
      <td><ul>
          <li><strong>Required</strong>:
            <ul>
              <li><code>name</code>, the name of the action tendency, which
                must be contained in the set of action tendencies identified in
                the enclosing <a
                href="#s2.1.1"><code>&lt;emotionml&gt;</code></a> or <a
                href="#s2.1.2"><code>&lt;emotion&gt;</code></a> element's
                <code>action-tendency-set</code> attribute.</li>
            </ul>
          </li>
          <li><strong>Optional</strong>:
            <ul>
              <li><a href="#s2.5.1"><code>value</code></a>: An
                <code>&lt;action-tendency&gt;</code> MAY contain either a
                <code>value</code> attribute or a <code>&lt;trace&gt;</code>
                element. </li>
              <li><a href="#s2.3.1"><code>confidence</code></a>, the
                annotator's confidence that the annotation given for this
                action tendency is correct.</li>
            </ul>
          </li>
        </ul>
      </td>
    </tr>
    <tr>
      <th>Occurrence</th>
      <td><code>&lt;action-tendency&gt;</code> elements occur as children of
        <code>&lt;emotion&gt;</code>. For any given action tendency name in the
        set, zero or one occurrence is allowed within an
        <code>&lt;emotion&gt;</code> element.</td>
    </tr>
  </tbody>
</table>

<p>One or more <code>&lt;action-tendency&gt;</code> elements jointly describe
an <a href="#term-emotion">emotion</a> or a <a
href="#term-emotion-related-state">related state</a> in terms of a set of <a
href="#term-action-tendency">action tendencies</a>. The names of the action
tendencies MUST belong to a clearly-identified set of action tendency names,
which MUST be defined according to <a href="#s3">Defining vocabularies for
representing emotions</a>.</p>

<p>The set of legal values of the <code>name</code> attribute is indicated in
the <code>action-tendency-set</code> attribute of the enclosing
<code>&lt;emotion&gt;</code> element. Different sets can be used, depending on
the requirements of the use case.</p>

<p>The degree to which an action tendency is present MAY be specified as a <a
href="#s2.5">Scale value</a>, either as a static value in the <a
href="#s2.5.1"><code>value</code></a> attribute, or as a dynamic trace over
time using the <a href="#s2.5.2"><code>&lt;trace&gt;</code></a> element. </p>

<p>Examples:</p>

<p>One well known use of action tendencies is by N. Frijda. This model uses a
number of action tendencies that are low level, diffuse behaviors from which
more concrete actions could be determined. It is provided in [<a
href="#ref-emotion-voc">Vocabularies for EmotionML</a>]. An example of someone
attempting to attract someone they like by being confident, strong and
attentive might look like this: </p>
<pre class="example">&lt;emotion action-tendency-set="http://www.w3.org/TR/emotion-voc/xml#frijda-action-tendencies"&gt;
    &lt;action-tendency name="approach" value="0.7"/&gt;   &lt;!-- get close --&gt;
    &lt;action-tendency name="being-with" value="0.8"/&gt; &lt;!-- be happy --&gt;
    &lt;action-tendency name="attending" value="0.7"/&gt;  &lt;!-- pay attention --&gt;
    &lt;action-tendency name="dominating" value="0.7"/&gt; &lt;!-- be assertive --&gt;
&lt;/emotion&gt;</pre>

<p>In some use cases, custom sets of application-specific action tendencies
will be required. The following example shows control values for a robot who
works in a factory and uses a custom set of action-tendencies, defining example
actions for a robot. In the example, the robot has very low battery, so it
needs to get ready to charge its battery and stop its work of picking up
boxes.</p>
<pre class="example">&lt;emotion action-tendency-set="http://www.example.com/custom/action/robot.xml"&gt;
    &lt;action-tendency name="charge-battery" value="0.9"/&gt; &lt;!-- need to charge battery soon --&gt;
    &lt;action-tendency name="pickup-boxes" value="0.3"/&gt;   &lt;!-- feeling tired, avoid work --&gt;
&lt;/emotion&gt;</pre>

<p></p>

<h3 id="s2.3">2.3 Meta-information</h3>

<h4 id="s2.3.1">2.3.1 The <code>confidence</code> attribute</h4>

<table class="defn" cellspacing="0" cellpadding="5" width="98%"
summary="property definition">
  <tbody>
    <tr>
      <th>Annotation</th>
      <td><code>confidence</code></td>
    </tr>
    <tr>
      <th>Definition</th>
      <td>A representation of the degree of confidence or probability that a
        certain element of the representation is correct. </td>
    </tr>
    <tr>
      <th>Occurrence</th>
      <td>An optional attribute of <a
        href="#s2.2.1"><code>&lt;category&gt;</code></a>, <a
        href="#s2.2.2"><code>&lt;dimension&gt;</code></a>, <a
        href="#s2.2.3"><code>&lt;appraisal&gt;</code></a> and <a
        href="#s2.2.4"><code>&lt;action-tendency&gt;</code></a> elements.</td>
    </tr>
  </tbody>
</table>

<p></p>

<p>Confidence MAY be indicated separately for each of the <a
href="#s2.2">Representations of emotions and related states</a>. For example,
the confidence that the <code>&lt;category&gt;</code> is assumed correctly is
independent from the confidence that the position on a dimension is correctly
indicated. </p>

<p>Rooted in the tradition of statistics a confidence is given in an interval
from 0 to 1, resembling a probability. Insofar, the confidence is a <a
href="#s2.5">Scale value</a>.</p>

<p>Examples:</p>

<p>In the following, one simple example is provided for each element that can
carry a <code>confidence</code> attribute. </p>

<p>The first example indicates a very high confidence that surprise is the
emotion to annotate.</p>
<pre class="example">&lt;emotion category-set="http://www.w3.org/TR/emotion-voc/xml#big6"&gt;
    &lt;category name="surprise" confidence="0.95"/&gt;
&lt;/emotion</pre>

<p>The next example illustrates using <code>confidence</code> to indicate that
the annotation of high arousal is probably correct, but the annotation of
slightly positive pleasure may or may not be correct.</p>
<pre class="example">&lt;emotion dimension-set="http://www.w3.org/TR/emotion-voc/xml#pad-dimensions"&gt;
    &lt;dimension name="arousal" value="0.8" confidence="0.9"/&gt;
    &lt;dimension name="pleasure" value="0.6" confidence="0.3"/&gt;
&lt;/emotion&gt;</pre>

<p>Finally, an example for the case of intensity: A high confidence is given
that the emotion has a low intensity.</p>
<pre class="example"> &lt;emotion dimension-set="http://www.w3.org/TR/emotion-voc/xml#intensity-dimension"&gt;
    &lt;dimension name="intensity" value="0.1" confidence="0.8"/&gt;
&lt;/emotion&gt;</pre>

<p>Note that, as stated, obviously an emotional annotation can be a combination
of the above, as in the following example: the intensity of the emotion is
quite probably low, but if we have to guess, we would say the emotion is
boredom.</p>
<pre class="example">&lt;emotion category-set="http://www.w3.org/TR/emotion-voc/xml#everyday-categories"
         dimension-set="http://www.w3.org/TR/emotion-voc/xml#intensity-dimension"&gt;
    &lt;category name="bored" confidence="0.1"/&gt;
    &lt;dimension name="intensity" value="0.1" confidence="0.8"/&gt;
&lt;/emotion&gt;</pre>

<p></p>

<h4 id="s2.3.2">2.3.2 The <code>expressed-through</code> attribute</h4>

<table class="defn" cellspacing="0" cellpadding="5" width="98%"
summary="property definition">
  <tbody>
    <tr>
      <th>Annotation</th>
      <td><code>expressed-through</code></td>
    </tr>
    <tr>
      <th>Definition</th>
      <td>The modality, or list of modalities, through which the emotion is
        expressed. An attribute of type <code>xsd:nmtokens</code> which
        contains a space delimited set of values from an open set of values
        including: {<code>gaze</code>, <code>face</code>, <code>head</code>,
        <code>torso</code>, <code>gesture</code>, <code>leg</code>,
        <code>voice</code>, <code>text</code>, <code>locomotion</code>,
        <code>posture</code>, <code>physiology</code>, ...}.</td>
    </tr>
    <tr>
      <th>Occurrence</th>
      <td>An optional attribute of <a
        href="#s2.1.2"><code>&lt;emotion&gt;</code></a> elements.</td>
    </tr>
  </tbody>
</table>

<p>The <code>expressed-through</code> attribute describes the modality through
which an emotion is produced, usually by a human being. It is not the technical
modality by which it was detected, e.g. "face" rather than "camera" and "voice"
rather than "microphone". The <code>expressed-through</code> attribute is
agnostic about the use case: when detecting emotion, it represents the modality
from which the emotion has been detected; when generating emotion-related
system behavior, it represents the modality through which the emotion is to be
expressed.</p>

<p>The list of values provided covers a broad range of modalities through which
emotions may be expressed. These values SHOULD be used if they are appropriate.
The list is an open set in order to allow for more fine-grained distinctions
such as "eyes" vs. "mouth" etc.</p>

<p>The <code>expressed-through</code> attribute is not specific about the
sensors used for observing the modality. These can be specified using the <a
href="#s2.3.3"><code>&lt;info&gt;</code></a> element, or by the
<code>emma:mode</code> attribute in an enclosing [<a href="#ref-EMMA">EMMA</a>]
document.</p>

<p>Example:</p>

<p>In the following example the emotion is expressed through the voice.</p>
<pre class="example">&lt;emotion category-set="http://www.w3.org/TR/emotion-voc/xml#everyday-categories"
         expressed-through="voice"&gt;
    &lt;category name="satisfied"/&gt;
&lt;/emotion&gt;</pre>

<p>In case of multimodal expression of an emotion, a list of space separated
modalities can be indicated in the <code>expressed-through</code> attribute,
like in the following example in which the two values "face" and "voice" are
used.</p>
<pre class="example">&lt;emotion category-set="http://www.w3.org/TR/emotion-voc/xml#everyday-categories"
         expressed-through="face voice"&gt;
    &lt;category name="satisfied"/&gt;
&lt;/emotion&gt;&gt;</pre>

<p></p>

<p>See also the examples in sections <a href="#s5.1.2">5.1.2 Automatic
recognition of emotions</a>, <a href="#s5.1.3">5.1.3 Generation of
emotion-related system behavior</a> and <a href="#s5.2.3">5.2.3 Use with
SMIL</a>.</p>

<p></p>

<h4 id="s2.3.3">2.3.3 The <code>&lt;info&gt;</code> element</h4>

<table class="defn" cellspacing="0" cellpadding="5" width="98%"
summary="property definition">
  <tbody>
    <tr>
      <th>Annotation</th>
      <td><code>&lt;info&gt;</code></td>
    </tr>
    <tr>
      <th>Definition</th>
      <td>This element can be used to annotate arbitrary metadata.</td>
    </tr>
    <tr>
      <th>Children</th>
      <td>One or more elements in a different namespace than the <a
        href="#s4.1">EmotionML namespace</a>, providing metadata.</td>
    </tr>
    <tr>
      <th>Attributes</th>
      <td><ul>
          <li><strong>Optional</strong>:
            <ul>
              <li><code>id</code>, a unique identifier for the info element, of
                type <code>xsd:ID</code>.</li>
            </ul>
          </li>
        </ul>
      </td>
    </tr>
    <tr>
      <th>Occurence</th>
      <td>A single <code>&lt;info&gt;</code> element MAY occur as a child of
        the <code>&lt;emotionml</code>&gt; root tag to indicate global
        metadata, i.e. the annotations are valid for the document scope;
        furthermore, a single <code>&lt;info&gt;</code> element MAY occur as a
        child of each <code>&lt;emotion&gt;</code> element to indicate local
        metadata that is only valid for that <code>&lt;emotion&gt;</code>
        element.</td>
    </tr>
  </tbody>
</table>

<p>This element can contain arbitrary XML data in a different namespace (one
option could be [<a href="#ref-rdf">RDF</a>] data), either on a document global
level or on a local "per annotation element" level.</p>

<p>Several initiatives of standardizing metadata exist, such as [<a
href="#ref-imdi">IMDI</a>] and [<a href="#ref-clarin">CLARIN</a>]. Metadata may
contain information on a large spectrum of elements such as: location
description (continent, country, address), content type (e.g., genre, task,
modalities), session (title, a recording date, a group of participants); each
participant may be defined by her role in the session (e.g. annotator, filmer),
her name, her social family role, etc.</p>

<p>Examples:</p>

<p>In the following example, the automatic classification for an annotation
document was performed by a classifier based on Gaussian Mixture Models (GMM);
the speakers of the annotated elements were of different German origins.</p>
<pre class="example">&lt;emotionml xmlns="http://www.w3.org/2009/10/emotionml"
        xmlns:classifiers="http://www.example.com/meta/classify/"
        xmlns:origin="http://www.example.com/meta/local/"
        category-set="http://www.w3.org/TR/emotion-voc/xml#big6"&gt;
    &lt;info&gt;
        &lt;classifiers:classifier classifiers:name="GMM"/&gt;
    &lt;/info&gt;

    &lt;emotion&gt;
        &lt;info&gt;&lt;origin:localization value="bavarian"/&gt;&lt;/info&gt;
        &lt;category name="happiness"/&gt;
    &lt;/emotion&gt;

    &lt;emotion&gt;
        &lt;info&gt;&lt;origin:localization value="swabian"/&gt;&lt;/info&gt;
        &lt;category name="sadness"/&gt;
    &lt;/emotion&gt;
&lt;/emotionml&gt;</pre>

<p></p>

<p>The following example uses the IMDI metadata language to represent
information about the annotator who produced the emotion annotation in the
current document, in a global <code>&lt;info&gt;</code> element.</p>
<pre>&lt;emotionml xmlns="http://www.w3.org/2009/10/emotionml"
           xmlns:imdi="http://www.mpi.nl/IMDI/Schema/IMDI"&gt;
&lt;info&gt;
  &lt;imdi:Actors&gt;
      &lt;imdi:Actor&gt;
           &lt;imdi:Role&gt;Annotator&lt;/imdi:Role&gt;
           &lt;imdi:Name&gt;John&lt;/imdi:Name&gt;
           &lt;imdi:FullName&gt;John Smith Junior&lt;/imdi:FullName&gt;
           &lt;imdi:Code&gt;JS&lt;/imdi:Code&gt;
           &lt;imdi:FamilySocialRole&gt;Teacher&lt;/imdi:FamilySocialRole&gt;
          ...
      &lt;/imdi:Actor&gt;
  &lt;/imdi:Actors&gt;
&lt;/info&gt;
 ...
&lt;emotion&gt;...&lt;/emotion&gt;
&lt;emotion&gt;...&lt;/emotion&gt;
&lt;/emotionml&gt; </pre>

<p></p>

<p>The following example illustrates how <code>&lt;info&gt;</code> can be used
for annotating information on sensors through which an affective signal has
been detected. In the global <code>&lt;info&gt;</code> section, the sensors
used in the particular scenario are specified. Apart from their ID, information
on the modality observed by this sensor is provided as well as information on
the confidence for that sensor. In this example, the modality "posture" is
observed by a camera and a chair equipped with pressure sensors. For some
reason it is decided that emotion estimates based on camera data should be
trusted more than those based on chair data. Within the
<code>&lt;emotion&gt;</code> elements, <code>&lt;info&gt;</code> is used to
specify which sensor has been used to calculate the actual emotion value.</p>
<pre>&lt;emotionml xmlns="http://www.w3.org/2009/10/emotionml"
    xmlns:sensors="http://www.example.com/meta/sensors/"
    category-set="http://www.w3.org/TR/emotion-voc/xml#everyday-categories"&gt;

  &lt;info&gt;
    &lt;sensors:sensor id="camera1" confidence="0.9" expressed-through="posture"/&gt;
    &lt;sensors:sensor id="chair" confidence="0.3" expressed-through="posture"/&gt;
    ...
  &lt;/info&gt;

  &lt;emotion expressed-through="posture"&gt;
    &lt;info&gt;
      &lt;sensors:sensor idref="camera1"/&gt;
    &lt;/info&gt;
    &lt;category name="angry"/&gt;
  &lt;/emotion&gt;

  &lt;emotion expressed-through="posture"&gt;
    &lt;info&gt;
      &lt;sensors:sensor idref="chair"/&gt;
    &lt;/info&gt;
    &lt;category name="neutral"/&gt;
  &lt;/emotion&gt;

&lt;/emotionml&gt;</pre>

<p></p>

<h3 id="s2.4">2.4 References and time</h3>

<h4 id="s2.4.1">2.4.1 The <code>&lt;reference&gt;</code> element</h4>

<table class="defn" cellspacing="0" cellpadding="5" width="98%"
summary="property definition">
  <tbody>
    <tr>
      <th>Annotation</th>
      <td><code>&lt;reference&gt;</code></td>
    </tr>
    <tr>
      <th>Definition</th>
      <td>References may be used to relate the emotion annotation to the "rest
        of the world", more specifically to the emotional expression, the
        experiencing subject, the trigger, and the target of the emotion.</td>
    </tr>
    <tr>
      <th>Children</th>
      <td>None</td>
    </tr>
    <tr>
      <th>Attributes</th>
      <td><ul>
          <li><strong>Required</strong>:
            <ul>
              <li><code>uri</code>, a URI identifying the actual reference
                target. The URI MAY be extended by a media fragment, as
                explained in section <a href="#s2.4.2.4">2.4.2.4</a>.</li>
            </ul>
          </li>
          <li><strong>Optional</strong>:
            <ul>
              <li><code>role</code>, the type of relation between the emotion
                and the external item referred to; one of
                "<code>expressedBy</code>" (default),
                "<code>experiencedBy</code>", "<code>triggeredBy</code>",
                "<code>targetedAt</code>".</li>
              <li><code>media-type</code>, an attribute of type
                <code>xsd:string</code> holding the MIME type of the data that
                the <code>uri</code> attribute points to.</li>
            </ul>
          </li>
        </ul>
      </td>
    </tr>
    <tr>
      <th>Occurrence</th>
      <td>Multiple <code>&lt;reference&gt;</code> elements MAY occur as
        children of <code>&lt;emotion&gt;</code>.</td>
    </tr>
  </tbody>
</table>

<p>A <code>&lt;reference&gt;</code> element provides a link to media as a URI
[<a href="#ref-rfc3986">RFC3986</a>]. The semantics of references are described
by the <code>role</code> attribute which, if present, MUST have one of four
values:</p>
<ul>
  <li>"<code>expressedBy</code>" indicates that the reference points to
    observable behavior expressing the emotion. This is the default value if
    the <code>role</code> attribute is not explicitly stated;</li>
  <li>"<code>experiencedBy</code>" indicates that the reference points to the
    subject experiencing the emotion;</li>
  <li>"<code>triggeredBy</code>" indicates that the reference points to an
    emotion-eliciting event that caused an emotion and/or related
  appraisals;</li>
  <li>"<code>targetedAt</code>" indicates that the reference points to an
    object towards which an emotion-related action, or action tendency, is
    directed.</li>
</ul>

<p>For reference targets representing a period of time, start and end time MAY
be denoted by using the media fragments syntax, as explained in section <a
href="#s2.4.2.4">2.4.2.4</a>.</p>

<p>The <code>media-type</code> attribute MAY be used to differentiate between
different media types such as audio, video, text, etc. </p>

<p>There is no restriction regarding the number of
<code>&lt;reference&gt;</code> elements that MAY occur as children of
<code>&lt;emotion&gt;</code>.</p>

<p>Examples:</p>

<p>The following example illustrates the reference to two different URIs having
a different <code>role</code> with respect to the emotion: one reference points
to the emotion's expression, a video clip showing a user expressing the
emotion; the other reference points to the trigger that caused the emotion, in
this case another video clip that was seen by the person who expressed the
emotion.</p>

<p></p>
<pre class="example">&lt;emotion ... &gt;
    ...
    &lt;reference uri="http://www.example.com/data/video/v1.avi?t=2,13" role="expressedBy"/&gt;
    &lt;reference uri="http://www.example.com/events/e12.xml" role="triggeredBy"/&gt;
&lt;/emotion&gt;</pre>

<p>Several references may follow as children of one
<code>&lt;emotion&gt;</code> tag, even having the same <code>role</code>; for
example, the following annotation refers to a portion of a video and to
physiological sensor data, both of which expressed the emotion:</p>
<pre class="example">&lt;emotion ... &gt;
    ...
    &lt;reference uri="http://www.example.com/data/video/v1.avi?t=2,13" role="expressedBy"/&gt;
    &lt;reference uri="http://www.example.com/data/physio/ph7.txt" role="expressedBy"/&gt;
&lt;/emotion&gt;</pre>

<p>It is possible to explicitly indicate the MIME type of the item that the
reference refers to:</p>
<pre class="example">&lt;emotion ... &gt;
    ...
    &lt;reference uri="http://www.example.com/data/video/v1.avi?t=2,13" media-type="video/mp4"/&gt;
&lt;/emotion&gt;</pre>

<p></p>

<h4 id="s2.4.2">2.4.2 Timestamps</h4>

<h5 id="s2.4.2.1">2.4.2.1 Absolute time</h5>

<table class="defn" cellspacing="0" cellpadding="5" width="98%"
summary="property definition">
  <tbody>
    <tr>
      <th>Annotation</th>
      <td><code>start</code>, <code>end</code></td>
    </tr>
    <tr>
      <th>Definition</th>
      <td>Attributes to denote the starting and ending absolute times. They are
        of type <code>xsd:nonNegativeInteger</code> and indicate the number of
        milliseconds since 1 January 1970 00:00:00 GMT.</td>
    </tr>
    <tr>
      <th>Occurrence</th>
      <td>The attributes MAY occur inside an <code>&lt;emotion&gt;</code>
        element.</td>
    </tr>
  </tbody>
</table>

<p><code>start</code> and <code>end</code> attributes denote the absolute
starting and ending times at which an <a href="#term-emotion">emotion</a> or <a
href="#term-emotion-related-state">related state</a> happened. This might be
used for example with an "emotional diary" application. </p>

<p>Examples:</p>

<p>In the following example, the emotion category "surprise" is annotated,
immediately followed by the category "happiness". The <code>start</code> and
<code>end</code> attributes specify for each <code>emotion</code> element the
absolute beginning and ending times.</p>
<pre class="example">&lt;emotion category-set="http://www.w3.org/TR/emotion-voc/xml#big6"
         start="1268647200" end="1268647330"&gt;
    &lt;category name="surprise"/&gt;
&lt;/emotion&gt;
&lt;emotion category-set="http://www.w3.org/TR/emotion-voc/xml#big6"
         start="1268647331" end="1268647400"&gt;
    &lt;category name="happiness"/&gt;
&lt;/emotion&gt;</pre>

<p>The <code>end</code> value MUST be greater than or equal to the
<code>start</code> value.</p>

<p>The ECMAScript Date object's getTime() function is a way to determine the
absolute time.</p>

<h5 id="s2.4.2.2">2.4.2.2 Duration</h5>

<table class="defn" summary="property definition" width="98%" cellpadding="5"
cellspacing="0">
  <tbody>
    <tr>
      <th>Annotation</th>
      <th><code>duration</code></th>
    </tr>
    <tr>
      <th>Definition</th>
      <td>Attribute <span>of type <code>xsd:nonNegativeInteger</code></span>,
        defaulting to zero. It specifies the duration of the event in
        milliseconds.</td>
    </tr>
    <tr>
      <th>Occurrence</th>
      <td>This attribute MAY occur inside an <code>&lt;emotion&gt;</code>
        element.</td>
    </tr>
  </tbody>
</table>

<p></p>

<p>The duration of an input in milliseconds MAY be specified with the
<code>duration</code> attribute. The <code>duration</code> attribute MAY be
used either in combination with the <code>start</code> or
<code>offset-to-start</code> attribute or independently.</p>

<p>A <code>start</code> or <code>offset-to-start</code> attribute together with
the <code>duration</code> attribute set to zero MAY be used to indicate a
single timestamp on a time axis.</p>

<p>Examples:</p>

<p>In the following example, the <code>start</code> and <code>duration</code>
of the emotion category "surprise" are annotated:</p>
<pre class="example">&lt;emotion category-set="http://www.w3.org/TR/emotion-voc/xml#big6"
         start="1268647200" duration="130"&gt;
    &lt;category name="surprise"/&gt;
&lt;/emotion&gt;</pre>

<p></p>

<h5 id="s2.4.2.3">2.4.2.3 Relative time</h5>

<table class="defn" summary="property definition" width="98%" cellpadding="5"
cellspacing="0">
  <tbody>
    <tr>
      <th>Annotation</th>
      <th><code>time-ref-uri</code></th>
    </tr>
    <tr>
      <th>Definition</th>
      <td>Attribute of type <code>xsd:anyURI</code> indicating the URI used to
        anchor the relative timestamp.</td>
    </tr>
    <tr>
      <th>Occurrence</th>
      <td>This attribute MAY occur inside an <code>&lt;emotion&gt;</code>
        element.</td>
    </tr>
    <tr>
      <th>Annotation</th>
      <th><code>time-ref-anchor-point</code></th>
    </tr>
    <tr>
      <th>Definition</th>
      <td>Attribute with a value of <code>start</code> or <code>end</code>,
        defaulting to <code>start</code>. It indicates whether to measure the
        time from the start or end of the interval designated with
        <code>time-ref-uri</code>.</td>
    </tr>
    <tr>
      <th>Occurrence</th>
      <td>This attribute MAY occur inside an <code>&lt;emotion&gt;</code>
        element.</td>
    </tr>
    <tr>
      <th>Annotation</th>
      <th><code>offset-to-start</code></th>
    </tr>
    <tr>
      <th>Definition</th>
      <td>Attribute <span>of type <code>xsd:integer</code></span>, defaulting
        to zero. It specifies the offset in milliseconds for the start of input
        from the anchor point designated with
        <span><code>time-ref-uri</code></span> and
        <span><code>time-ref-anchor-point</code></span></td>
    </tr>
    <tr>
      <th>Occurrence</th>
      <td>This attribute MAY occur inside an <code>&lt;emotion&gt;</code>
        element.</td>
    </tr>
  </tbody>
</table>

<p>Relative timestamps define the start of an input relative to the start or
end of a reference interval such as another input.</p>

<p></p>

<p>The reference interval is designated with <code>time-ref-uri</code>
attribute. This MAY be combined with <code>time-ref-anchor-point</code>
attribute to specify whether the anchor point is the start or end of this
interval. The start of an input relative to this anchor point is then specified
with <code>offset-to-start</code> attribute.</p>

<p>The <code>time-ref-uri</code> attribute can point to a custom-defined
timestamp or can be, for example, a session identifier.</p>

<p></p>

<p>Examples:</p>

<p>Here is an example where the emotion "surprise" occurs two seconds after the
reference time point:</p>
<pre class="example">&lt;emotion category-set="http://www.w3.org/TR/emotion-voc/xml#big6"
         time-ref-uri="#my_session_id" offset-to-start="2000"&gt;
    &lt;category name="surprise"/&gt;
&lt;/emotion&gt;</pre>

<p></p>

<h5 id="s2.4.2.4">2.4.2.4 Timing in media</h5>

<table class="defn" cellspacing="0" cellpadding="5" width="98%"
summary="property definition">
  <tbody>
    <tr>
      <th>Annotation</th>
      <td><code>URI fragment: t</code></td>
    </tr>
    <tr>
      <th>Definition</th>
      <td>Attributes to denote start and endpoint of an annotation in a media
        stream. Allowed values must be conform with the Media Fragments
        Specification [<a href="#ref-MediaFragments">Media Fragments</a>]</td>
    </tr>
    <tr>
      <th>Occurence</th>
      <td>The URI fragment MAY occur in the <code>uri</code> attribute of a
        <code>&lt;reference&gt;</code> element.</td>
    </tr>
  </tbody>
</table>

<p>Temporal clipping is denoted by the name t, and specified as an interval
with a begin time and an end time. Either or both may be omitted, with the
begin time defaulting to 0 seconds and the end time defaulting to the duration
of the source media. The interval is half-open: the begin time is considered
part of the interval whereas the end time is considered to be the first time
point that is not part of the interval. If a single number only is given, this
is the begin time.</p>

<p></p>

<p>Temporal clipping can be specified either as Normal Play Time (npt) [<a
href="http://www.ietf.org/rfc/rfc2326.txt">RFC 2326</a>], as SMPTE timecodes,
[<a href="#ref-SMPTE">SMPTE</a>], or as real-world clock time (clock) [<a
href="#ref-rfc2326">RFC 2326</a>]. Begin and end times are always specified in
the same format. The format is specified by name, followed by a colon (:), with
npt: being the default.</p>

<p></p>

<p>Examples:</p>

<p>In the following example, the emotion category "happiness" is displayed in
an audio file called "myAudio.wav" from the 3rd to the 9th second.</p>
<pre class="example">&lt;emotion category-set="http://www.w3.org/TR/emotion-voc/xml#big6"&gt;
    &lt;category name="happiness"/&gt;
    &lt;reference uri="myAudio.wav#t=3,9"/&gt;
&lt;/emotion&gt;</pre>

<p>In the following example, the emotion category "happiness" is displayed in a
video file called "myVideo.avi" in SMPTE values, resulting in the time interval
[120,121.5).</p>
<pre class="example">&lt;emotion category-set="http://www.w3.org/TR/emotion-voc/xml#big6"&gt;
    &lt;category name="happiness"/&gt;
    &lt;reference uri="myVideo.avi#t=smpte-30:0:02:00,0:02:01:15"/&gt;
&lt;/emotion&gt;</pre>

<p></p>

<p>A last example states this in a video file in real-world clock time code, as
a 1 min interval on 26th Jul 2009 from 11hrs, 19min, 1sec.</p>
<pre class="example">&lt;emotion category-set="http://www.w3.org/TR/emotion-voc/xml#big6"&gt;
    &lt;category name="happiness"/&gt;
    &lt;reference uri="myVideo.avi#t=clock:2009-07-26T11:19:01Z,2009-07-26T11:20:01Z"/&gt;
&lt;/emotion&gt;</pre>

<p></p>

<h3 id="s2.5">2.5 Scale values</h3>

<p>Scale values are needed to represent content in <a
href="#s2.2.1"><code>&lt;category&gt;</code></a>, <a
href="#s2.2.2"><code>&lt;dimension&gt;</code></a>, <a
href="#s2.2.3"><code>&lt;appraisal&gt;</code></a> and <a
href="#s2.2.4"><code>&lt;action-tendency&gt;</code></a> elements, as well as in
<a href="#s2.3.1"><code>confidence</code></a>.</p>

<p>Representations of scale values can be static or dynamic. A static, constant
scale value is represented using the <code>value</code> attribute; for dynamic
values, their evolution over time is expressed using the
<code>&lt;trace&gt;</code> element.</p>

<h4 id="s2.5.1">2.5.1 The <code>value</code> attribute</h4>

<table class="defn" cellspacing="0" cellpadding="5" width="98%"
summary="property definition">
  <tbody>
    <tr>
      <th>Annotation</th>
      <td><code>value</code></td>
    </tr>
    <tr>
      <th>Definition</th>
      <td>Representation of a static scale value.</td>
    </tr>
    <tr>
      <th>Occurrence</th>
      <td>The <a href="#s2.2.2"><code>&lt;dimension&gt;</code></a> element MUST
        contain either a <code>value</code> attribute or a
        <code>&lt;trace&gt;</code> element; <a
        href="#s2.2.1"><code>&lt;category&gt;</code></a>, <a
        href="#s2.2.3"><code>&lt;appraisal&gt;</code></a> and <a
        href="#s2.2.4"><code>&lt;action-tendency&gt;</code></a> MAY contain
        either a <code>value</code> attribute or a <code>&lt;trace&gt;</code>
        element.</td>
    </tr>
  </tbody>
</table>

<p>The <code>value</code> attribute represents a static scale value of the
enclosing element.</p>

<p>Conceptually, a scale can represent concepts that vary from "nothing" to "a
lot" (unipolar scales), or concepts that vary between two opposites, from "very
negative" to "very positive" (bipolar scales). Both are represented in
EmotionML using floating point values from the interval [0;1]. The min and max
values of the scale SHOULD be interpreted as the extreme values, for both
unipolar and bipolar scales. For example in a <code>&lt;category&gt;</code>, a
<code>value="0"</code> SHOULD be interpreted to mean absolutely no emotion
(emotionless); a <code>value="1.0"</code> SHOULD be interpreted to mean emotion
at maximum intensity (pure uncontrolled emotion). For bipolar scales, such as
the valence dimension, a value of 0 represents the most negative possible
value, whereas a value of 1 represents the most positive value possible. The
neutral middle point of the scale is at 0.5.</p>

<p>Here are several examples for the usage of scales with EmotionML. </p>
<pre class="example">&lt;emotion dimension-set="http://www.w3.org/TR/emotion-voc/xml#fsre-dimensions"&gt;
    &lt;dimension name="arousal" value="0.4"/&gt; &lt;!-- a bit less than average arousal --&gt;
    &lt;dimension name="valence" value="0.6"/&gt; &lt;!-- a bit above average valence --&gt;
&lt;/emotion&gt;

&lt;emotion category-set="http://www.w3.org/TR/emotion-voc/xml#everyday-categories"&gt;
    &lt;category name="angry" value="0.5"/&gt; &lt;!-- anger at medium intensity --&gt;
&lt;/emotion&gt;

&lt;emotion appraisal-set="http://www.w3.org/TR/emotion-voc/xml#scherer-appraisals"&gt;
    &lt;appraisal name="suddenness" value="0.9"/&gt; &lt;!-- appraisal as a very sudden event --&gt;
&lt;/emotion&gt;

&lt;emotion action-tendency-set="http://www.w3.org/TR/emotion-voc/xml#frijda-action-tendencies"&gt;
    &lt;action-tendency name="approach" value="0.3"/&gt; &lt;!-- a rather weak tendency to approach --&gt;
&lt;/emotion&gt;</pre>

<p>Further examples of the <code>value</code> attribute can be found in the
context of the <a href="#s2.2.1"><code>&lt;category&gt;</code></a>, <a
href="#s2.2.2"><code>&lt;dimension&gt;</code></a>, <a
href="#s2.2.3"><code>&lt;appraisal&gt;</code></a> and <a
href="#s2.2.4"><code>&lt;action-tendency&gt;</code></a> elements.</p>

<h4 id="s2.5.2">2.5.2 The <code>&lt;trace&gt;</code> element</h4>

<table class="defn" cellspacing="0" cellpadding="5" width="98%"
summary="property definition">
  <tbody>
    <tr>
      <th>Annotation</th>
      <td><code>&lt;trace&gt;</code></td>
    </tr>
    <tr>
      <th>Definition</th>
      <td>Representation of the time evolution of a dynamic scale value.</td>
    </tr>
    <tr>
      <th>Children</th>
      <td>None</td>
    </tr>
    <tr>
      <th>Attributes</th>
      <td><ul>
          <li><strong>Required</strong>:
            <ul>
              <li><code>freq</code>, a sampling frequency in Hz.</li>
              <li><code>samples</code>, a space-separated list of numeric scale
                values from the interval [0;1] representing the scale value of
                the enclosing element as it changes over time.</li>
            </ul>
          </li>
        </ul>
      </td>
    </tr>
    <tr>
      <th>Occurrence</th>
      <td><p>The <a href="#s2.2.2"><code>&lt;dimension&gt;</code></a> element
        MUST contain either a <code>value</code> attribute or a
        <code>&lt;trace&gt;</code> element; <a
        href="#s2.2.1"><code>&lt;category&gt;</code></a>, <a
        href="#s2.2.3"><code>&lt;appraisal&gt;</code></a> and <a
        href="#s2.2.4"><code>&lt;action-tendency&gt;</code></a> MAY contain
        either a <code>value</code> attribute or a <code>&lt;trace&gt;</code>
        element.</p>
      </td>
    </tr>
  </tbody>
</table>

<p>A <code>&lt;trace&gt;</code> element represents the time course of a scale
value.</p>

<p>The <code>freq</code> attribute indicates the sampling frequency at which
the values listed in the <code>samples</code> attribute are given. </p>

<p>NOTE: The <code>&lt;trace&gt;</code> representation requires a periodic
sampling of values. In order to represent values that are sampled
aperiodically, separate <code>&lt;emotion&gt;</code> annotations with
appropriate timing information and individual <code>value</code> attributes may
be used.</p>

<p>Examples:</p>

<p>The following example illustrates the use of a trace to represent an episode
of fear during which the emotion's intensity is rising, first gradually, then
quickly to a very high value. Values are taken at a sampling frequency of 10
Hz, i.e. one value every 100 ms.</p>
<pre class="example">&lt;emotion category-set="http://www.w3.org/TR/emotion-voc/xml#big6"&gt;
  &lt;category name="fear"&gt;
    &lt;trace freq="10Hz"
           samples="0.1 0.1 0.15 0.2 0.2 0.25 0.25 0.25 0.3 0.3 0.35 0.5 0.7 0.8 0.85 0.85"/&gt;
  &lt;/category&gt;
&lt;/emotion&gt;</pre>

<p>The following example combines a trace of the appraisal "suddenness" with a
global confidence that the values represent the facts properly. There is a
sudden peak of suddenness; the annotator is reasonably certain that the
annotation is correct:</p>
<pre class="example">&lt;emotion appraisal-set="http://www.w3.org/TR/emotion-voc/xml#scherer-appraisals"&gt;
  &lt;appraisal name="suddenness" confidence="0.75"&gt;
    &lt;trace freq="10Hz" samples="0.1 0.1 0.1 0.1 0.1 0.7 0.8 0.8 0.8 0.8 0.4 0.2 0.1 0.1 0.1"/&gt;
  &lt;/appraisal&gt;
&lt;/emotion&gt;</pre>

<p></p>

<h2 id="s3">3 Defining vocabularies for representing emotions</h2>

<p>EmotionML markup MUST refer to one or more vocabularies to be used for
representing emotion-related states, as specified in the context of the <a
href="#s2.1.1"><code>&lt;emotionml&gt;</code></a> and <a
href="#s2.1.2"><code>&lt;emotion&gt;</code></a> elements. Due to the lack of
agreement in the community, the EmotionML specification does not preview a
single default set which should apply if no set is indicated. Instead, the user
MUST explicitly state the set of descriptor names used.</p>

<p>The document [<a href="#ref-emotion-voc">Vocabularies for EmotionML</a>]
provides a number of emotion vocabularies which are likely to be of general
interest. In order to promote interoperability, users SHOULD verify if one of
the vocabularies defined in that document is suitable for their application. If
that is not the case, users can define their own custom vocabularies as defined
in the present section.</p>

<h3 id="s3.1">3.1 Mechanism for defining vocabularies</h3>

<p>The syntax for defining emotion vocabularies is based on the element
<code>&lt;vocabulary&gt;</code> and its child <code>&lt;item&gt;</code>. </p>

<h4 id="s3.1.1">3.1.1 The <code>&lt;vocabulary&gt;</code> element</h4>

<table class="defn" cellspacing="0" summary="property definition"
cellpadding="5" width="98%">
  <tbody>
    <tr>
      <th>Annotation</th>
      <td><code>&lt;vocabulary&gt;</code></td>
    </tr>
    <tr>
      <th>Definition</th>
      <td>Contains the definition of an emotion vocabulary.</td>
    </tr>
    <tr>
      <th>Children</th>
      <td>A <code>&lt;vocabulary&gt;</code> element MUST contain one ore more
        <a href="#s3.1.2"><code>&lt;item&gt;</code></a> elements. A
        <code>&lt;vocabulary&gt;</code> element MAY contain a single <a
        href="#s2.3.3"><code>&lt;info&gt;</code></a> element, providing
        arbitrary metadata about the vocabulary itself. </td>
    </tr>
    <tr>
      <th>Attributes</th>
      <td><ul>
          <li><strong>Required</strong>:
            <ul>
              <li><code>type</code>, MUST be one of "<code>category</code>",
                "<code>dimension</code>", "<code>appraisal</code>" or
                "<code>action-tendency</code>". </li>
              <li><code>id</code>, an unique vocabulary identifier of type
                <code>xsd:ID</code>. </li>
            </ul>
          </li>
        </ul>
      </td>
    </tr>
    <tr>
      <th>Occurrence</th>
      <td><p>One or more <code>&lt;vocabulary&gt;</code> elements MAY occur as
        direct children of an <a
        href="#s2.1.1"><code>&lt;emotionml&gt;</code></a> element. </p>
      </td>
    </tr>
  </tbody>
</table>

<p>Vocabulary definitions, when present, occur as direct children of the
document root element <a href="#s2.1.1"><code>&lt;emotionml&gt;</code></a>. It
is possible to refer to a vocabulary defined in the same or in a separate
EmotionML document, through URIs specified by the values of the attributes
<code>category-set</code>, <code>dimension-set</code>,
<code>appraisal-set</code> and <code>action-tendency-set</code> of the <a
href="#s2.1.2"><code>&lt;emotion&gt;</code></a> element.</p>

<p>The value of the <code>type</code> attribute explicitly states whether the
vocabulary represents category names, dimension elements, appraisal elements or
action tendency elements. </p>

<h4 id="s3.1.2">3.1.2 The <code>&lt;item&gt;</code> element</h4>

<table class="defn" cellspacing="0" summary="property definition"
cellpadding="5" width="98%">
  <tbody>
    <tr>
      <th>Annotation</th>
      <td><code>&lt;item&gt;</code></td>
    </tr>
    <tr>
      <th>Definition</th>
      <td>Represents the definition of one vocabulary item, associated with a
        value which can be used in the "name" attribute of <a
        href="#s2.2.1"><code>&lt;category&gt;</code></a>, <a
        href="#s2.2.2"><code>&lt;dimension&gt;</code></a>, <a
        href="#s2.2.3"><code>&lt;appraisal&gt;</code></a> or <a
        href="#s2.2.4"><code>&lt;action-tendency&gt;</code></a> (depending on
        the type of vocabulary being defined).</td>
    </tr>
    <tr>
      <th>Children</th>
      <td>An <code>&lt;item&gt;</code> element MAY contain a single <a
        href="#s2.3.3"><code>&lt;info&gt;</code></a> element, providing
        arbitrary metadata about the vocabulary item.</td>
    </tr>
    <tr>
      <th>Attributes</th>
      <td><ul>
          <li><strong>Required</strong>:
            <ul>
              <li><code>name</code>: a name for the item, used to refer to this
                item. An <code>&lt;item&gt;</code> MUST NOT have the same name
                as any other <code>&lt;item&gt;</code> within the same
                <code>&lt;vocabulary&gt;</code>.</li>
            </ul>
          </li>
        </ul>
      </td>
    </tr>
    <tr>
      <th>Occurrence</th>
      <td>One or more <code>&lt;item&gt;</code> elements occur as direct
        children of a <a href="#s3.1.1"><code>&lt;vocabulary&gt;</code></a>
        element. </td>
    </tr>
  </tbody>
</table>

<p>An <code>&lt;item&gt;</code> represents the definition of one vocabulary
item. A <a href="#s3.1.1"><code>&lt;vocabulary&gt;</code></a> MUST contain at
least one <code>&lt;item&gt;</code> element. </p>

<p></p>

<p>Examples:</p>

<p>In the following example, three vocabularies are wrapped into a single
EmotionML document. Their <code>id</code> attributes are: "big6",
"fsre-dimensions" and "frijda-subset". They are used to represent categories,
dimensions and action tendencies respectively. The first
<code>&lt;emotion&gt;</code> element specifies the emotion vocabularies used
through the attributes <code>category-set</code> and
<code>action-tendency-set</code>, while the second <code>&lt;emotion&gt;</code>
element uses the attribute <code>dimension-set</code>. </p>
<pre>&lt;emotionml version="1.0" xmlns="http://www.w3.org/2009/10/emotionml"&gt;

    &lt;!-- Vocabulary definitions --&gt;

    &lt;vocabulary type="category" id="big6"&gt;
        &lt;item name="anger"/&gt;
        &lt;item name="disgust"/&gt;
        &lt;item name="fear"/&gt;
        &lt;item name="happiness"/&gt;
        &lt;item name="sadness"/&gt;
        &lt;item name="surprise"/&gt;
    &lt;/vocabulary&gt;

    &lt;vocabulary type="dimension" id="fsre-dimensions"&gt;
        &lt;item name="valence"/&gt;
        &lt;item name="potency"/&gt;
        &lt;item name="arousal"/&gt;
        &lt;item name="unpredictability"/&gt;
    &lt;/vocabulary&gt;

    &lt;vocabulary type="action-tendency" id="frijda-subset"&gt;
        &lt;item name="approach"/&gt;
        &lt;item name="avoidance"/&gt;
        &lt;item name="rejecting"/&gt;
    &lt;/vocabulary&gt;

    &lt;!-- Emotion elements --&gt;

    &lt;emotion category-set="#big6" action-tendency-set="#frijda-subset"&gt;
        &lt;category name="fear"/&gt;
        &lt;action-tendency name="approach" value="0.0"/&gt;
        &lt;action-tendency name="avoidance" value="0.9"/&gt;
    &lt;/emotion&gt;

    &lt;emotion dimension-set="#fsre-dimensions"&gt;
        &lt;dimension name="arousal" value="0.3"/&gt;
    &lt;/emotion&gt;

&lt;/emotionml&gt;</pre>

<p></p>

<h2 id="s4">4 Conformance</h2>

<h3 id="s4.1">4.1 EmotionML namespace</h3>

<p>The EmotionML namespace is "http://www.w3.org/2009/10/emotionml". All
EmotionML elements MUST use this namespace.</p>

<h3 id="s4.2">4.2 Use with other namespaces</h3>

<p>The EmotionML namespace is intended to be used with other XML namespaces as
per the Namespaces in XML Recommendation (1.0 [<a
href="#ref-XMLNS-10">XML-NS10</a>] or 1.1 [<a
href="#ref-XMLNS-11">XML-NS11</a>], depending on the version of XML being
used).</p>

<h3 id="s4.3">4.3 Schema validation and processor validation of EmotionML
documents</h3>

<p>The EmotionML schema is designed to validate the structural integrity of an
EmotionML document or document fragment, but cannot verify whether the emotion
descriptors used in the <code>name</code> attribute of
<code>&lt;category&gt;</code>, <code>&lt;dimension&gt;</code>,
<code>&lt;appraisal&gt;</code> and <code>&lt;action-tendency</code>&gt; are
consistent with the vocabularies indicated in the respective
<code>category-set</code>, <code>dimension-set</code>,
<code>appraisal-set</code> and <code>action-tendency-set</code> attributes.</p>

<p>It is the responsibility of an EmotionML processor to verify that the use of
descriptor names and values is consistent with the vocabulary definition.</p>

<h2 id="s5">5 Examples</h2>

<p><i>This section is informative.</i></p>

<h3 id="s5.1">5.1 Examples of emotion annotation</h3>

<h4 id="s5.1.1">5.1.1 Manual annotation of emotional material</h4>

<h5 id="example_annotation_images">Annotation of static images</h5>

<p>An image gets annotated with several emotion categories at the same time,
but different intensities.</p>
<pre class="example">&lt;emotionml xmlns="http://www.w3.org/2009/10/emotionml"
           xmlns:meta="http://www.example.com/metadata"
           category-set="http://www.example.com/custom/hall-matsumoto-emotions.xml"&gt;
   &lt;info&gt;
      &lt;meta:media-type&gt;image&lt;/meta:media-type&gt;
      &lt;meta:media-id&gt;disgust&lt;/meta:media-id&gt;
      &lt;meta:media-set&gt;JACFEE-database&lt;/meta:media-set&gt;
      &lt;meta:doc&gt;Example adapted from (Hall &amp; Matsumoto 2004) http://www.davidmatsumoto.info/Articles/2004_hall_and_matsumoto.pdf
      &lt;/meta:doc&gt;
   &lt;/info&gt;

   &lt;emotion&gt;
       &lt;category name="Disgust" value="0.82"/&gt;
       &lt;category name="Contempt" value="0.35"/&gt;
       &lt;category name="Anger" value="0.12"/&gt;
       &lt;category name="Surprise" value="0.53"/&gt;
   &lt;/emotion&gt;
&lt;/emotionml&gt;</pre>

<h5 id="example_annotation_videos">Annotation of videos</h5>

<p>Example 1: Annotation of a whole video: several emotions are annotated with
different intensities.</p>
<pre class="example">&lt;emotionml xmlns="http://www.w3.org/2009/10/emotionml"
           xmlns:meta="http://www.example.com/metadata"
           category-set="http://www.example.com/custom/humaine-database-labels.xml"&gt;
    &lt;info&gt;
        &lt;meta:media-type&gt;video&lt;/meta:media-type&gt;
        &lt;meta:media-name&gt;ed1_4&lt;/meta:media-name&gt;
        &lt;meta:media-set&gt;humaine database&lt;/meta:media-set&gt;
        &lt;meta:coder-set&gt;JM-AB-UH&lt;/meta:coder-set&gt;
    &lt;/info&gt;
    &lt;emotion&gt;
        &lt;category name="Amusement" value="0.52"/&gt;
        &lt;category name="Irritation" value="0.63"/&gt;
        &lt;category name="Relaxed" value="0.02"/&gt;
        &lt;category name="Frustration" value="0.87"/&gt;
        &lt;category name="Calm" value="0.21"/&gt;
        &lt;category name="Friendliness" value="0.28"/&gt;
    &lt;/emotion&gt;
&lt;/emotionml&gt;</pre>

<p>Example 2: Annotation of a video segment, where two emotions are annotated
for overlapping but not identical timespans.</p>
<pre class="example">&lt;emotionml xmlns="http://www.w3.org/2009/10/emotionml"
           xmlns:meta="http://www.example.com/metadata"
           category-set="http://www.example.com/custom/emotv-labels.xml"&gt;
    &lt;info&gt;
        &lt;meta:media-type&gt;video&lt;/meta:media-type&gt;
        &lt;meta:media-name&gt;ext-03&lt;/meta:media-name&gt;
        &lt;meta:media-set&gt;EmoTV&lt;/meta:media-set&gt;
        &lt;meta:coder&gt;4&lt;/meta:coder&gt;
    &lt;/info&gt;

    &lt;emotion&gt;
        &lt;category name="irritation" value="0.46"/&gt;
        &lt;reference uri="file:ext03.avi?t=3.24,15.4"&gt;
    &lt;/emotion&gt;
    &lt;emotion&gt;
        &lt;category name="despair" value="0.48"/&gt;
        &lt;reference uri="file:ext03.avi?t=5.15,17.9"/&gt;
    &lt;/emotion&gt;
&lt;/emotionml&gt;</pre>

<h4 id="s5.1.2">5.1.2 Automatic recognition of emotions</h4>

<p>This example shows how automatically annotated data from three affective
sensor devices might be stored or communicated.</p>

<p>It shows an excerpt of an episode experienced on 23 November 2001 from 14:36
onwards (absolute start time is 1006526160 milliseconds since 1 January 1970
00:00:00 GMT). Each device detects an emotion, but at slightly different times
and for different durations.</p>

<p>The next entry of observed emotions occurs about 6 minutes later (absolute
start time is 1006526520 milliseconds since 1 January 1970 00:00:00 GMT). Only
the physiology sensor has detected a short glimpse of anger, for the visual and
IR camera it was below their individual threshold so no entry from them.</p>

<p>For simplicity, all devices use categorical annotations and the same set of
categories. Obviously it would be possible, and even likely, that different
devices from different manufacturers provide their data annotated with
different emotion sets.</p>
<pre class="example">&lt;emotionml xmlns="http://www.w3.org/2009/10/emotionml"
    category-set="http://www.w3.org/TR/emotion-voc/xml#everyday-categories"&gt;
 ...
&lt;emotion start="1006526160" expressed-through="face"&gt;
  &lt;!--the first modality detects excitement.
      It is a camera observing the face. A URI to the database
      is provided to access the video stream.--&gt;
  &lt;category name="excited"/&gt;
  &lt;reference uri="http://www.example.com/facedb#t=26,98"/&gt;
&lt;/emotion&gt;

&lt;emotion start="1006526160" expressed-through="facial-skin-color"&gt;
  &lt;!--the second modality detects anger. It is an IR camera
      observing the face. A URI to the database
      is provided to access the video stream.--&gt;
  &lt;category name="angry"/&gt;
  &lt;reference uri="http://www.example.com/skindb#t=23,108"/&gt;
&lt;/emotion&gt;

&lt;emotion start="1006526160" expressed-through="physiology"&gt;
  &lt;!--the third modality detects excitement again. It is a
      wearable device monitoring physiological changes in the
      body. A URI to the database
      is provided to access the data stream.--&gt;
  &lt;category name="excited"/&gt;
  &lt;reference uri="http://www.example.com/physiodb#t=19,101"/&gt;
&lt;/emotion&gt;

&lt;emotion start="1006526520" expressed-through="physiology"&gt;
  &lt;category name="angry"/&gt;
  &lt;reference uri="http://www.example.com/physiodb2#t=2,6"/&gt;
&lt;/emotion&gt;
 ...
&lt;/emotionml&gt;</pre>

<p></p>

<p>Note that handling of complex emotions is not explicitly specified. This
example assumes that parallel occurrences of emotions will be determined on the
time stamp.</p>

<h4 id="s5.1.3">5.1.3 Generation of emotion-related system behavior</h4>

<h5 id="example_generation_mpeg4">Generation of facial expressions in an MPEG-4
face model</h5>

<p>The MPEG-4 standard offers 68 parameters, called Facial Animation Parameters
FAPs, to animate a 3D facial model. 66 of these parameters correspond to low
level parameters. These parameters act on the facial feature points defining a
3D facial model. They specify how these feature points are displaced. They
simulate muscular contraction. On the other hand, two FAPs, namely FAP1 and
FAP2, refer respectively to viseme and expression. FAP2 corresponds to one of
the six basic facial expressions (anger, disgust, fear, happiness, sadness and
surprise). The expressions associated to the six emotions are defined by
textual descriptions [<a href="#ref-Ostermann2002">Ostermann, 2002</a>].</p>

<p>In emotion theory, the idea of mixing emotions to create new emotions is
disputed. For the purposes of facial expression modeling, however, it is
possible to simulate different emotions as linear combinations of the six basic
facial expressions. MPEG-4 allows the linear combination of any two of these
expressions: emotion_1 * intensity_1 + emotion_2 * intensity_2. For example,
[<a href="#ref-Raouzaiou2005">Raouzaiou et al., 2005</a>] found that the
expressions of depression and guilt can be obtained by combinations of fear and
sadness with different intensities, while the expression of suspicion is
obtained by combining anger and disgust.</p>

<p>In EmotionML it is possible to represent the emotional input to an MPEG-4
based facial animation system using multiple <code>&lt;category&gt;</code>
elements, for example as follows.</p>
<pre class="example">&lt;emotion xmlns="http://www.w3.org/2009/10/emotionml"
         category-set="http://www.w3.org/TR/emotion-voc/xml#big6"&gt;
  &lt;!-- attempt to express suspicion as a combination of anger and disgust --&gt;
  &lt;category name="anger" value="0.5"/&gt;
  &lt;category name="disgust" value="0.3"/&gt;
&lt;/emotion&gt;</pre>

<h5 id="example_generation_robot">Generation of robot behavior</h5>

<p>The following example describes various aspects of an emotionally competent
robot whose battery is nearly empty. The robot is in a global state of high
arousal, negative pleasure and low dominance, i.e. a negative state of distress
paired with some urgency but quite limited power to influence the situation. It
has a tendency to seek a recharge and to avoid picking up boxes. However,
sensor data displays an unexpected obstacle on the way to the charging station.
This triggers planning of expressive behavior of frowning. The annotations are
grouped into a stand-alone EmotionML document here; in the real world, the
various aspects would more likely be embedded into different specialized markup
in various parts of the Robot architecture.</p>
<pre class="example">&lt;emotionml xmlns="http://www.w3.org/2009/10/emotionml"
           xmlns:meta="http://www.example.com/metadata"&gt;
    &lt;info&gt;
        &lt;meta:name&gt;robbie the robot example&lt;/meta:name&gt;
    &lt;/info&gt;

    &lt;!-- Robot's current global state configuration: negative, active, powerless --&gt;
    &lt;emotion dimension-set="http://www.w3.org/TR/emotion-voc/xml#pad-dimensions"&gt;
        &lt;dimension name="pleasure" value="0.2"/&gt;
        &lt;dimension name="arousal" value="0.8"/&gt;
        &lt;dimension name="dominance" value="0.3"/&gt;
    &lt;/emotion&gt;

    &lt;!-- Robot's action tendencies: want to recharge --&gt;
    &lt;emotion action-tendency-set="http://www.example.com/custom/action/robot.xml"&gt;
        &lt;action-tendency name="charge-battery" value="0.9"/&gt;
        &lt;action-tendency name="seek-shelter" value="0.7"/&gt;
        &lt;action-tendency name="pickup-boxes" value="0.1"/&gt;
    &lt;/emotion&gt;

    &lt;!-- Appraised value of incoming event: obstacle detected, appraised as novel and unpleasant --&gt;
    &lt;emotion appraisal-set="http://www.w3.org/TR/emotion-voc/xml#scherer-appraisals"&gt;
        &lt;appraisal name="suddenness" value="0.8" confidence="0.4"/&gt;
        &lt;appraisal name="intrinsic-pleasantness" value="0.2" confidence="0.8"/&gt;
        &lt;reference role="triggeredBy" uri="file:scannerdata.xml#obstacle27"/&gt;
    &lt;/emotion&gt;

    &lt;!-- Robot's planned facial gestures: will frown --&gt;
    &lt;emotion category-set="http://www.example.com/custom/robot-emotions.xml"
        expressed-through="face"&gt;
        &lt;category name="frustration"/&gt;
        &lt;reference role="expressedBy" uri="file:behavior-repository.xml#frown"/&gt;
    &lt;/emotion&gt;
&lt;/emotionml&gt;</pre>

<h3 id="s5.2">5.2 Examples of possible use with other markup languages</h3>

<p>One intended use of EmotionML is as a plug-in for existing markup languages.
For compatibility with text-annotating markup languages such as <a
href="#ref-SSML">SSML</a>, EmotionML avoids the use of text nodes. All
EmotionML information is encoded in element and attribute structures.</p>

<p>This section illustrates the concept using three existing W3C markup
languages: <a href="#ref-EMMA">EMMA</a>, <a href="#ref-SSML">SSML</a>, and <a
href="#ref-SMIL">SMIL</a>.</p>

<h4 id="s5.2.1">5.2.1 Use with EMMA</h4>

<p>EMMA is made for representing arbitrary analysis results; one of them could
be the emotional state. The following example represents an analysis of a
non-verbal vocalization; its emotion is described as a low-intensity state,
maybe boredom.</p>
<pre class="example">&lt;emma:emma version="1.0" xmlns:emma="http://www.w3.org/2003/04/emma"
        xmlns="http://www.w3.org/2009/10/emotionml"&gt;
    &lt;emma:interpretation emma:start="12457990" emma:end="12457995" emma:mode="voice" emma:verbal="false"&gt;

        &lt;emotion category-set="http://www.w3.org/TR/emotion-voc/xml#everyday-categories"&gt;
            &lt;category name="bored" value="0.1" confidence="0.1"/&gt;
        &lt;/emotion&gt;

    &lt;/emma:interpretation&gt;
&lt;/emma:emma&gt;</pre>

<p></p>

<p>In the folllowing example, the EMMA <code>&lt;emma:derivation&gt;</code>
element is used to represent multiple emotion interpretations associated with
audio and video media sources. The first and the third interpretations specify
the same emotion category, "content", while the result of the second one is
"amused". The consolidated emotion is the result of some processing made on the
interpretations included in the derivation element. In this case it is
"content", which is the most frequent category within the available
interpretations.</p>
<pre class="example">&lt;emma:emma version="1.0" xmlns:emma="http://www.w3.org/2003/04/emma" xmlns="http://www.w3.org/2009/10/emotionml"&gt;

    &lt;emma:derivation&gt;

        &lt;emma:interpretation id="text1" emma:start="12457960" emma:end="12457995" emma:mode="voice"
                emma:verbal="true" emma:signal="http://example.com/signals/emo123.wav"
                emma:process="http://example.com/text_analysis.xml"&gt;
            &lt;emma:literal&gt;I feel happy&lt;/emma:literal&gt;
            &lt;emotion category-set="http://www.w3.org/TR/emotion-voc/xml#everyday-categories"&gt;
                &lt;category name="content" value="0.7" confidence="0.7"/&gt;
            &lt;/emotion&gt;
        &lt;/emma:interpretation&gt;

        &lt;emma:interpretation id="voice1" emma:start="12457960" emma:end="12457995" emma:mode="voice"
                emma:verbal="false" emma:signal="http://example.com/signals/emo123.wav"
                emma:process="http://example.com/voice_analysis.xml"&gt;
            &lt;emotion category-set="http://www.w3.org/TR/emotion-voc/xml#everyday-categories"&gt;
                &lt;category name="amused" value="0.4" confidence="0.5"/&gt;
            &lt;/emotion&gt;
        &lt;/emma:interpretation&gt;

        &lt;emma:interpretation id="video1" emma:start="12457980" emma:end="12458000" emma:mode="video"
                emma:verbal="false" emma:signal="http://example.com/signals/emo123.mpg"
                emma:process="http://example.com/video_analysis.xml"&gt;
            &lt;emotion category-set="http://www.w3.org/TR/emotion-voc/xml#everyday-categories"&gt;
                &lt;category name="content" value="0.5" confidence="0.7"/&gt;
            &lt;/emotion&gt;
        &lt;/emma:interpretation&gt;

    &lt;/emma:derivation&gt;


    &lt;emma:interpretation id="multimodal1" emma:start="12457960" emma:end="12458000"
            emma:medium="acoustic visual" emma:mode="voice video"&gt;
        &lt;emma:derived-from resource="#text1" composite="true"/&gt;
        &lt;emma:derived-from resource="#voice1" composite="true"/&gt;
        &lt;emma:derived-from resource="#video1" composite="true"/&gt;
        &lt;emotion category-set="http://www.w3.org/TR/emotion-voc/xml#everyday-categories"&gt;
            &lt;category name="content" value="0.6" confidence="0.7"/&gt;
        &lt;/emotion&gt;
    &lt;/emma:interpretation&gt;

&lt;/emma:emma&gt;</pre>

<h4 id="s5.2.2">5.2.2 Use with SSML</h4>

<p>Two options for using EmotionML with SSML can be illustrated.</p>

<p>First, it is possible with [<a href="#ref-SSML1.1">SSML 1.1</a>] to use
arbitrary markup belonging to a different namespace anywhere in an SSML
document; only SSML processors that support the markup would take it into
account. Therefore, it is possible to insert EmotionML below, for example, an
<code>&lt;s&gt;</code> element representing a sentence; the intended meaning is
that the enclosing sentence should be spoken with the given emotion, in this
case a moderately worried tone of voice:</p>
<pre class="example">&lt;?xml version="1.0"?&gt;
&lt;speak version="1.1" xmlns="http://www.w3.org/2001/10/synthesis"
         xmlns:emo="http://www.w3.org/2009/10/emotionml"
         xml:lang="en-US"&gt;
    &lt;s&gt;
        &lt;emo:emotion category-set="http://www.w3.org/TR/emotion-voc/xml#everyday-categories"&gt;
            &lt;emo:category name="worried" value="0.4"/&gt;
        &lt;/emo:emotion&gt;

        Do you need help?
    &lt;/s&gt;
&lt;/speak&gt;</pre>

<p>Second, a future version of SSML could explicitly preview the annotation of
paralinguistic information, which could fill the gap between the
extralinguistic, speaker-constant settings of the <code>&lt;voice&gt;</code>
tag and the linguistic elements such as <code>&lt;s&gt;</code>,
<code>&lt;emphasis&gt;</code>, <code>&lt;say-as&gt;</code> etc. The following
example assumes that there is a <code>&lt;style&gt;</code> tag for
paralinguistic information in a future version of SSML. The style could embed
an <code>&lt;emotion&gt;</code>, as follows:</p>
<pre class="example">&lt;?xml version="1.0"?&gt;
&lt;speak version="x.y" xmlns="http://www.w3.org/2001/10/synthesis"
         xmlns:emo="http://www.w3.org/2009/10/emotionml"
         xml:lang="en-US"&gt;
    &lt;s&gt;
      &lt;style&gt;
        &lt;emo:emotion category-set="http://www.w3.org/TR/emotion-voc/xml#everyday-categories"&gt;
            &lt;emo:category name="worried" value="0.4"/&gt;
        &lt;/emo:emotion&gt;

        Do you need help?
      &lt;/style&gt;
    &lt;/s&gt;
&lt;/speak&gt;</pre>

<p>Alternatively, the <code>&lt;style&gt;</code> could refer to a previously
defined <code>&lt;emotion&gt;</code>, for example:</p>
<pre class="example">&lt;?xml version="1.0"?&gt;
&lt;speak version="x.y" xmlns="http://www.w3.org/2001/10/synthesis"
         xmlns:emo="http://www.w3.org/2009/10/emotionml"
         xml:lang="en-US"&gt;
    &lt;emo:emotion category-set="http://www.w3.org/TR/emotion-voc/xml#everyday-categories"
                 id="somewhatWorried"&gt;
        &lt;emo:category name="worried" value="0.4"/&gt;
    &lt;/emo:emotion&gt;

    &lt;s&gt;
        &lt;style ref="#somewhatWorried"&gt;
            Do you need help?
        &lt;/style&gt;
    &lt;/s&gt;
&lt;/speak&gt;</pre>

<h4 id="s5.2.3">5.2.3 Use with SMIL</h4>

<p>Using EmotionML for the use case of generating system behavior requires
elements of scheduling and surface form realization which are not part of
EmotionML. Necessarily, this use case relies on other languages to provide the
needed functionality. This is in line with the aim of EmotionML to serve as a
specialized plug-in language.</p>

<p>This example illustrates the idea in terms of a simplified version of a
storytelling application. A virtual agent tells a story using voice and facial
animation. The expression in face and voice is influenced by the rendering
engine in terms of EmotionML. The engine in this example uses SMIL [<a
href="#ref-SMIL">SMIL</a>] for defining the temporal relation between events;
EmotionML is used via SMIL's generic <code>&lt;ref&gt;</code> element. In
general it is the engine which knows how to render the emotion in the virtual
agent's expressive capabilities. To override this, the second
<code>&lt;emotion&gt;</code> contains an explicit request to realize the
emotional expression using both face and voice modalities.</p>

<p>ridinghood.smil:</p>
<pre class="example">&lt;smil xmlns="http://www.w3.org/ns/SMIL" version="3.0"&gt;
  &lt;head&gt; ... &lt;/head&gt;
  &lt;body&gt;
    &lt;par duration="8s"&gt;
      &lt;img src="file:forest.jpg"/&gt;
      &lt;smileText&gt;The little girl was enjoying the walk in the forest.&lt;/smileText&gt;
      &lt;ref src="file:ridinghood.emotionml#emotion1"/&gt;
    &lt;/par&gt;
    &lt;par duration="5s"&gt;
      &lt;img src="file:wolf.jpg"/&gt;
      &lt;smileText&gt;Suddenly a dark shadow appeared in front of her.&lt;/smileText&gt;
      &lt;ref src="file:ridinghood.emotionml#emotion2"/&gt;
    &lt;/par&gt;

  &lt;/body&gt;
&lt;/smil&gt;</pre>

<p></p>

<p>ridinghood.emotionml:</p>
<pre class="example">&lt;emotionml xmlns="http://www.w3.org/2009/10/emotionml"
    category-set="http://www.w3.org/TR/emotion-voc/xml#everyday-categories"
    appraisal-set="http://www.w3.org/TR/emotion-voc/xml#scherer-appraisals"&gt;

  &lt;emotion id="emotion1"&gt;
    &lt;category name="content" value="0.7"/&gt;
  &lt;/emotion&gt;

  &lt;emotion id="emotion2" expressed-through="face voice"&gt;
    &lt;category name="afraid" value="0.9"/&gt;
    &lt;appraisal name="suddenness" value="0.9"/&gt;
    &lt;appraisal name="intrinsic-pleasantness" value="0.1"/&gt;
  &lt;/emotion&gt;
&lt;/emotionml&gt;</pre>

<p></p>

<p>Similar principles for decoupling emotion markup from the temporal
organization of generating system behavior can be applied using other
representations, including interactive setups.</p>

<p></p>

<h2 id="s6">6 References</h2>

<h3 id="s6.1">6.1 Normative references</h3>
<dl>
  <dt id="ref-EMMA">EMMA</dt>
    <dd><a href="http://www.w3.org/TR/2004/REC-speech-synthesis-20040907/"
      shape="rect">EMMA: Extensible MultiModal Annotation markup language
      version 1.0</a>, Michael Johnston et al., Editors. World Wide Web
      Consortium, 11 December 2007. </dd>
  <dt id="ref-MediaFragments">Media Fragments URI</dt>
    <dd><a href="http://www.w3.org/TR/2011/WD-media-frags-20110317/">Media
      Fragments URI 1.0</a>, Raphaël Troncy et al., Editors. World Wide Web
      Consortium, W3C Working Draft 17 March 2011.</dd>
  <dt id="ref-rdf">RDF</dt>
    <dd><a href="http://www.w3.org/TR/rdf-syntax-grammar/">RDF/XML Syntax
      Specification (Revised)</a>, Dave Beckett, Editor. World Wide Web
      Consortium, W3C Recommendation 10 February 2004.</dd>
  <dt id="ref-rfc2119">RFC2119</dt>
    <dd><a href="http://www.ietf.org/rfc/rfc2119.txt">Key words for use in RFCs
      to Indicate Requirement Levels</a>, S. Bradner, Editor. IETF RFC 2119,
      March 1997.</dd>
  <dt><a id="ref-rfc2326">RFC 2326</a></dt>
    <dd><a href="http://www.ietf.org/rfc/rfc2326.txt">Real Time Streaming
      Protocol (RTSP)</a>, H. Schulzrinne et al., Editors. IETF RFC 2326, April
      1998.</dd>
  <dt id="ref-rfc3986">RFC3986</dt>
    <dd><a href="http://www.ietf.org/rfc/rfc3986.txt">Uniform Resource
      Identifier (URI): Generic Syntax</a>, T. Berners-Lee et al., Editors.
      IETF RFC 3986, January 2005.</dd>
  <dt id="ref-SMIL">SMIL</dt>
    <dd><a href="http://www.w3.org/TR/SMIL3/" shape="rect">Synchronized
      Multimedia Integration Language (SMIL) Version 3.0</a>, Dick Bulterman et
      al., Editors. W3C Recommendation, 1 December 2008.</dd>
  <dt><a id="ref-SMPTE">SMPTE</a></dt>
    <dd>SMPTE RP 136 Time and Control Codes for 24, 25 or 30 Frame-Per-Second
      Motion-Picture Systems.</dd>
  <dt id="ref-SSML">SSML</dt>
    <dd><a href="http://www.w3.org/TR/2004/REC-speech-synthesis-20040907/"
      shape="rect">Speech Synthesis Markup Language (SSML) Version 1.0</a>,
      Daniel C. Burnett, et al., Editors. World Wide Web Consortium, W3C
      Recommendation, 7 September 2004. </dd>
  <dt id="ref-SSML1.1">SSML 1.1</dt>
    <dd><a href="http://www.w3.org/TR/2010/REC-speech-synthesis11-20100907/"
      shape="rect">Speech Synthesis Markup Language (SSML) Version 1.1</a>,
      Daniel C. Burnett, et al., Editors. World Wide Web Consortium, W3C
      Recommendation, 7 September 2010. </dd>
  <dt id="ref-XMLNS-10">XML-NS10</dt>
    <dd><a href="http://www.w3.org/TR/2006/REC-xml-names-20060816/">Namespaces
      in XML 1.0</a>, Tim Bray et al., Editors. World Wide Web Consortium, W3C
      Recommendation, 16 August 2006.</dd>
  <dt id="ref-XMLNS-11">XML-NS11</dt>
    <dd><a href="http://www.w3.org/TR/xml-names11/">Namespaces in XML 1.1</a>,
      Tim Bray et al., Editors. World Wide Web Consortium, W3C Recommendation,
      2006.</dd>
  <dt id="ref-xml-schema">XML Schema</dt>
    <dd><a href="http://www.w3.org/TR/xmlschema-1/">XML Schema Part 1:
      Structures Second Edition</a>, Henry S. Thompson et al., Editors. World
      Wide Web Consortium, W3C Recommendation, 2004.</dd>
</dl>

<h3 id="s6.2">6.2 Informative references</h3>
<dl>
  <dt id="ref-clarin">CLARIN</dt>
    <dd><a href="http://www.clarin.eu/files/wg2-4-metadata-doc-v5.pdf">CLARIN
      Metadata Infrastructure for Language Resources and Technology</a>,Version
      5, D. Broeder et al., Editors. Common Language Resources and Technology
      Infrastructure Report, 4 February 2009.</dd>
  <dt id="ref-emotion-xg">Emotion Incubator Group</dt>
    <dd><a href="http://www.w3.org/2005/Incubator/emotion/XGR-emotion/">W3C
      Emotion Incubator Group</a>, M. Schröder, E. Zovato, H. Pirker, C.
      Peter, F. Burkhardt, Editors. Final Report of the Emotion Incubator Group
      at the World Wide Web Consortium, 10 July 2007. </dd>
  <dt id="ref-emotionml-xg">Emotion Markup Language Incubator Group</dt>
    <dd><a
      href="http://www.w3.org/2005/Incubator/emotion/XGR-emotionml/">Elements
      of an EmotionML 1.0</a>, M. Schröder, Editor. Final Report of the
      Emotion Markup Language Incubator Group at the World Wide Web Consortium,
      20 November 2008. </dd>
  <dt id="ref-emotionml-req">EmotionML Requirements</dt>
    <dd><a
      href="http://www.w3.org/2005/Incubator/emotion/XGR-requirements-20080513/">Emotion
      Markup Language: Requirements with Priorities</a>. F. Burkhardt and M.
      Schröder. W3C Incubator Group Report, 13 May 2008. </dd>
  <dt id="ref-imdi">IMDI</dt>
    <dd><a href="http://www.mpi.nl/corpus/manuals/manual-imdi-editor.pdf">IMDI
      Editor version 3.2</a>, B. Hellwig and D. van Uytvanck. ISLE Metadata
      Initiative Report, 19 June 2007.</dd>
  <dt id="ref-Ortony1988">Ortony et al., 1988</dt>
    <dd>Ortony, A., Clore, G. L., &amp; Collins, A. (1988). The Cognitive
      Structure of Emotion. Cambridge, UK: Cambridge University Press.</dd>
  <dt id="ref-Ostermann2002">Ostermann, 2002</dt>
    <dd>Ostermann, J. (2002). Face Animation in MPEG-4. In: MPEG-4 Facial
      Animation - The Standard Implementation and Applications (I.S. Pandzic
      and R. Forchheimer, eds.), pp. 17-55. England: Wiley.</dd>
  <dt id="ref-Raouzaiou2005">Raouzaiou et al., 2005</dt>
    <dd>Raouzaiou, A., Spyrou, E., Karpouzis, K. and Kollias, S. (2005).
      Emotion Synthesis: an Intermediate Expressions’ Generator System in the
      MPEG-4 Framework. International Workshop VLBV05, 15-16 September 2005,
      Sardinia, Italy.</dd>
  <dt id="ref-emotion-voc">Vocabularies for EmotionML</dt>
    <dd><a
      href="http://www.w3.org/TR/2011/WD-emotion-voc-20110407/">Vocabularies
      for EmotionML</a>. M. Schröder and C. Pelachaud, Editors. W3C Working
      Draft, 7 April 2011. </dd>
</dl>

<h2 id="s7">7 Acknowledgments</h2>

<p>The authors wish to acknowledge the contributions by all members of the
Multimodal Interaction Working Group, the Emotion Markup Language Incubator
Group and the Emotion Incubator Group, as well as the participants to the W3C
Workshop on EmotionML, in particular the following persons (in alphabetic
order):</p>
<ul>
  <li>Kazuyuki Ashimura, W3C</li>
  <li>Andrew Breen, Nuance Communications</li>
  <li>Roddy Cowie, Queen's University Belfast</li>
  <li>Deborah Dahl, Conversational Technologies</li>
  <li>Sarah Jane Delany, Dublin Institute of Technology</li>
  <li>Dylan Evans, University College Cork</li>
  <li>Nestor Garay Vitoria, University of the Basque Country</li>
  <li>Alain Giboin, INRIA Sophia Antipolis</li>
  <li>Bill Jarrold, SRI International</li>
  <li>Michael Johnston, AT&amp;T</li>
  <li>Kostas Karpouzis, Image, Video and Multimedia Systems Lab (IVML-NTUA)</li>
  <li>Myriam Lamolle, University of Paris VIII</li>
  <li>Tim Llewellyn, nViso</li>
  <li>Jean-Claude Martin, CNRS</li>
  <li>Alessandro Oltramari, CNR</li>
  <li>Hannes Pirker, Austrian Research Institute for Artificial
  Intelligence</li>
  <li>Björn Schuller, Technische Universität München</li>
  <li>Jianhua Tao, Chinese Academy of Sciences</li>
  <li>Ian Wilson, Emotion AI</li>
  <li>Gill Windall, University of Greenwich</li>
  <li>Idoia Zearreta, University of the Basque Country</li>
</ul>
<hr />

<h2 id="changelog">Appendix A: Changes</h2>

<p><em>This section is informative.</em></p>

<h3 id="changes-current">Changes in the current Working Draft</h3>

<p>This section summarizes the main changes since the <a
href="http://www.w3.org/TR/2010/WD-emotionml-20100729/">previous working draft
of 29 July 2010</a>.</p>
<ul>
  <li>Changes specific to EmotionML
    <ul>
      <li>The document now distinguishes between normative and non-normative
        sections.</li>
      <li>A <a href="#s3.1">mechanism for defining emotion vocabularies</a> was
        specified.</li>
      <li>The definitions of emotion vocabularies were moved from the
        specification into a dedicated W3C Working Draft [<a
        href="#ref-emotion-voc">Vocabularies for EmotionML</a>].</li>
      <li>The <a href="#s2.2.1"><code>&lt;category&gt;</code></a> element was
        harmonized with the other emotion descriptors to allow a
        <code>value</code> attribute or a <code>&lt;trace&gt;</code> child
        element indicating the intensity of that category. Multiple
        <code>&lt;category&gt;</code> elements are now allowed within a single
        <code>&lt;emotion&gt;</code> to reflect the possible co-presence of
        these categories. The <code>&lt;intensity&gt;</code> element was
        removed since the usual use is now covered by the <code>value</code>
        attribute in <code>&lt;category&gt;</code>.</li>
      <li>The specification of scale values through the <code>value</code>
        attribute or the <code>&lt;trace&gt;</code> child element was made
        optional for <a href="#s2.2.3"><code>&lt;appraisal&gt;</code></a> and
        <a href="#s2.2.4"><code>&lt;action-tendency&gt;</code></a> elements, in
        order to allow for the possibility to merely represent the fact that a
        certain appraisal or action tendency is present, irrespective of its
        intensity.</li>
      <li>A mechanism for indicating <a
        href="#s2.4.2.2"><code>duration</code></a> and <a
        href="#s2.4.2.3">relative timestamps</a> was added.</li>
      <li>More examples were added to illustrate possible uses of the <a
        href="#s2.3.3"><code>&lt;info&gt;</code></a> element for representing
        metadata.</li>
      <li>An <a href="#example_generation_mpeg4">example</a> was added to
        illustrate the use of EmotionML in the context of predicting mixed
        emotions in an MPEG-4-based facial animation model.</li>
      <li>An <a href="#s5.2.1">example</a> was added to illustrate the use of
        EMMA for representing the derivation of a consolidated emotion from
        individual emotion observations.</li>
      <li>All examples were updated to be consistent with [<a
        href="#ref-emotion-voc">Vocabularies for EmotionML</a>] where
        appropriate.</li>
    </ul>
  </li>
</ul>
<ul>
  <li>Consistency issues with other W3C specifications
    <ul>
      <li>To avoid confusion with the <code>emma:mode</code> attribute in [<a
        href="#ref-EMMA">EMMA</a>], the <code>modality</code> attribute was
        renamed to <a href="#s2.3.2"><code>expressed-through</code></a>.</li>
    </ul>
  </li>
</ul>

<p></p>

<h3 id="changes-wd2">Changes in Working Draft 2 (29 July 2010)</h3>
<ul>
  <li>Changes specific to EmotionML
    <ul>
      <li>A mechanism for pointing to emotion vocabularies was agreed. An
        emotion vocabulary is now identified by a URI in the attributes
        category-set, dimension-set, appraisal-set and action-tendency-set of
        <a href="#s2.1.2">&lt;emotion&gt;</a> or, in the sense of a
        document-wide default, of <a href="#s2.1.1">&lt;emotionml&gt;</a>. The
        consistency of an EmotionML annotation with the indicated vocabulary
        must be verified by an EmotionML processor; it cannot be verified using
        Schema validation. The <a href="#s4.3">section on validation</a> was
        updated accordingly.</li>
      <li>A collection of <a href="#s3.1">emotion vocabularies</a> was compiled
        which may be useful defaults for many users. The list is incomplete and
        not fully developed, but is already published in this form to elicit
        feedback.</li>
      <li>The notion of <a href="#s2.5">Scale values</a> was simplified to only
        allow for continuous values in the range [0;1].</li>
      <li>The notion of a confidence-trace was dropped for simplicity.</li>
    </ul>
  </li>
</ul>
<ul>
  <li>Consistency issues with other W3C specifications
    <ul>
      <li>The syntax for representing dimensions, appraisals and action
        tendencies was changed to be more in line with the expectation that
        user-defined strings figure in attribute values rather than element or
        attribute names. The specification now defines <a
        href="#s2.2.2"><code>&lt;dimension&gt;</code></a>, <a
        href="#s2.2.3"><code>&lt;appraisal&gt;</code></a> and <a
        href="#s2.2.4"><code>&lt;action-tendency&gt;</code></a> elements with a
        <code>name</code> attribute.</li>
      <li>The <a href="#s2.4.2">representation of time</a> was synchronized
        with <a href="#ref-EMMA">EMMA</a>: the specification now uses
        <code>start</code> and <code>end</code> attributes to represent
        absolute time, and <a href="#ref-MediaFragments">Media Fragment
        URIs</a> to refer to portions of media files.</li>
      <li>Metadata is now represented using an <code><a
        href="#s2.3.3">&lt;info&gt;</a></code> element, in synchrony with <a
        href="#ref-EMMA">EMMA</a>.</li>
      <li>The <code>&lt;link&gt;</code> element was renamed to <a
        href="#s2.4.1"><code>&lt;reference&gt;</code></a> to avoid a name clash
        with the <code>&lt;link&gt;</code> element in HTML, which has a
        different scope and syntax.</li>
    </ul>
  </li>
</ul>

<p></p>

<p></p>
</body>
</html>