index.html 60.4 KB

Raw Blame History Permalink

<?xml version="1.0" encoding="UTF-8"?>
<!DOCTYPE html PUBLIC "-//W3C//DTD XHTML 1.0 Transitional//EN"
    "http://www.w3.org/TR/xhtml1/DTD/xhtml1-transitional.dtd">
<html xmlns="http://www.w3.org/1999/xhtml" lang="EN" xml:lang="EN">
<head>
<meta name="generator" content="HTML Tidy, see www.w3.org" />
<meta http-equiv="Content-Type"
content="text/html; charset=UTF-8" />
<title>Best practices for creating MMI Modality Components</title>
<style type="text/css">
code           { font-family: monospace; }

div.constraint,
div.issue,
div.note,
div.notice     { margin-left: 2em; }

ol.enumar      { list-style-type: decimal; }
ol.enumla      { list-style-type: lower-alpha; }
ol.enumlr      { list-style-type: lower-roman; }
ol.enumua      { list-style-type: upper-alpha; }
ol.enumur      { list-style-type: upper-roman; }


div.exampleInner pre { margin-left: 1em;
                       margin-top: 0em; margin-bottom: 0em}
div.exampleOuter {border: 4px double gray;
                  margin: 0em; padding: 0em}
div.exampleInner { background-color: #d5dee3;
                   border-top-width: 4px;
                   border-top-style: double;
                   border-top-color: #d3d3d3;
                   border-bottom-width: 4px;
                   border-bottom-style: double;
                   border-bottom-color: #d3d3d3;
                   padding: 4px; margin: 0em }
div.exampleWrapper { margin: 4px }
div.exampleHeader { font-weight: bold;
                    margin: 4px}

table {
        width:80%;
        border:1px solid #000;
        border-collapse:collapse;
        font-size:90%;
    }

td,th{
        border:1px solid #000;
        border-collapse:collapse;
        padding:5px;
    }


caption{
        background:#ccc;
        font-size:140%;
        border:1px solid #000;
        border-bottom:none;
        padding:5px;
        text-align:center;
    }

img.center {
  display: block;
  margin-left: auto;
  margin-right: auto;
}
p.caption {
  text-align: center
}

.RFC2119 {
  text-transform: lowercase;
  font-style: italic;
}
</style>

<link rel="stylesheet" type="text/css"
href="http://www.w3.org/StyleSheets/TR/W3C-WG-NOTE.css" />
</head>
<body>
<div class="head">
<p><a href="http://www.w3.org/"><img
src="http://www.w3.org/Icons/w3c_home" alt="W3C" height="48"
width="72" /></a></p>

<h1><a name="title" id="title"></a>Best practices for creating MMI
Modality Components</h1>

<h2><a name="w3c-doctype" id="w3c-doctype"></a>W3C Working Group
Note <i>1</i> <i>March</i> <i>2011</i></h2>

<dl>
<dt>This version:</dt>

<dd><a
href="http://www.w3.org/TR/2011/NOTE-mmi-mcbp-20110301/">http://www.w3.org/TR/2011/NOTE-mmi-mcbp-20110301/</a></dd>

<dt>Latest version:</dt>

<dd><a
href="http://www.w3.org/TR/mmi-mcbp/">http://www.w3.org/TR/mmi-mcbp/</a></dd>

<dt>Previous version:</dt>

<dd>This is the first version.</dd>

<dt>Editor:</dt>

<dd>Ingmar Kliche , Deutsche Telekom AG</dd>

<dt>Authors:</dt>

<dd>Deborah Dahl, Invited Expert</dd>

<dd>James A. Larson, Invited Expert</dd>

<dd>B. Helena Rodriguez, Telecom ParisTech</dd>

<dd>Muthuselvam Selvaraj, until 2009 while at HP</dd>
</dl>

<p class="copyright"><a
href="http://www.w3.org/Consortium/Legal/ipr-notice#Copyright">Copyright</a>&nbsp;©&nbsp;<i>
2011</i>&nbsp;<a href="http://www.w3.org/"><acronym
title="World Wide Web Consortium">W3C</acronym></a><sup>®</sup> (<a
href="http://www.csail.mit.edu/"><acronym
title="Massachusetts Institute of Technology">MIT</acronym></a>, <a
href="http://www.ercim.eu/"><acronym
title="European Research Consortium for Informatics and Mathematics">
ERCIM</acronym></a>, <a href="http://www.keio.ac.jp/">Keio</a>),
All Rights Reserved. W3C <a
href="http://www.w3.org/Consortium/Legal/ipr-notice#Legal_Disclaimer">
liability</a>, <a
href="http://www.w3.org/Consortium/Legal/ipr-notice#W3C_Trademarks">
trademark</a> and <a
href="http://www.w3.org/Consortium/Legal/copyright-documents">document
use</a> rules apply.</p>
</div>

<hr />
<div>
<h2><a name="abstract" id="abstract"></a>Abstract</h2>

<p>This document describes Modality Components in the MMI
Architecture which are responsible for controlling the various
input and output modalities on various devices by providing
guidelines and suggestions for designing Modality Components. Also
this document shows several possible examples of Modality
Components, (1) face identification, (2) form-filling using
handwriting recognition and (3) video display.</p>
</div>

<div>
<h2><a name="status" id="status"></a>Status of this Document</h2>

<p><em>This section describes the status of this document at the
time of its publication. Other documents may supersede this
document. A list of current W3C publications and the latest
revision of this technical report can be found in the <a
href="http://www.w3.org/TR/">W3C technical reports index</a> at
http://www.w3.org/TR/.</em></p>

<p>This is the 1 March 2011 <a
href="http://www.w3.org/2005/10/Process-20051014/tr.html#WGNote">W3C
Working Group Note</a> of "Best practices for creating MMI Modality
Components". The Multimodal Interaction Working Group once
published a Working Draft of the "<a
href="http://www.w3.org/TR/2008/WD-mmi-arch-20081016/#creating_MC">Multimodal
Architecture and Interfaces (MMI Architecture)</a>" on 16 October
2008 with this content. However, the Working Group concluded that
the description on how to create Modality Components and examples
of possible Modality Components should be published as a Working
Group Note rather than part of the MMI Architecture specification.
The goal of this Working Group Note is to provide guidelines and
suggestions for designing Modality Components in the MMI
Architecture and make it easier to author concrete Modality
Components for multimodal Web applications. Also this document
shows several possible examples of Modality Components, (1) face
identification, (2) form-filling using handwriting recognition and
(3) video display.</p>

<p>This W3C Working Group Note has been developed by the <a
href="http://www.w3.org/2002/mmi/">Multimodal Interaction Working
Group</a> of the W3C <a
href="http://www.w3.org/2002/mmi/Activity.html">Multimodal
Interaction Activity</a>.</p>

<p>Comments for this note are welcomed and should have a subject
starting with the prefix '[ARCH]'. Please send them to <a
href="mailto:www-multimodal@w3.org">www-multimodal@w3.org</a>, the
public email list for issues related to Multimodal. This list is <a
href="http://lists.w3.org/Archives/Public/www-multimodal/">archived</a>
and acceptance of this archiving policy is requested automatically
upon first post. To subscribe to this list send an email to <a
href="mailto:www-multimodal-request@w3.org">www-multimodal-request@w3.org</a>
with the word "subscribe" in the subject line.</p>

<p>For more information about the Multimodal Interaction Activity,
please see the <a
href="http://www.w3.org/2002/mmi/Activity">Multimodal Interaction
Activity statement</a>.</p>

<p>This document was produced by a group operating under the <a
href="http://www.w3.org/Consortium/Patent-Policy-20040205/">5
February 2004 W3C Patent Policy</a>. W3C maintains a <a
href="http://www.w3.org/2004/01/pp-impl/34607/status">public list
of any patent disclosures</a> made in connection with the
deliverables of the group; that page also includes instructions for
disclosing a patent. An individual who has actual knowledge of a
patent which the individual believes contains <a
href="http://www.w3.org/Consortium/Patent-Policy-20040205/#def-essential">
Essential Claim(s)</a> must disclose the information in accordance
with <a
href="http://www.w3.org/Consortium/Patent-Policy-20040205/#sec-Disclosure">
section 6 of the W3C Patent Policy.</a></p>

<p>Publication as a Working Group Note does not imply endorsement
by the W3C Membership. This is a draft document and may be updated,
replaced or obsoleted by other documents at any time. It is
inappropriate to cite this document as other than work in
progress.</p>
</div>

<div class="toc">
<h2><a name="contents" id="contents"></a>Table of Contents</h2>

<p class="toc">1 <a href="#introduction">Introduction</a><br />
2 <a href="#Modality_component_guidelines">Modality component
guidelines</a><br />
&nbsp;&nbsp;&nbsp;&nbsp;2.1 <a href="#Guideline1">Guideline 1: Each
modality component must implement all of the MMI life-cycle
events.</a><br />
&nbsp;&nbsp;&nbsp;&nbsp;2.2 <a href="#Guideline2">Guideline 2:
Identify other functions of the modality component that are
relevant to the interaction manager.</a><br />
&nbsp;&nbsp;&nbsp;&nbsp;2.3 <a href="#Guideline3">Guideline 3: If
the component uses media, specify the media format. For example,
audio formats for speech recognition, or InkML for handwriting
recognition.</a><br />
&nbsp;&nbsp;&nbsp;&nbsp;2.4 <a href="#Guideline4">Guideline 4:
Specify protocols for use between the component and the Interaction
Manager (IM) (e.g., SIP or HTTP).</a><br />
&nbsp;&nbsp;&nbsp;&nbsp;2.5 <a href="#Guideline5">Guideline 5:
Specify supported human languages, e.g., English, German, Chinese
and locale, if relevant.</a><br />
&nbsp;&nbsp;&nbsp;&nbsp;2.6 <a href="#Guideline6">Guideline 6:
Specify supporting languages required by the component, if
any.</a><br />
&nbsp;&nbsp;&nbsp;&nbsp;2.7 <a href="#Guideline7">Guideline 7:
Modality components sending data to the interaction manager must
use the format where appropriate.</a><br />
&nbsp;&nbsp;&nbsp;&nbsp;2.8 <a href="#Guideline8">Guideline 8:
Specify error codes and their meanings to be returned to the
IM.</a><br />
3 <a href="#Modality_component_design_suggestions">Modality
component design suggestions</a><br />
&nbsp;&nbsp;&nbsp;&nbsp;3.1 <a href="#DesignSuggestion1">Design
suggestion 1: Consider constructing a complex modality component
with multiple functions if one function handles the errors
generated by another function.</a><br />
&nbsp;&nbsp;&nbsp;&nbsp;3.2 <a href="#DesignSuggestion2">Design
suggestion 2: Consider constructing a complex modality component
with multiple functions rather than several simple modality
components if the functions need to be synchronized.</a><br />
&nbsp;&nbsp;&nbsp;&nbsp;3.3 <a href="#DesignSuggestion3">Design
suggestion 3: Consider constructing a nested modality component
with multiple child modality components if the children modality
components are frequently used together but do not handle ther
errors generated by the other children components and the children
components do not need to be extensively synchronized.</a><br />
4 <a href="#Example_simple_modality_Face_Identification">Example
simple modality: Face Identification</a><br />
&nbsp;&nbsp;&nbsp;&nbsp;4.1 <a
href="#Functions_of_a_Possible_Face_Identification_Component">Functions
of a Possible Face Identification Component</a><br />
&nbsp;&nbsp;&nbsp;&nbsp;4.2 <a href="#Event_Syntax">Event
Syntax</a><br />
&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;4.2.1 <a
href="#Examples_of_events_for_starting_the_component">Examples of
events for starting the component</a><br />
&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;4.2.2 <a
href="#Example_output_event">Example output event</a><br />
5 <a
href="#Example_simple_modality_Form-filling_using_Handwriting_Recognition">
Example simple modality: Form-filling using Handwriting
Recognition</a><br />
&nbsp;&nbsp;&nbsp;&nbsp;5.1 <a
href="#Functions_of_a_Possible_Handwriting_Recognition_Component">Functions
of a Possible Handwriting Recognition Component</a><br />
&nbsp;&nbsp;&nbsp;&nbsp;5.2 <a
href="#handwriting_recognition:Event_Syntax">Event Syntax</a><br />
&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;5.2.1 <a
href="#handwriting_recognition:Examples_of_events_for_preparing_the_component">
Examples of events for preparing the component</a><br />
&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;5.2.2 <a
href="#handwriting_recognition:Examples_of_events_for_starting_the_component">
Examples of events for starting the component</a><br />
&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;5.2.3 <a
href="#handwriting_recognition:Example_output_event">Example output
event</a><br />
6 <a href="#Example_simple_modality_Video_Display">Example simple
modality: Video Display</a><br />
&nbsp;&nbsp;&nbsp;&nbsp;6.1 <a
href="#Functions_of_a_Possible_Video_Display_Component">Functions
of a Possible Video Display Component</a><br />
&nbsp;&nbsp;&nbsp;&nbsp;6.2 <a
href="#Event_Syntax_video_output">Event Syntax</a><br />
&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;6.2.1 <a
href="#Examples_of_events_for_starting_the_component_video_output">Examples
of events for starting the component</a><br />
</p>

<h3><a name="appendices" id="appendices"></a>Appendix</h3>

<p class="toc">A <a href="#references">References</a><br />
</p>
</div>

<hr />
<div class="body">
<div class="div1">
<h2><a name="introduction" id="introduction"></a>1
Introduction</h2>

<p>The W3C Multimodal Interaction (MMI) Working Group develops an
architecture <a href="#MMI-ARCH">[MMI-ARCH]</a> for the Multimodal
Interaction framework <a href="#MMIF">[MMIF]</a>. The Multimodal
Architecture describes a general and flexible framework for
interoperability of the various components of the multimodal
framework (e.g. modality components (MC) and the interaction
manager (IM)) in an abstract way. Among others it defines
interfaces and messages between the constituents of the framework,
but it is up to the implementation to decide how these messages are
transferred in case of a distributed implementation.</p>

<p>This Note is an informative supplement to the Multimodal
Architecture and Interfaces specification <a
href="#MMI-ARCH">[MMI-ARCH]</a>. In contrast to the Multimodal
Architecture specification, which defines normative conformance for
multimodal constituents, the intention of this document is to
provide additional informative guidelines for authors of MMI
modality components. Its purpose is to assist authors in maximizing
the usefulness of their Multimodal Architecture conformant
constituents by describing additional information which will enable
constituents to be more easily incorporated into a multimodal
system. This additional suggested information includes, for
example, descriptions of how the constituent behaves with respect
to the optional aspects of the Architecture. The specific goals of
the guidelines in this document are to:</p>

<ol>
<li>promote interoperability when constituents from different
vendors are used in the same system by suggesting important
information that can be provided along with a constituent that will
enable others to use the constituent effectively (<a
href="#Modality_component_guidelines"><b>2 Modality component
guidelines</b></a> )</li>

<li>provide suggestions for authoring Modality Components in order
to maximize their effectiveness (<a
href="#Modality_component_design_suggestions"><b>3 Modality
component design suggestions</b></a> )</li>

<li>provide illustrations of these suggestions using sample
modality components for face identification and handwriting
recognition (<a
href="#Example_simple_modality_Face_Identification"><b>4 Example
simple modality: Face Identification</b></a> and <a
href="#Example_simple_modality_Form-filling_using_Handwriting_Recognition">
<b>5 Example simple modality: Form-filling using Handwriting
Recognition</b></a> )</li>
</ol>

<br />
<br />
</div>

<div class="div1">
<h2><a name="Modality_component_guidelines"
id="Modality_component_guidelines"></a>2 Modality component
guidelines</h2>

<p>The following guidelines guarantee that modalities are portable
from interaction manager to interaction manager.</p>

<div class="div2">
<h3><a name="Guideline1" id="Guideline1"></a>2.1 Guideline 1: Each
modality component must implement all of the MMI life-cycle
events.</h3>

<p>The MMI life-cycle events are the mechanism through which a
modality component communicates with the interaction manager. The
Modality Component (MC) author must define how the modality
component will respond to each life-cycle event. A modality
component must respond to every life-cycle event it receives from
the interaction manager in the cases where a response is required,
as defined by the MMI Architecture. For example, if a modality
component presents a static display, it must respond to a
&lt;pause&gt; event with a &lt;pauseResponse&gt; event even if the
static display modality component does nothing else in response to
the &lt;pause&gt; event.</p>

<p>For each life-cycle event, define the parameters and syntax of
the "data" element of the corresponding the life-cycle event that
will be used in performing that function. For example, the
&lt;startRequest&gt; event for a speech recognition modality
component might include parameters like timeout, confidence
threshold, max n-best, and grammar.</p>
</div>

<div class="div2">
<h3><a name="Guideline2" id="Guideline2"></a>2.2 Guideline 2:
Identify other functions of the modality component that are
relevant to the interaction manager.</h3>

<p>Define an &lt;extensionNotification&gt; event to communicate
these functions to and from the interaction manager.</p>
</div>

<div class="div2">
<h3><a name="Guideline3" id="Guideline3"></a>2.3 Guideline 3: If
the component uses media, specify the media format. For example,
audio formats for speech recognition, or InkML for handwriting
recognition.</h3>
</div>

<div class="div2">
<h3><a name="Guideline4" id="Guideline4"></a>2.4 Guideline 4:
Specify protocols for use between the component and the Interaction
Manager (IM) (e.g., SIP or HTTP).</h3>
</div>

<div class="div2">
<h3><a name="Guideline5" id="Guideline5"></a>2.5 Guideline 5:
Specify supported human languages, e.g., English, German, Chinese
and locale, if relevant.</h3>
</div>

<div class="div2">
<h3><a name="Guideline6" id="Guideline6"></a>2.6 Guideline 6:
Specify supporting languages required by the component, if
any.</h3>

<p>For example:</p>

<ul>
<li><a href="#SSML">[SSML]</a> for a speech synthesis simple
modality component</li>

<li><a href="#SRGS">[SRGS]</a> and <a href="#SISR">[SISR]</a> for a
speech recognition simple modality component</li>

<li><a href="#VoiceXML">[VoiceXML]</a>, <a href="#SSML">[SSML]</a>,
<a href="#SRGS">[SRGS]</a>, and <a href="#SISR">[SISR]</a> for a
speech complex modality component</li>
</ul>
</div>

<div class="div2">
<h3><a name="Guideline7" id="Guideline7"></a>2.7 Guideline 7:
Modality components sending data to the interaction manager must
use the <a href="#EMMA">[EMMA]</a> format where appropriate.</h3>

<p>If a modality component captures or generates information, then
it should format the information using the EMMA format and use an
extension event to send that information to the interaction
manager.</p>
</div>

<div class="div2">
<h3><a name="Guideline8" id="Guideline8"></a>2.8 Guideline 8:
Specify error codes and their meanings to be returned to the
IM.</h3>

<p>The MC developer must specify all error codes that are specific
to the component. If the MC is based on another technology, the
developer can provide a reference to that technology specification.
For instance, if the MC is based on VoiceXML, a reference to the
VoiceXML spec for VoiceXML errors can be included instead of
listing each VoiceXML error.</p>

<p>Errors such as XML errors and MMI protocol errors must be
handled in accordance with the guidelines laid out in the MMI
architecture. These errors do not need to be documented.</p>
</div>
</div>

<div class="div1">
<h2><a name="Modality_component_design_suggestions"
id="Modality_component_design_suggestions"></a>3 Modality component
design suggestions</h2>

<p>The following design suggestions should be helpful for modality
component authors to make modalities portable from interaction
manager to interaction manager.</p>

<div class="div2">
<h3><a name="DesignSuggestion1" id="DesignSuggestion1"></a>3.1
Design suggestion 1: Consider constructing a complex modality
component with multiple functions if one function handles the
errors generated by another function.</h3>

<p>For example, if the ASR fails to recognize a user's utterance, a
prompt may be presented to the user asking the user to try again by
the TTS function. As another example, if the ASR fails to recognize
a user's utterance, a GUI function might display the n-best list on
a screen so the user can select the desired word. Efficiency
concerns may indicate that two modality components be combined into
a single complex modality component.</p>
</div>

<div class="div2">
<h3><a name="DesignSuggestion2" id="DesignSuggestion2"></a>3.2
Design suggestion 2: Consider constructing a complex modality
component with multiple functions rather than several simple
modality components if the functions need to be synchronized.</h3>

<p>For example, a TTS function must be synchronized with a visual
talking head so that the lip movements are synchronized with the
words. As another example, a TTS functions presents information
about the each graphical item that the user places "in focus."
Again, efficiency concerns may indicate that the TTS and talking
head be two modality components be combined into a single complex
modality component.</p>
</div>

<div class="div2">
<h3><a name="DesignSuggestion3" id="DesignSuggestion3"></a>3.3
Design suggestion 3: Consider constructing a nested modality
component with multiple child modality components if the children
modality components are frequently used together but do not handle
ther errors generated by the other children components and the
children components do not need to be extensively
synchronized.</h3>

<p>Writing an application using a nested modality component may be
easier than writing the same application using multiple modality
components if the nested modality component hides much of the
complexity of managing the children modality components.</p>
</div>
</div>

<div class="div1">
<h2><a name="Example_simple_modality_Face_Identification"
id="Example_simple_modality_Face_Identification"></a>4 Example
simple modality: Face Identification</h2>

<div class="div2">
<h3><a name="Functions_of_a_Possible_Face_Identification_Component"
id="Functions_of_a_Possible_Face_Identification_Component"></a>4.1
Functions of a Possible Face Identification Component</h3>

<p>Consider a theoretical face identification modality component
that takes an image or images of a face and returns the set of
possible matches and the confidence of the face identification
software in each match. An API to that modality component would
include events for starting the component, providing data, and for
receiving results back from the component.</p>

<p>This particular example includes the information needed to run
this component in the "startRequest" and "doneNotification" events;
that is, in this example no "extensionNotification" events are
used, although extensionNotification events could be part of
another modality component's API. This example assumes that an
image has already been acquired from some source; however, another
possibility would be to also include image acquisition in the
operation of the component.</p>

<p>Depending on the capabilities of the modality component, other
possible information that might be included would be some useful
non-functional information as the capturing context of the still
picture (e.g. indoor picture or outdoor picture) or the type of
image (e.g. a portrait photography or a street photography) or
would be some technical information as the algorithm to be used or
the image format to expect. We emphasize that this is just an
example to indicate the kinds of information that might be used by
a multimodal application that includes face recognition. The actual
interface used in real applications should be defined by experts in
the field.</p>

<p>The use case is a face identification component that identifies
one of a set of employees on the basis of face images.</p>

<p>The MMI Runtime Framework could use the following events to
communicate with such a component.</p>

<a
name="table:Component_behavior_of_face_identification_with_respect_to_modality_component_rules"
 id="table:Component_behavior_of_face_identification_with_respect_to_modality_component_rules">
</a>

<table border="1"
summary="Component behavior of face identification with respect to modality component guidelines">
<caption>Table 1: Component behavior of Face Identification with
respect to modality component guidelines.</caption>

<thead>
<tr>
<th>Guideline</th>
<th>Component Information</th>
</tr>
</thead>

<tbody>
<tr>
<td>Guideline 1: Each modality component must implement all of the
MMI life cycle events.</td>
<td>See Table 2 for the details of the implementation of the
life-cycle events.</td>
</tr>

<tr>
<td>Guideline 2: Identify other functions of the modality component
that are relevant to the interaction manager.</td>
<td>All the functions of the component are covered in the
life-cycle events, no other functions are needed.</td>
</tr>

<tr>
<td>Guideline 3: If the component uses media, specify the media
format.</td>
<td>The component uses the jpeg format for images to be identified
and for its image database.</td>
</tr>

<tr>
<td>Guideline 4: Specify protocols supported by the component for
transmitting media (e.g. SIP).</td>
<td>The component uses HTTP for transmitting media.</td>
</tr>

<tr>
<td>Guideline 5: Specify supported human languages.</td>
<td>This component does not support any human languages.</td>
</tr>

<tr>
<td>Guideline 6: Specify supporting languages required by the
component.</td>
<td>This component does not require any markup languages.</td>
</tr>

<tr>
<td>Guideline 7: Modality components sending data to the
interaction manager must use the EMMA format.</td>
<td>This component uses EMMA.</td>
</tr>
</tbody>
</table>

<a
name="table:Component_behavior_of_face_identification_for_life-cycle_event"
 id="table:Component_behavior_of_face_identification_for_life-cycle_event">
</a>

<table border="1"
summary="Component behavior of face identification for each life-cycle event">
<caption>Table 2: Component behavior of face identification for
each life-cycle event.</caption>

<thead>
<tr>
<th>Life Cycle Event</th>
<th>Component Implementation</th>
</tr>
</thead>

<tbody>
<tr>
<td>newContextRequest</td>
<td>(Standard) The component requests a new context from the
IM.</td>
</tr>

<tr>
<td>newContextResponse</td>
<td>(Standard) The component starts a new context and assigns the
new context id to it.</td>
</tr>

<tr>
<td>prepareRequest</td>
<td>The component prepares resources to be used in identification,
specifically, the image database.</td>
</tr>

<tr>
<td>prepareResponse</td>
<td>(Standard) If the database of known users is not found, the
error message "known users not found" is returned in the
&lt;statusInfo&gt; element.</td>
</tr>

<tr>
<td>startRequest</td>
<td>The component starts processing if possible, using a specified
image, image database, threshold, and limit on the size of nbest
results to be returned.</td>
</tr>

<tr>
<td>startResponse</td>
<td>(Standard) If the database of known users is not found, the
error message "known users not found" is returned in the
&lt;statusInfo&gt; element.</td>
</tr>

<tr>
<td>doneNotification</td>
<td>Identification results in EMMA format are reported in the
"data" field.The mode is "photograph", the medium is "visual", the
function is "identification", and verbal is "false".</td>
</tr>

<tr>
<td>cancelRequest</td>
<td>This component stops processing when it receives a
"cancelRequest". It always performs a hard stop whether or not the
IM requests a hard stop.</td>
</tr>

<tr>
<td>cancelResponse</td>
<td>(Standard)</td>
</tr>

<tr>
<td>pauseRequest</td>
<td>This component cannot pause.</td>
</tr>

<tr>
<td>pauseResponse</td>
<td>&lt;statusInfo&gt; field is "cannot pause".</td>
</tr>

<tr>
<td>resumeRequest</td>
<td>This component cannot resume.</td>
</tr>

<tr>
<td>resumeResponse</td>
<td>&lt;statusInfo&gt; field is "cannot resume".</td>
</tr>

<tr>
<td>extensionNotification</td>
<td>This component does not use "extensionNotification". It ignores
any "extensionNotification" events that are sent to it by the
IM.</td>
</tr>

<tr>
<td>clearContextRequest</td>
<td>(Standard)</td>
</tr>

<tr>
<td>clearContextResponse</td>
<td>(Standard)</td>
</tr>

<tr>
<td>statusRequest</td>
<td>(Standard)</td>
</tr>

<tr>
<td>statusResponse</td>
<td>The component returns a standard life cycle response. The
"automaticUpdate" attribute is "false", because this component does
not supply automatic updates.</td>
</tr>
</tbody>
</table>

<p><em>Note:</em> "(Standard)" means that the component does not do
anything over and above the actions specified by the MMI
Architecture.</p>
</div>

<div class="div2">
<h3><a name="Event_Syntax" id="Event_Syntax"></a>4.2 Event
Syntax</h3>

<div class="div3">
<h4><a name="Examples_of_events_for_starting_the_component"
id="Examples_of_events_for_starting_the_component"></a>4.2.1
Examples of events for starting the component</h4>

<p>To start the component, a startRequest event from the IM to the
face identification component is sent, asking it to start an
identification. It assumes that images found at a certain URI are
to be identified by comparing them against a known set of employees
found at another URI. The confidence threshold of the component is
set to .5 and the IM requests a maximum of five possible
matches.</p>

<div class="exampleInner">
<pre>
&lt;mmi xmlns="http://www.w3.org/2008/04/mmi-arch" version="1.0"&gt;
  &lt;mmi:startRequest source="uri:RTFURI" context="URI-1" requestID="request-1"&gt;
    &lt;mmi:data&gt;
      &lt;face-identification-parameters threshold=".5" unknown="someURI" known="uri:employees" max-nbest="5"/&gt;
    &lt;/mmi:data&gt;
  &lt;/mmi:startRequest&gt;
&lt;/mmi:mmi&gt;
</pre>
</div>

<p>As part of support for the life-cycle events, a modality
component is required to respond to a startRequest event with a
startResponse event. Here's an example of a startResponse from the
face identification component to the IM informing the IM that the
face identification component has successfully started.</p>

<div class="exampleInner">
<pre>
&lt;mmi xmlns="http://www.w3.org/2008/04/mmi-arch" version="1.0"&gt;
  &lt;mmi:startResponse source="uri:faceURI" context="URI-1" requestID="request-1" status="success"/&gt;
 &lt;/mmi:mmi&gt;
</pre>
</div>

<p>Here's an example of a startResponse event from the face
identification component to the IM in the case of failure, with an
example failure message. In this case the failure message indicates
that the known images cannot be found.&nbsp;</p>

<div class="exampleInner">
<pre>
&lt;mmi xmlns="http://www.w3.org/2008/04/mmi-arch" version="1.0"&gt;
  &lt;mmi:startResponse source="uri:faceURI" context="URI-1" requestID="request-1" status="failure"&gt;
    &lt;mmi:statusInfo&gt;
      known users not found
    &lt;/mmi:statusInfo&gt;
  &lt;/mmi:startResponse&gt;
&lt;/mmi:mmi&gt;
</pre>
</div>
</div>

<div class="div3">
<h4><a name="Example_output_event"
id="Example_output_event"></a>4.2.2 Example output event</h4>

<p>Here's an example of an output event, sent from the face
identification component to the IM, using EMMA to represent the
identification results. Two results with different confidences are
returned.</p>

<div class="exampleInner">
<pre>
&lt;mmi xmlns="http://www.w3.org/2008/04/mmi-arch" version="1.0"&gt;
  &lt;mmi:doneNotification source="uri:faceURI" context="URI-1" status="success" requestID="request-1"&gt;
    &lt;mmi:data&gt;
      &lt;emma:emma version="1.0" xmlns:emma="http://www.w3.org/2003/04/emma"&gt;
        &lt;emma:one-of emma:medium="visual" emma:verbal="false" emma:mode="photograph" emma:function="identification"&gt;
          &lt;emma:interpretation id="int1" emma:confidence=".75"&gt;
            &lt;person&gt;12345&lt;/person&gt;
            &lt;name&gt;Mary Smith&lt;/name&gt;
          &lt;/emma:interpretation&gt;
          &lt;emma:interpretation id="int2" emma:confidence=".6"&gt;
            &lt;person&gt;67890&lt;/person&gt;
            &lt;name&gt;Jim Jones&lt;/name&gt;
          &lt;/emma:interpretation&gt;
        &lt;/emma:one-of&gt;
      &lt;/emma:emma&gt;
    &lt;/mmi:data&gt;
  &lt;/mmi:doneNotification&gt;
&lt;/mmi:mmi&gt;
</pre>
</div>

<p>This is an example of EMMA output in the case where the face
image doesn't match any of the employees.</p>

<div class="exampleInner">
<pre>
&lt;mmi xmlns="http://www.w3.org/2008/04/mmi-arch" version="1.0"&gt;
  &lt;mmi:doneNotification source="uri:faceURI" context="URI-1" status="success" requestID="request-1" &gt;
    &lt;mmi:data&gt;
      &lt;emma:emma version="1.0" xmlns:emma="http://www.w3.org/2003/04/emma"&gt;
        &lt;emma:interpretation id="int1" emma:confidence="0.0"
         uninterpreted="true" emma:medium="visual" emma:mode="photograph"
         emma:function="identification"/&gt;
      &lt;/emma:emma&gt;
    &lt;/mmi:data&gt;
  &lt;/mmi:doneNotification&gt;
&lt;/mmi:mmi&gt;
</pre>
</div>
</div>
</div>
</div>

<div class="div1">
<h2><a
name="Example_simple_modality_Form-filling_using_Handwriting_Recognition"
 id="Example_simple_modality_Form-filling_using_Handwriting_Recognition">
</a>5 Example simple modality: Form-filling using Handwriting
Recognition</h2>

<div class="div2">
<h3><a
name="Functions_of_a_Possible_Handwriting_Recognition_Component"
id="Functions_of_a_Possible_Handwriting_Recognition_Component"></a>5.1
Functions of a Possible Handwriting Recognition Component</h3>

<p>Consider an ink recognition modality component for Handwriting
Recognition (HWR) that takes digital ink written using an
electronic pen or stylus, performs recognition and returns the
recognized text. An API to such a modality component would include
events for initializing the component, requesting for recognition
by providing digital ink data, and for receiving recognized text
result (possibly an n-best list) back from the component as shown
in the below figure.</p>

<img class="center" src="images/InkRecognition-ja.png"
alt="Example of Japanese handwriting recognition" />

<p>Figure 1: Example of Japanese handwriting recognition</p>

<p>This example assumes that handwriting ink is captured,
represented in W3C InkML format and sent to the IM requesting for
recognition to text. The following sequences of events explain the
ink recognition request.</p>

<ol>
<li>The IM requests the ink recognition modality by sending the
"prepareRequest" event along with the parameters for configuring
the HWR system.</li>

<li>Ink recognition modality responds with the "prepareResponse"
event with the status of the configuration of the HWR system.</li>

<li>IM sends the "startRequest" event to the ink recognition
modality where the event's data field contains the InkML data to be
recognized.</li>

<li>Once the recognition is completed, the ink recognition modality
notifies the results to the IM using the "doneNotification" event
along with the recognition choices (N-best list).</li>
</ol>

<p>The use case is a form-filling application which accepts
handwriting input provided by the user on the form fields. The
inputs are recognized and displayed back as text in the
corresponding fields. An ink capture modality may be used to
capture the ink and send it to IM for recognition. The
communication between the ink capture modality and the IM is not
covered here for the sake of brevity. The below section explains
the details of the communication between the MMI Runtime Framework
(RTF) of the IM and the ink recognition modality.</p>

<a
name="table:Component_behavior_of_handwriting_recognition_with_respect_to_modality_component_rules"
 id="table:Component_behavior_of_handwriting_recognition_with_respect_to_modality_component_rules">
</a>

<table border="1"
summary="Component behavior of handwriting recognition with respect to modality component guidelines">
<caption>Table 3: Component behavior of Ink modality with respect
to modality component guidelines.</caption>

<thead>
<tr>
<th>Guideline</th>
<th>Component Information</th>
</tr>
</thead>

<tbody>
<tr>
<td>Guideline 1: Each modality component must implement all of the
MMI life cycle events.</td>
<td>See Table 4 for the details of the implementation of the
life-cycle events.</td>
</tr>

<tr>
<td>Guideline 2: Identify other functions of the modality component
that are relevant to the interaction manager.</td>
<td>All the functions of the component are covered in the
life-cycle events, no other functions are needed.</td>
</tr>

<tr>
<td>Guideline 3: If the component uses media, specify the media
format.</td>
<td>The component uses W3C InkML format to represent handwriting
data (digital ink).</td>
</tr>

<tr>
<td>Guideline 4: Specify protocols supported by the component for
transmitting media (e.g. SIP).</td>
<td>The component uses HTTP for transmitting media. Other standard
protocols such as TCP may also be explored.</td>
</tr>

<tr>
<td>Guideline 5: Specify supported human languages.</td>
<td>Virtually any human language script can be supported based on
the HWR component capability.</td>
</tr>

<tr>
<td>Guideline 6: Specify supporting languages required by the
component.</td>
<td>W3C InkML for representing the handwriting data.</td>
</tr>

<tr>
<td>Guideline 7: Modality components sending data to the
interaction manager must use the EMMA format.</td>
<td>This component uses EMMA.</td>
</tr>
</tbody>
</table>

<a
name="table:Component_behavior_of_handwriting_recognition_for_life-cycle_event"
 id="table:Component_behavior_of_handwriting_recognition_for_life-cycle_event">
</a>

<table border="1"
summary="Component behavior of handwriting recognition for each life-cycle event">
<caption>Table 4: Component behavior of handwriting recognition for
each life-cycle event.</caption>

<thead>
<tr>
<th>Life Cycle Event</th>
<th>Component Implementation</th>
</tr>
</thead>

<tbody>
<tr>
<td>newContextRequest</td>
<td>(Standard) The component requests a new context from the
IM.</td>
</tr>

<tr>
<td>newContextResponse</td>
<td>(Standard) The component starts a new context and assigns the
new context id to it.</td>
</tr>

<tr>
<td>prepareRequest</td>
<td>The component prepares resources to be used in recognition.
Based on the 'script' parameter, it first selects an appropriate
recognizer. It also configures the recognizer with other parameters
such as recognition confidence threshold, limit on the size of
n-best results to be returned etc., when available.</td>
</tr>

<tr>
<td>prepareResponse</td>
<td>(Standard) If the recognizer failed to find a matching
recognizer for the request language script, a relevant error
message is returned in the &lt;statusInfo&gt; element.</td>
</tr>

<tr>
<td>startRequest</td>
<td>The component performs recognition of the handwriting
input.</td>
</tr>

<tr>
<td>startResponse</td>
<td>(Standard)The status of recognition as "success" or "failure"
is returned in the &lt;statusInfo&gt; element.</td>
</tr>

<tr>
<td>doneNotification</td>
<td>Identification results in EMMA format are reported in the
"data" field. The mode is "ink", the medium is "tactile", the
function is "transcription", and verbal is "true".</td>
</tr>

<tr>
<td>cancelRequest</td>
<td>This component stops processing when it receives a
"cancelRequest". It always performs a hard stop irrespective of the
IM request.</td>
</tr>

<tr>
<td>cancelResponse</td>
<td>(Standard)</td>
</tr>

<tr>
<td>pauseRequest</td>
<td>This component cannot pause.</td>
</tr>

<tr>
<td>pauseResponse</td>
<td>&lt;statusInfo&gt; field is "cannot pause".</td>
</tr>

<tr>
<td>resumeRequest</td>
<td>This component cannot resume.</td>
</tr>

<tr>
<td>resumeResponse</td>
<td>&lt;statusInfo&gt; field is "cannot resume".</td>
</tr>

<tr>
<td>extensionNotification</td>
<td>This component does not use "extensionNotification". It ignores
any "extensionNotification" events that are sent to it by the
IM.</td>
</tr>

<tr>
<td>clearContextRequest</td>
<td>(Standard)</td>
</tr>

<tr>
<td>clearContextResponse</td>
<td>(Standard)</td>
</tr>

<tr>
<td>statusRequest</td>
<td>(Standard)</td>
</tr>

<tr>
<td>statusResponse</td>
<td>The component returns a standard life cycle response. The
"automaticUpdate" attribute is "false", because this component does
not supply automatic updates.</td>
</tr>
</tbody>
</table>

<p><em>Note:</em> "(Standard)" means that the component does not do
anything over and above the actions specified by the MMI
Architecture.</p>
</div>

<div class="div2">
<h3><a name="handwriting_recognition:Event_Syntax"
id="handwriting_recognition:Event_Syntax"></a>5.2 Event Syntax</h3>

<div class="div3">
<h4><a
name="handwriting_recognition:Examples_of_events_for_preparing_the_component"
 id="handwriting_recognition:Examples_of_events_for_preparing_the_component">
</a>5.2.1 Examples of events for preparing the component</h4>

<p>IM send a prepareRequest event to the ink recognition component.
The ink recognition component selects an appropriate recognizer
that matches the given language script, in this example it is set
to "English_Lowercase". The "RecoGrammar.xml" grammar file contains
constraints that aid the recognizer. The confidence threshold of
the component is set to .7 and the IM requests a maximum of five
possible matches. Based on the capability of the recognizer, other
possible parameters such as a 'user profile' that contains
user-specific information can be provided.</p>

<div class="exampleInner">
<pre>
&lt;mmi:mmi version="1.0" xmlns:mmi="http://www.w3.org/2008/04/mmi-arch"&gt;
  &lt;mmi:prepareRequest source="uri:RTFURI" context="URI-1" requestID="request-1"&gt;
    &lt;mmi:data&gt;
      &lt;ink-recognition-parameters grammar="RecoGrammar.xml" threshold=".7" script="English_Lowercase" max-nbest="5"/&gt;
    &lt;/mmi:data&gt;
  &lt;/mmi:prepareRequest&gt;
&lt;/mmi:mmi&gt;
</pre>
</div>

<p>As part of support for the life cycle events, a modality
component is required to respond to a prepareRequest event with a
prepareResponse event. Here's an example of a prepareResponse from
the ink recognition component to the IM informing the IM that the
ink recognition component has successfully initialized.</p>

<div class="exampleInner">
<pre>
&lt;mmi:mmi xmlns:mmi="http://www.w3.org/2008/04/mmi-arch" version="1.0"&gt;
  &lt;mmi:prepareResponse source="uri:inkRecognizerURI" context="URI-1" requestID="request-1" status="success"/&gt;
&lt;/mmi:mmi&gt;
</pre>
</div>

<p>Here's an example of a prepareResponse event from the ink
recognition component to the IM in the case of failure, with an
example failure message. In this case the failure message indicates
that the language script is not supported.</p>

<div class="exampleInner">
<pre>
&lt;mmi:mmi xmlns:mmi="http://www.w3.org/2008/04/mmi-arch" version="1.0"&gt;
  &lt;mmi:prepareResponse source="uri:inkRecognizerURI" context="URI-1" requestID="request-1" status="failure"&gt;
    &lt;mmi:statusInfo&gt;
      Given language script not supported
    &lt;/mmi:statusInfo&gt;
  &lt;/mmi:prepareResponse&gt;
&lt;/mmi:mmi&gt;
</pre>
</div>
</div>

<div class="div3">
<h4><a
name="handwriting_recognition:Examples_of_events_for_starting_the_component"
 id="handwriting_recognition:Examples_of_events_for_starting_the_component">
</a>5.2.2 Examples of events for starting the component</h4>

<p>To start the component and recognize the handwriting data, a
startRequest event from the IM to the ink recognition component is
sent. The data field of the event contains InkML representation of
the ink data.</p>

<p>Along with the ink, additional information such as the reference
co-ordinate system and capture device's resolution may also be
provided in the InkML data. The below example shows that the ink
strokes have X and Y channels and the ink has been captured at a
resolution of 1000 DPI. The example ink data contains strokes of
the Japanese character "手" (te) which means "hand".</p>

<div class="exampleInner">
<pre>
&lt;mmi:mmi xmlns:mmi="http://www.w3.org/2008/04/mmi-arch" version="1.0"&gt;
  &lt;mmi:startRequest source="uri:inkRecognizerURI" context="URI-1" requestID="request-1"&gt;
    &lt;mmi:data&gt;
      &lt;ink:ink version="1.0" xmlns:ink="http://www.w3.org/2003/InkML"&gt;
        &lt;ink:definitions&gt;
          &lt;ink:context id="device1Context"&gt;
            &lt;ink:traceFormat id="strokeFormat&gt;
              &lt;ink:channel name="X" type="decimal"&gt;
                &lt;ink:channelProperty name="resolution" value="1000" units="1/in"/&gt;
              &lt;/ink:channel&gt;
              &lt;ink:channel name="Y" type="decimal"&gt;
                &lt;ink:channelProperty name="resolution" value="1000" units="1/in"/&gt;
              &lt;/ink:channel&gt;
            &lt;/ink:traceFormat&gt;
          &lt;/ink:context&gt;
        &lt;/ink:definitions&gt;
       &lt;ink:traceGroup contextRef="#device1Context"&gt;
        &lt;ink:trace&gt;
          106 81, 105 82, 104 84, 103 85, 101 88, 100 90, 99 91, 97 97,
          89 105, 88 107, 87 109, 86 110, 84 111, 84 112, 82 113, 78 117,
          74 121, 72 122, 70 123, 68 125, 67 125, 66 126, 65 126, 63 127,
          57 129, 53 133, 47 135, 46 136, 45 136, 44 137, 43 137, 43 137
        &lt;/ink:trace&gt;

        &lt;ink:trace&gt;
          28 165, 29 165, 31 165, 33 165, 35 164, 37 164, 38 164, 40 163,
          42 163, 45 163, 49 162, 51 162, 53 162, 56 162, 58 162, 64 160,
          69 160, 71 159, 74 159, 76 159, 78 159, 86 157, 91 157, 95 157,
          96 157, 99 157, 101 157, 103 157, 109 155, 111 155, 114 155,
          116 155, 119 155, 121 154, 124 154, 126 154, 127 154, 129 154,
          131 154, 134 153, 135 153, 136 153, 137 153, 138 153, 139 153,
          140 153, 141 153, 142 153, 143 153, 144 153, 145 153, 145 153
        &lt;/ink:trace&gt;

        &lt;ink:trace&gt;
          10 218, 12 218, 14 218, 20 216, 25 216, 28 216, 31 216, 34 216,
          37 216, 45 216, 53 216, 58 215, 60 215, 63 215, 68 215, 72 215,
          74 215, 77 215, 85 212, 88 212, 94 210, 100 208, 105 208, 107 208,
          109 208, 110 208, 111 207, 114 207, 115 207, 119 207, 121 207,
          123 207, 124 207, 128 206, 130 205, 131 205, 134 205, 136 205,
          137 205, 138 205, 139 204, 140 204, 141 204, 142 204, 143 204,
          144 204, 145 204, 146 204, 147 204, 148 204, 149 204, 150 204,
          151 203, 152 203, 153 203, 154 203, 155 203, 156 203, 158 203,
          159 202, 160 202, 161 202, 162 202, 163 202, 164 202, 165 202,
          166 202, 167 202, 168 202, 169 202, 170 202, 171 202, 172 202,
          173 202, 173 201, 173 201
        &lt;/ink:trace&gt;

        &lt;ink:trace&gt;
          78 128, 78 127, 79 127, 79 128, 80 129, 80 130, 81 132, 82 133,
          82 134, 83 135, 84 137, 85 139, 86 141, 87 142, 88 144, 89 146,
          94 152, 95 153, 96 155, 98 160, 99 162, 100 165, 101 167, 101 169,
          102 173, 102 176, 102 181, 102 183, 102 185, 102 186, 104 192,
          104 195, 104 197, 104 199, 104 201, 104 203, 104 205, 104 206,
          104 207, 104 208, 104 209, 104 210, 104 211, 104 213, 104 214,
          104 215, 104 216, 104 217, 104 218, 104 220, 103 222, 102 223,
          102 224, 102 223, 102 224, 103 225, 103 228, 103 229, 103 230,
          103 231, 103 232, 103 233, 103 236, 103 239, 103 242, 103 243,
          103 247, 103 248, 102 249, 102 250, 102 251, 101 251, 100 253,
          99 255, 99 256, 98 257, 97 258, 97 259, 96 260, 96 261, 95 262,
          95 263, 94 264, 94 265, 93 266, 93 267, 92 268, 91 269, 91 270,
          90 271, 90 272, 89 273, 89 274, 88 275, 88 276, 87 276, 87 277,
          86 277, 86 278, 85 279, 85 280, 84 281, 83 282, 82 284, 82 285,
          81 285, 80 286, 79 287, 78 288, 77 288, 77 289, 76 290, 75 290,
          75 291, 74 291, 74 290, 74 289, 74 288, 74 287, 73 287, 73 286,
          73 285, 72 284, 72 281, 71 280, 70 279, 70 278, 69 277, 68 276,
          67 275, 65 274, 62 272, 60 271, 59 271, 58 270, 57 270, 56 269,
          55 268, 54 268, 53 267, 52 267, 51 267, 49 267, 48 267, 48 266,
          48 266
        &lt;/ink:trace&gt;
       &lt;/ink:traceGroup&gt;
      &lt;/ink:ink&gt;
    &lt;/mmi:data&gt;
   &lt;/mmi:startRequest&gt;
&lt;/mmi:mmi&gt;
</pre>
</div>

<p>As part of support for the life cycle events, a modality
component is required to respond to a startRequest event with a
startResponse event. Here's an example of a startResponse from the
ink recognition component to the IM informing the IM that the ink
recognition component has successfully started.</p>

<div class="exampleInner">
<pre>
&lt;mmi:mmi xmlns:mmi="http://www.w3.org/2008/04/mmi-arch" version="1.0"&gt;
  &lt;mmi:startResponse source="uri:inkRecognizerURI" context="URI-1" requestID="request-1" status="success"/&gt;
&lt;/mmi:mmi&gt;
</pre>
</div>

<p>Here's an example of a startResponse event from the ink
recognition component to the IM in the case of failure, with an
example failure message. In this case the failure message indicates
that the recognition failed due to invalid data format of the
handwriting data.</p>

<div class="exampleInner">
<pre>
&lt;mmi:mmi xmlns:mmi="http://www.w3.org/2008/04/mmi-arch" version="1.0"&gt;
  &lt;mmi:startResponse source="uri:inkRecognizerURI" context="URI-1" requestID="request-1" status="failure"&gt;
    &lt;mmi:statusInfo&gt;
      Invalid data format
    &lt;/mmi:statusInfo&gt;
  &lt;/mmi:startResponse&gt;
&lt;/mmi:mmi&gt;
</pre>
</div>
</div>

<div class="div3">
<h4><a name="handwriting_recognition:Example_output_event"
id="handwriting_recognition:Example_output_event"></a>5.2.3 Example
output event</h4>

<p>Here's an example of an output event, sent from the ink
recognition component to the IM, using EMMA to represent the
identification results. Two results with different confidences are
returned.</p>

<div class="exampleInner">
<pre>
&lt;mmi:mmi xmlns:mmi="http://www.w3.org/2008/04/mmi-arch" version="1.0"&gt;
  &lt;mmi:doneNotification source="uri:inkRecognizerURI" context="URI-1" status="success" requestID="request-1"&gt;
    &lt;mmi:data&gt;
      &lt;emma:emma version="1.0" xmlns:emma="http://www.w3.org/2003/04/emma"&gt;
        &lt;emma:one-of emma:medium="tactile" emma:verbal="true"
         emma:mode="ink" emma:function="transcription"&gt;
          &lt;emma:interpretation id="int1" emma:confidence=".8"&gt;
            &lt;text&gt; 手 &lt;/text&gt;
          &lt;/emma:interpretation&gt;
          &lt;emma:interpretation id="int2" emma:confidence=".7"&gt;
           &lt;text&gt; 于 &lt;/text&gt;
          &lt;/emma:interpretation&gt;
        &lt;/emma:one-of&gt;
      &lt;/emma:emma&gt;
    &lt;/mmi:data&gt;
  &lt;/mmi:doneNotification&gt;
&lt;/mmi:mmi&gt;
</pre>
</div>

<p>This is an example of EMMA output in the case where the
recognizer is unable to find a suitable match to the input
handwriting. The EMMA output contains an empty interpretation
result.</p>

<div class="exampleInner">
<pre>
&lt;mmi:mmi xmlns:mmi="http://www.w3.org/2008/04/mmi-arch" version="1.0"&gt;
  &lt;mmi:doneNotification source="uri:inkRecognizerURI" context="URI-1" status="success" requestID="request-1" &gt;
    &lt;mmi:data&gt;
      &lt;emma:emma version="1.0" xmlns:emma="http://www.w3.org/2003/04/emma"&gt;
        &lt;emma:interpretation id="int1" emma:confidence="0.0"
         emma:medium="tactile" emma:verbal="true" emma:mode="ink"
         emma:function="transcription" emma:uninterpreted="true"/&gt;
      &lt;/emma:emma&gt;
    &lt;/mmi:data&gt;
  &lt;/mmi:doneNotification&gt;
&lt;/mmi:mmi&gt;
</pre>
</div>
</div>
</div>
</div>

<div class="div1">
<h2><a name="Example_simple_modality_Video_Display"
id="Example_simple_modality_Video_Display"></a>6 Example simple
modality: Video Display</h2>

<div class="div2">
<h3><a name="Functions_of_a_Possible_Video_Display_Component"
id="Functions_of_a_Possible_Video_Display_Component"></a>6.1
Functions of a Possible Video Display Component</h3>

<p>Consider a theoretical video display modality component that
receives a video file and displays it in a screen. An API to that
modality component would include events for starting the component,
providing the video file and for receiving player status
information back from the component. This example includes the
information needed to run this component in the "startRequest"
event and shows a display codec problem. In order to focus in the
behavior of the output modality component, this example assumes
that the video file is given from some source; however, another
possibility would be to also include video acquisition in a
composite (input/output) and more complex real-time display
component. Depending on the capabilities of the modality component,
other possible information that might be included would be the
video formats supported. The MMI Runtime Framework could use the
following events to communicate with such a component.</p>

<a name="table:Component_behavior_of_video_component"
id="table:Component_behavior_of_video_component"></a>

<table border="1"
summary="Component behavior of video component with respect to modality component guidelines">
<caption>Table 5: Component behavior of Video Display with respect
to modality component guidelines.</caption>

<thead>
<tr>
<th>Guideline</th>
<th>Component Information</th>
</tr>
</thead>

<tbody>
<tr>
<td>Guideline 1: Each modality component must implement all of the
MMI life cycle events.</td>
<td>See Table 6 for the details of the implementation of the
life-cycle events.</td>
</tr>

<tr>
<td>Guideline 2: Identify other functions of the modality component
that are relevant to the interaction manager.</td>
<td>All the functions of the component are covered in the
life-cycle events, no other functions are needed.</td>
</tr>

<tr>
<td>Guideline 3: If the component uses media, specify the media
format.</td>
<td>The component uses for the moment only the h.264 codec
format.</td>
</tr>

<tr>
<td>Guideline 4: Specify protocols supported by the component for
transmitting media (e.g. SIP).</td>
<td>The component uses HTTP for transmitting media.</td>
</tr>

<tr>
<td>Guideline 5: Specify supported human languages.</td>
<td>This component does not support any human languages.</td>
</tr>

<tr>
<td>Guideline 6: Specify supporting languages required by the
component.</td>
<td>This component does not require any markup languages.</td>
</tr>

<tr>
<td>Guideline 7: Modality components sending data to the
interaction manager must use the EMMA format.</td>
<td>This component uses EMMA.</td>
</tr>
</tbody>
</table>

<a
name="table:Component_behavior_of_video_output_for_life-cycle_event"
 id="table:Component_behavior_of_video_output_for_life-cycle_event">
</a>

<table border="1"
summary="Component behavior of video output component for each life-cycle event">
<caption>Table 6: Component behavior of Video Display for each
life-cycle event.</caption>

<thead>
<tr>
<th>Life Cycle Event</th>
<th>Component Implementation</th>
</tr>
</thead>

<tbody>
<tr>
<td>newContextRequest</td>
<td>(Standard) The component requests a new context from the
IM.</td>
</tr>

<tr>
<td>newContextResponse</td>
<td>(Standard) The component starts a new context and assigns the
new context id to it.</td>
</tr>

<tr>
<td>prepareRequest</td>
<td>The component prepares resources to be used in display
configuration, specifically, the supported formats table.</td>
</tr>

<tr>
<td>prepareResponse</td>
<td>(Standard) If the recognizer failed to find a matching
recognizer for the request language script, a relevant error
message is returned in the &lt;statusInfo&gt; element.</td>
</tr>

<tr>
<td>startRequest</td>
<td>The component starts displaying video if possible. The
&lt;mmi:data&gt; element might hold a
&lt;video-display-parameters&gt; element containing a "videoFile"
attribute. The "videoFile" attribute contains the URI referencing
the video content.</td>
</tr>

<tr>
<td>startResponse</td>
<td>(Standard)If the current video format (WVM) is not found in the
supported codec formats table, the error message "codec not
supported" is returned in the &lt;statusInfo&gt; element.</td>
</tr>

<tr>
<td>doneNotification</td>
<td>Display state in EMMA format are reported in the "data" field.
The mode is "video", the medium is "visual", the function is
"playing", and verbal is "false".</td>
</tr>

<tr>
<td>cancelRequest</td>
<td>This component stops processing when it receives a
"cancelRequest". It always performs a hard stop irrespective of the
IM request.</td>
</tr>

<tr>
<td>cancelResponse</td>
<td>(Standard)</td>
</tr>

<tr>
<td>pauseRequest</td>
<td>(Standard)</td>
</tr>

<tr>
<td>pauseResponse</td>
<td>(Standard)</td>
</tr>

<tr>
<td>resumeRequest</td>
<td>(Standard)</td>
</tr>

<tr>
<td>resumeResponse</td>
<td>(Standard)</td>
</tr>

<tr>
<td>extensionNotification</td>
<td>This component does not use "extensionNotification". It ignores
any "extensionNotification" events that are sent to it by the
IM.</td>
</tr>

<tr>
<td>clearContextRequest</td>
<td>(Standard)</td>
</tr>

<tr>
<td>clearContextResponse</td>
<td>(Standard)</td>
</tr>

<tr>
<td>statusRequest</td>
<td>(Standard)</td>
</tr>

<tr>
<td>statusResponse</td>
<td>The component returns a standard life cycle response. The
"automaticUpdate" attribute is "false", because this component does
not supply automatic updates.</td>
</tr>
</tbody>
</table>

<p><em>Note:</em> "(Standard)" means that the component does not do
anything over and above the actions specified by the MMI
Architecture.</p>
</div>

<div class="div2">
<h3><a name="Event_Syntax_video_output"
id="Event_Syntax_video_output"></a>6.2 Event Syntax</h3>

<div class="div3">
<h4><a
name="Examples_of_events_for_starting_the_component_video_output"
id="Examples_of_events_for_starting_the_component_video_output"></a>6.2.1
Examples of events for starting the component</h4>

<p>To start the component, a startRequest event from the IM to the
display component is sent, asking it to start a video display. It
gives information about a video file in a certain URI.</p>

<div class="exampleInner">
<pre>
&lt;mmi xmlns="http://www.w3.org/2008/04/mmi-arch" version="1.0"&gt;
  &lt;mmi:startRequest source="uri:RTFURI" context="URI-1" requestID="request-1"&gt;
    &lt;mmi:data&gt;
      &lt;video-display-parameters videoFile="someURI"/&gt;
    &lt;/mmi:data&gt;
  &lt;/mmi:startRequest&gt;
&lt;/mmi:mmi&gt;
</pre>
</div>

<p>As part of support for the life-cycle events, a modality
component is required to respond to a startRequest event with a
startResponse event. Here's an example of a startResponse from the
display component to the IM informing the IM that the component is
successfully started and video is displaying.</p>

<div class="exampleInner">
<pre>
 &lt;mmi xmlns="http://www.w3.org/2008/04/mmi-arch" version="1.0"&gt;
  &lt;mmi:startResponse source="uri:displayURI" context="URI-1" requestID="request-1" status="success"/&gt;
 &lt;/mmi:mmi&gt;
</pre>
</div>

<p>Here's an example of a startResponse event from the display
component to the IM in the case of failure, with an example failure
message. In this case the failure message indicates that the video
codec is not supported.</p>

<div class="exampleInner">
<pre>
&lt;mmi xmlns="http://www.w3.org/2008/04/mmi-arch" version="1.0"&gt;
  &lt;mmi:startResponse source="uri:displayURI" context="URI-1" requestID="request-1" status="failure"&gt;
    &lt;mmi:statusInfo&gt;
      WVM codec not supported
    &lt;/mmi:statusInfo&gt;
  &lt;/mmi:startResponse&gt;
&lt;/mmi:mmi&gt;
</pre>
</div>
</div>
</div>
</div>
</div>

<div class="back">
<h2><a name="references" id="references"></a>A References</h2>

<dl>
<dt class="label"><a name="MMI-ARCH"
id="MMI-ARCH"></a>MMI-ARCH</dt>

<dd><a
href="http://www.w3.org/TR/2008/WD-mmi-arch-20080414/"><cite>"Multimodal
Architecture and Interfaces (Working Draft)"</cite></a> , Jim
Barnett et al. editors. This specification describes a loosely
coupled architecture for multimodal user interfaces, which allows
for co-resident and distributed implementations, and focuses on the
role of markup and scripting, and the use of well defined
interfaces between its constituents. World Wide Web Consortium,
2011.</dd>

<dt class="label"><a name="MMIF" id="MMIF"></a>MMIF</dt>

<dd><a href="http://www.w3.org/TR/mmi-framework/"><cite>"W3C
Multimodal Interaction Framework"</cite></a> , James A. Larson,
T.V. Raman and Dave Raggett, editors, World Wide Web Consortium,
2003.</dd>

<dt class="label"><a name="EMMA" id="EMMA"></a>EMMA</dt>

<dd><a href="http://www.w3.org/TR/emma/"><cite>"Extensible
multimodal Annotation markup language (EMMA)"</cite></a>, Michael
Johnson et al. editors. EMMA is an XML format for annotating
application specific interpretations of user input with information
such as confidence scores, time stamps, input modality and
alternative recognition hypotheses, World Wide Web Consortium,
2009.</dd>

<dt class="label"><a name="VoiceXML"
id="VoiceXML"></a>VoiceXML</dt>

<dd><a href="http://www.w3.org/TR/voicexml20/"><cite>"Voice
Extensible Markup Language (VoiceXML) Version 2.1"</cite></a> ,
Matt Oshry et al. editors. World Wide Web Consortium, 2007.</dd>

<dt class="label"><a name="SSML" id="SSML"></a>SSML</dt>

<dd><a
href="http://www.w3.org/TR/speech-synthesis11/"><cite>"Speech
Synthesis Markup Language (SSML) Version 1.1"</cite></a> , Daniel
C. Burnett et al. editors. World Wide Web Consortium, 2010.</dd>

<dt class="label"><a name="SRGS" id="SRGS"></a>SRGS</dt>

<dd><a href="http://www.w3.org/TR/speech-grammar/"><cite>"Speech
Recognition Grammar Specification Version 1.0"</cite></a> , Andrew
Hunt et al. editors. World Wide Web Consortium, 2004.</dd>

<dt class="label"><a name="SISR" id="SISR"></a>SISR</dt>

<dd><a
href="http://www.w3.org/TR/semantic-interpretation/"><cite>"Semantic
Interpretation for Speech Recognition (SISR) Version
1.0"</cite></a> , Luc Van Tichelen al. editors. World Wide Web
Consortium, 2004.</dd>
</dl>
</div>
</body>
</html>