index.html 64 KB

Raw Blame History Permalink

<?xml version="1.0" encoding="UTF-8"?>
<!DOCTYPE html PUBLIC "-//W3C//DTD XHTML 1.0 Strict//EN"
    "http://www.w3.org/TR/xhtml1/DTD/xhtml1-strict.dtd">

<html lang="EN" xmlns="http://www.w3.org/1999/xhtml" xml:lang="EN">
<head>
  <meta name="generator" content=
  "HTML Tidy for Linux/x86 (vers 1 September 2005), see www.w3.org" />
  <meta http-equiv="Content-Type" content=
  "text/html; charset=utf-8" />

  <title>Authoring Applications for the Multimodal
  Architecture</title>
<style type="text/css">
/*<![CDATA[*/
code           { font-family: monospace; }

div.constraint,
div.issue,
div.note,
div.notice     { margin-left: 2em; }

ol.enumar      { list-style-type: decimal; }
ol.enumla      { list-style-type: lower-alpha; }
ol.enumlr      { list-style-type: lower-roman; }
ol.enumua      { list-style-type: upper-alpha; }
ol.enumur      { list-style-type: upper-roman; }


div.exampleInner pre { margin-left: 1em;
                       margin-top: 0em; margin-bottom: 0em}
div.exampleOuter {border: 4px double gray;
                  margin: 0em; padding: 0em}
div.exampleInner { background-color: #d5dee3;
                   border-top-width: 4px;
                   border-top-style: double;
                   border-top-color: #d3d3d3;
                   border-bottom-width: 4px;
                   border-bottom-style: double;
                   border-bottom-color: #d3d3d3;
                   padding: 4px; margin: 0em }
div.exampleWrapper { margin: 4px }
div.exampleHeader { font-weight: bold;
                    margin: 4px}
/*]]>*/
</style>
  <link type="text/css" rel="stylesheet" href=
  "http://www.w3.org/StyleSheets/TR/W3C-WG-NOTE" />
</head>

<body>
  <div class="head">
    <p><a href="http://www.w3.org/"><img width="72" height="48"
    alt="W3C" src="http://www.w3.org/Icons/w3c_home" /></a></p>

    <h1><a id="title" name="title"></a>Authoring Applications for
    the Multimodal Architecture</h1>

    <h2><a id="w3c-doctype" name="w3c-doctype"></a>W3C Working Group Note 2 July 2008</h2>

    <dl>
    <dt>This version:</dt>
    <dd>
    <a href="http://www.w3.org/TR/2008/NOTE-mmi-auth-20080702/">
    http://www.w3.org/TR/2008/NOTE-mmi-auth-20080702/
    </a>
    </dd>

    <dt>Latest version:</dt>
    <dd>
    <a href="http://www.w3.org/TR/mmi-auth/">http://www.w3.org/TR/mmi-auth/</a>
    </dd>

    <dt>Previous version:</dt>
    <dd>This is the first version.</dd>

    <dt>Editor:</dt>
    <dd>Ingmar Kliche, Deutsche Telekom AG</dd>
    </dl>

<p class="copyright"><a href="http://www.w3.org/Consortium/Legal/ipr-notice#Copyright">Copyright</a> &copy; 2008 <a href="http://www.w3.org/"><acronym title="World Wide Web Consortium">W3C</acronym></a><sup>&reg;</sup> (<a href="http://www.csail.mit.edu/"><acronym title="Massachusetts Institute of Technology">MIT</acronym></a>, <a href="http://www.ercim.org/"><acronym title="European Research Consortium for Informatics and Mathematics">ERCIM</acronym></a>, <a href="http://www.keio.ac.jp/">Keio</a>), All Rights Reserved. W3C <a href="http://www.w3.org/Consortium/Legal/ipr-notice#Legal_Disclaimer">liability</a>, <a href="http://www.w3.org/Consortium/Legal/ipr-notice#W3C_Trademarks">trademark</a> and <a href="http://www.w3.org/Consortium/Legal/copyright-documents">document use</a> rules apply.</p>

  </div>
  <hr />

  <div>
    <h2><a id="abstract" name="abstract"></a>Abstract</h2>

    <p>This document describes a multimodal system which implements
    the W3C Multimodal Architecture and gives an example of a
    simple multimodal application authored using various W3C markup
    languages, including SCXML, CCXML, VoiceXML 2.1 and HTML.</p>
  </div>

  <div>
    <h2><a id="status" name="status"></a>Status of this
    Document</h2>

<p><em>This section describes the status of this document at the time of its publication. Other documents may supersede this document. A list of current W3C publications and the latest revision of this technical report can be found in the <a href="http://www.w3.org/TR/">W3C technical reports index</a> at http://www.w3.org/TR/.</em></p>

<p>
This is the 2 July 2008
<a href="http://www.w3.org/2005/10/Process-20051014/tr.html#WGNote">
W3C Working Group Note
</a>
of "Authoring Applications for the Multimodal Architecture".

The Multimodal Interaction Working Group published an updated
Working Draft of the
"Multimodal Architecture and Interfaces" (MMI Architecture)
<a href="#MMI-ARCH">[MMI-ARCH]</a>
on 14 April 2008.

However, the Working Draft currently does not take a position
on several aspects of multimodal applications.

These include the startup phase, how components find each other, and
message transport, all of which must be addressed in actual
applications.

In order to provide a concrete illustration of a multimodal
application based on the Multimodal Architecture using current W3C
technologies, the Working Group has prepared this Note.

The goal of this Note is to make it easier to author concrete
multimodal Web applications.

This document represents the views of the group at the time of
publication.

The group is planning to enhance the MMI Architecture specification
based on this Working Group Note if the feedback is positive.
</p>

<p>
This document is one of a series produced by the
<a href="http://www.w3.org/2002/mmi/">
Multimodal Interaction Working Group</a>
of the W3C
<a href="http://www.w3.org/2002/mmi/Activity.html">
Multimodal Interaction Activity</a>.
</p>

    <p>For more information about the Multimodal Interaction
    Activity, please see the <a href=
    "http://www.w3.org/2002/mmi/Activity">Multimodal Interaction
    Activity statement</a>.</p>

    <p>Comments for this specification are welcomed and should have
    a subject starting with the prefix '[AUTH]'. Please send them
    to <a href=
    "mailto:www-multimodal@w3.org">www-multimodal@w3.org</a>, the
    public email list for issues related to Multimodal. This list
    is <a href=
    "http://lists.w3.org/Archives/Public/www-multimodal/">archived</a>
    and acceptance of this archiving policy is requested
    automatically upon first post. To subscribe to this list send
    an email to <a href= "mailto:www-multimodal-request@w3.org">www-multimodal-request@w3.org</a>
    with the word subscribe in the subject line.</p>

<p>Publication as a Working Group Note does not imply endorsement by the W3C Membership. This is a draft document and may be updated, replaced or obsoleted by other documents at any time. It is inappropriate to cite this document as other than work in progress.</p>

<p> This document was produced by a group operating under the <a href="http://www.w3.org/Consortium/Patent-Policy-20040205/">5 February 2004 W3C Patent Policy</a>. W3C maintains a <a rel="disclosure" href="http://www.w3.org/2004/01/pp-impl/34607/status">public list of any patent disclosures</a> made in connection with the deliverables of the group; that page also includes instructions for disclosing a patent. An individual who has actual knowledge of a patent which the individual believes contains <a href="http://www.w3.org/Consortium/Patent-Policy-20040205/#def-essential">Essential Claim(s)</a> must disclose the information in accordance with <a href="http://www.w3.org/Consortium/Patent-Policy-20040205/#sec-Disclosure">section 6 of the W3C Patent Policy</a>. </p>

  </div>

  <div class="toc">
    <h2><a id="contents" name="contents"></a>Table of Contents</h2>

    <p class="toc">1 <a href="#introduction">Introduction</a><br />
    2 <a href="#overview">Overview</a><br />
    3 <a href="#compontents">Implementation of the
    components</a><br />
    &nbsp;&nbsp;&nbsp;&nbsp;3.1 <a href="#RF">The Runtime
    Framework</a><br />
    &nbsp;&nbsp;&nbsp;&nbsp;3.2 <a href="#GUI-MC">GUI Modality
    Component</a><br />
    &nbsp;&nbsp;&nbsp;&nbsp;3.3 <a href="#VUI-MC">Voice Modality
    Component</a><br />
    4 <a href="#N1018A">Initiating multimodal sessions</a><br />
    5 <a href="#N10196">Authoring example</a><br />
    &nbsp;&nbsp;&nbsp;&nbsp;5.1 <a href=
    "#T-shirt.scxml">T-Shirt.scxml</a><br />
    &nbsp;&nbsp;&nbsp;&nbsp;5.2 <a href=
    "#captureColorSize.html">captureColorSize.html</a><br />
    &nbsp;&nbsp;&nbsp;&nbsp;5.3 <a href=
    "#dispatcher.ccxml">dispatcher.ccxml</a><br />
    &nbsp;&nbsp;&nbsp;&nbsp;5.4 <a href=
    "#captureColor.vxml">captureColor.vxml</a><br /></p>

    <h3><a id="appendices" name="appendices"></a>Appendices</h3>

    <p class="toc">A <a href=
    "#acknowledgments">Acknowledgments</a><br />
    B <a href="#references">References</a><br /></p>
  </div>
  <hr />

  <div class="body">
    <div class="div1">
      <h2><a id="introduction" name="introduction"></a>1
      Introduction</h2>

      <p>The W3C Multimodal Interaction (MMI) Working Group
      develops an architecture <a href="#MMI-ARCH">[MMI-ARCH]</a>
      for the Multimodal Interaction framework <a href=
      "#MMIF">[MMIF]</a>. The Multimodal Architecture describes a
      general and flexible framework for interoperability of the
      various components of the multimodal framework (e.g. modality
      components and the interaction manager) in an abstract way.
      Among others it defines interfaces and messages between the
      constituents of the framework, but it is up to the
      implementation to decide how these messages are transferred
      in case of a distributed implementation.</p>

      <p>The intention of this document is to provide a proposal of
      <em>how to implement</em> a multimodal runtime environment as
      well as an application based on the W3C Multimodal
      Architecture using <em>existing</em> W3C technologies. This
      proposal uses CCXML and VoiceXML to implement a voice
      modality component. Note that this is just one possibility
      for implementing it.</p>

      <p>Note: The W3C Voice Browser Working Group is currently
      developing VoiceXML 3.0 which is the next major release of
      VoiceXML and will enable voice browsers to fit into the W3C
      Multimodal Architecture as a modality component. As VoiceXML
      3.0 implementations are not yet available, this document
      relies on the existing VoiceXML 2.1 specification <a href=
      "#VoiceXML">[VoiceXML]</a>.</p>

      <p>The Multimodal Interaction Working Group itself wants to
      learn from this authoring example where improvements are
      possible and necessary. We also intend to present how we
      think that multimodal applications will be authored in the
      future.</p>
    </div>

    <div class="div1">
      <h2><a id="overview" name="overview"></a>2 Overview</h2>

      <p>The W3C Multimodal Architecture consists of the following
      <em>main</em> constituents (see also: <a href=
      "http://www.w3.org/TR/mmi-arch/#ArchDiagram">MMI Runtime
      Architecture Diagram</a>):</p>

      <ul>
        <li>Runtime Framework containing the Interaction
        Manager.</li>

        <li>Modality components.</li>
      </ul>

      <p>In this document we discuss a distributed implementation
      of the multimodal framework using the following components
      and technologies:</p>

      <ul>
        <li>Interaction Manager (IM) based on a state machine,
        described using <a href="#SCXML">[SCXML]</a>.</li>

        <li>GUI Modality component based on <a href=
        "#XHTML">[XHTML]</a> and ECMAScript.</li>

        <li>Voice Modality component based on <a href=
        "#VoiceXML">[VoiceXML]</a> and <a href=
        "#CCXML">[CCXML]</a>.</li>

        <li>Modality component API based on HTTP (event transport)
        and XML (event representation).</li>
      </ul>

      <p>The following figure shows all these components mapped to
      the <a href="http://www.w3.org/TR/mmi-arch/#ArchDiagram">MMI
      Runtime Architecture Diagram</a>:</p><img src=
      "images/MMI-Authoring-Environment.png" alt=
      "Runtime Architecture Diagram" />

      <p>The dashed boxes correspond to (logical) components within
      the MMI architecture whereas solid lines correspond to actual
      software or hardware components used to implement the
      system.</p>

      <p>The voice input/output device shown in the figure above
      may be a regular (mobile) phone or a Voice-over-IP (soft)
      phone. In any case a phone connection to a standard voice
      browser is used.</p>
    </div>

    <div class="div1">
      <h2><a id="compontents" name="compontents"></a>3
      Implementation of the components</h2>

      <p>This section discusses one possible implementation of the
      Multimodal Architecture.</p>

      <div class="div2">
        <h3><a id="RF" name="RF"></a>3.1 The Runtime Framework</h3>

        <p>The Runtime Framework provides the environment which
        hosts the SCXML interpreter. It has to provide an interface
        to receive events from external components (modality
        components) and must be able to inject these events into an
        existing SCXML session or to start SCXML interpreter
        sessions. The Runtime Framework also needs to provide the
        possibility to send events to external components (i.e.
        some implementation of the SCXML &lt;send&gt; tag). In the
        future this feature might be a covered by the "external
        communications module" of the SCXML specification (<a href=
        "#SCXML">[SCXML]</a>).</p>

        <p>An implementation of an SCXML interpreter written in
        Java is available open source from the Apache Software
        Foundation <a href="#commonsSCXML">[Apache Commons
        SCXML]</a>. One possibility for implementing a simple
        runtime framework could be to combine the Apache commons
        SCXML library <a href="#commonsSCXML">[Apache Commons
        SCXML]</a> with a J2EE servlet engine (e.g. <a href=
        "#apacheTomcat">[Apache Tomcat]</a>). The servlet engine
        would be used to implement the HTTP I/O
        processor.</p><img src=
        "images/MMI-InteractionManager_and_I-O-processor.png" alt=
        "SCXML based Interaction Manager" />

        <p>Even though HTTP might not be the most efficient
        solution as a transport protocol, it still has some
        advantages. It is a widely used protocol and available in
        nearly every programming language. In a distributed
        scenario, where the Interaction Manger (i.e. Runtime
        Framework) and the modality components are spread across
        the network, proxy and firewall problems are easy to solve.
        Also, our intended modality components (HTML browsers for
        graphical modality and VoiceXML browsers for voice
        modality) inherently support HTTP. Therefore we use HTTP
        for this proof-of-concept implementation proposal. Other,
        more scalable solutions might make use of other
        protocols.</p>

        <p>The Runtime Framework provides the I/O processor which
        receives HTTP requests from modality components (containing
        XML based life-cycle event representation). Based on the
        event semantics the Runtime Framework logic has either to
        start a new SCXML interpreter instance (when receiving a
        <em>mmi:newContextRequest</em> message) or to inject an
        event into a running SCXML interpreter instance.</p>

        <p>In this scenario in terms of transport the Runtime
        Framework acts as an HTTP server which receives HTTP
        requests and modality components are HTTP clients sending
        HTTP requests to the Runtime Framework. Therefore sending
        events from modality components to the Interaction Manager
        is relatively easy to implement using existing
        technologies.</p>

        <p>The multimodal runtime architecture also requires to
        send events from the Interaction Manger to the modality
        components asynchronously. To be able to leverage standard
        components like HTML and VoiceXML browsers as modality
        components (or modality component containers) events should
        still be transferred using HTTP (as HTML and VoiceXML
        browsers supporting the HTTP protocol natively). But the
        browsers act as HTTP clients only. Therefore the
        Interaction Manager has still the role of the HTTP server.
        According to the HTTP model the client has to initiate
        requests. To enable the Interaction Manager to send events
        to the modality component, the modality component therefore
        has to send HTTP requests to the Interaction Manager to ask
        for events. This technique is usually known as polling.
        Simple implementations have obvious drawbacks (e.g.
        increased network traffic, additional delay) but it is
        possible to optimize it to some extend (e.g. by blocking
        the HTTP request server side and using timeouts). This
        technique certainly has limitations for large scale
        implementations, but it is relatively easy to implement
        based on existing technologies and therefore a good choice
        for a proof-of-concept.</p>

        <p>Another promising approach could be <a href=
        "#COMET">[COMET]</a> which uses long living HTTP
        connections to stream data to the client. Again the client
        has to open the HTTP connection. The server will stream an
        HTTP response to the client and leaves the HTTP connection
        open until the next event has to be sent to the client.
        Meanwhile there are a lot of applications out there using
        this server-push technology. Unfortunately this technology
        is not well standardized yet and therefore requires browser
        dependent implementations. But it is a potential solution
        for the required server-push channel.</p>
      </div>

      <div class="div2">
        <h3><a id="GUI-MC" name="GUI-MC"></a>3.2 GUI Modality
        Component</h3>

        <p>The GUI modality component may be implemented using HTML
        and JavaScript.</p>

        <p>According to the rules defined for the Multimodal
        Architecture, the application logic resides within the
        Interaction Manager. Therefore the modality component has
        to send events (e.g. user initiated events like
        <em>click</em> or <em>change</em>) to the Interaction
        Manager. The Interaction Manager decides on possible
        reactions to this events and sends events to the modality
        component to instruct it to execute some action (e.g.
        displaying something).</p>

        <p>The modality component API may be implemented using
        <a href="#XMLHttpRequest">[XMLHttpRequest]</a> (also know
        as AJAX). Event handlers for user initiated events like
        <em>change</em> for text input elements and <em>click</em>
        events for button elements may easily convert these into
        XML representations (MMI life-cycle event representation,
        e.g. containing values of input fields) and sent them to
        the Interaction Manager using XMLHttpRequests.</p>

        <p>The following code snippet demonstrates the principle of
        how to send events to a server side Interaction Manager
        (assuming a servlet at someURL) using ECMAScript and
        XMLHttpRequests:</p>

        <div class="exampleInner">
          <pre>
/* The sendMmiLifecycleEvent() function sends the MMI lifecycle
   event, potentially containing data values like color. The implementation of
   this function is vendor specific. The function is called to send a life cycle
   event to the Runtime Framework using AJAX. The parameter "payload" contains a life
   cycle event object.
*/
function sendMmiLifecycleEvent(source, context, payload)
{
  var xmlHttpRequest = new XMLHttpRequest();

  // relative url, assuming that AJAX requests go to a url
  // relative to the documents url
  var url ="./someURL";

  var XMLpayload = payload.toXML(source, context);

  xmlHttpRequest.open("POST", url, true);
  xmlHttpRequest.onreadystatechange = readystatehandler;
  xmlHttpRequest.setRequestHeader('Content-Type', 'application/x-www-form-urlencoded');
  xmlHttpRequest.send(XMLpayload);

  function readystatehandler()
  {
    if (xmlHttpRequest.status == 200 || xmlHttpRequest.status==304) {
      // be quiet in case of success
      // alert("success");
    } else {
      // alert error
      alert("send failure");
    }
  }
}

// JavaScript Event (pseudo) object
function LifeCycleEvent(mmiEvType, eventType, fieldName, fieldValue)
{
  this.mmiEventType = mmiEvType;        // e.g. extension
  this.eventType = eventType;           // user initiated event, e.g. change
  this.fieldName = fieldName;           // e.g. HTML id of the field
  this.fieldValue = fieldValue;         // e.g. value of the field
}

// method of LifeCycleEvent object to generate XML string from its properties
LifeCycleEvent.prototype.toXML = function(source, context)
{
  var mmiLifeCycleEvent;

  mmiLifeCycleEvent  = '&amp;lt;mmi version="1.0" xmlns:mmi="http://www.w3.org/2008/04/mmi-arch"&amp;gt;';
  mmiLifeCycleEvent += '  &amp;lt;mmi:" +  this.mmiEventType + "\"";
  mmiLifeCycleEvent += '  mmi:source="' +  source + '" mmi:context="' +  context + '"&amp;gt;';
  mmiLifeCycleEvent += '  &amp;lt;mmi:data&amp;gt;';
  mmiLifeCycleEvent += '    &amp;lt;eventType&amp;gt;' + this.eventType + '&amp;lt;/eventType&amp;gt;';
  mmiLifeCycleEvent += '    &amp;lt;fieldName&amp;gt;' + this.fieldName + '&amp;lt;/fieldName&amp;gt;';
  mmiLifeCycleEvent += '    &amp;lt;fieldValue&amp;gt;' + this.fieldValue + '&amp;lt;/fieldValue&amp;gt;';
  mmiLifeCycleEvent += '  &amp;lt;/mmi:data&amp;gt;';
  mmiLifeCycleEvent += '  &amp;lt;/mmi:' +  this.mmiEventType + '&amp;gt;';
  mmiLifeCycleEvent += '&amp;lt;/mmi&amp;gt;";

  return mmiLifeCycleEvent;
}
</pre>
        </div>

        <p>As described above, receiving events from the
        Interaction Manager requires to send an HTTP request to the
        server (i.e. Runtime Framework). The response contains an
        XML coded event which represents an MMI life-cycle event.
        An event, indicating the change of the value of
        <em>color</em>, would be represented as a MMI life-cycle
        event "mmi:extension" (see <a href=
        "#MMI-ARCH">[MMI-ARCH]</a>) and could look like this:</p>

        <div class="exampleInner">
          <pre>
&lt;?xml version="1.0" encoding="UTF-8"?&gt;
&lt;mmi version="1.0" xmlns:mmi="http://www.w3.org/2008/04/mmi-arch"&gt;
  &lt;mmi:startRequest mmi:source="" mmi:target="" mmi:context=""&gt;
    &lt;mmi:contentURL href="someContentURI" max-age="" fetchtimeout="1s"&gt;
    &lt;mmi:data&gt;
    &lt;/mmi:data&gt;
  &lt;/mmi:extension&gt;
&lt;/mmi&gt;
</pre>
        </div>

        <p>It should be mentioned, that the content of the
        &lt;mmi:data&gt; element is application specific.</p>

        <p>This approach requires to send an asynchronous
        XMLHttpRequest to the Runtime Framework (expecting the
        request to be blocked at the server) and to interpret the
        response accordingly: either taking any local action based
        on the event semantics and/or re-sending another request to
        the Runtime Framework.</p>

        <div class="exampleInner">
          <pre>
/* This function handles all incoming MMI lifecycle events. They may be fetched
   from the server side interaction manager using AJAX. The returned XML document
   is the MMI lifecycle event.
*/
function handleIncomingMmiEvents(xml)
{
  // check if incoming message is MMI lifecycle event
  // perform a very simple check:
  if(xml.match("&lt;mmi:"))
  {
    // parse incoming xml string to DOM
    parser=new DOMParser();
    doc=parser.parseFromString(xml,"text/xml");

    var element = doc.documentElement;
    if(element.childNodes[0].nodeName=="mmi:newContextResponse")
    {
      _CONTEXT = element.childNodes[0].getAttribute("mmi:context");
    }
    else if(element.childNodes[0].nodeName=="mmi:extension")
    {
      if(element.childNodes[0].childNodes[0].nodeName=="mmi:data")
       {
        // Application specific extension
        // In this example we receive the name of a function and the params.
        // This has to be evaluated locally using eval().
        var functionname = element.childNodes[0].childNodes[0].childNodes[0].childNodes[0].nodeValue;
        var elementname = element.childNodes[0].childNodes[0].childNodes[1].childNodes[0].nodeValue;
        var elementvalue = element.childNodes[0].childNodes[0].childNodes[2].childNodes[0].nodeValue;
        eval(functionname + "(elementname ,elementvalue)");
      }
    }
    else if(element.childNodes[0].nodeName=="mmi:clearContextRequest")
    {
      // create new mmiLifeCycleEvent Object that signals the removal of the Context
      event = new LifeCycleEvent("clearContextResponse", "", "", "");

      // send clearContextResponse lifecycle event
      sendMmiLifecycleEvent(_SOURCE, _CONTEXT, event);
    }
    else
    {
      // unknown lifecycle event
      alert("MMI lifecycle event not handled.");
    }

    // send HTTP request to server to receive lifecycle event.
    readMmiLifecycleEvent();
  }
  else // check if contains "&lt;mmi:"
  {
    // --&gt; it is not a valid lifecycle event!
    alert("Error: wrong message!");
  }
}
</pre>
        </div>
      </div>

      <div class="div2">
        <h3><a id="VUI-MC" name="VUI-MC"></a>3.3 Voice Modality
        Component</h3>

        <p>The Voice Modality Component may be implemented using
        CCXML and VoiceXML 2.1.</p>

        <p>VoiceXML 2.1 does not provide an external eventing
        functionality. As CCXML 1.0 defines an external event
        interface (<a href=
        "http://www.w3.org/TR/ccxml/#basichttpio">Basic HTTP Event
        I/O Processor</a>), which allows to inject external events
        into a running CCXML session or to start new CCXML
        sessions, CCXML will therefore be used as a event bridge
        between VoiceXML and the Interaction Manager. CCXML will
        receive events from the Interaction Manager and - depending
        on the event semantics - start a VoiceXML dialog.</p>

        <p>VoiceXML will be used to implement the actual voice user
        interface (play prompt and control ASR). User input
        collected by VoiceXML will be returned to CCXML. CCXML has
        the ability to send HTTP requests to external components.
        This feature will be used to send events back to the
        multimodal runtime framework to inject events into the
        SCXML based Interaction Manager.</p>

        <p>Due to the fact that VoiceXML must return to CCXML (and
        hence exit) to return results (e.g. recognition results)
        the VoiceXML user interface has to be implemented as small
        independent scripts. Each script corresponds to a single
        action, like <em>play a prompt</em> or <em>start grammar
        and listen to user input</em>.</p>
      </div>
    </div>

    <div class="div1">
      <h2><a id="N1018A" name="N1018A"></a>4 Initiating multimodal
      sessions</h2>

      <p>Now, as we have described the basics of all constituents,
      we need to define the setup of a multimodal session.</p>

      <p>A multimodal session may be initiated using a GUI modality
      component. The user starts a web browser and loads a HTML
      document from a given URL. Upon load, the HTML document
      registers corresponding event handlers (e.g. for change
      events) and is able to send messages to the Interaction
      Manager using AJAX (i.e. XMLHttpRequests).</p>

      <p>The HTML document may contain a special text input field
      which is used to collect the users phone number or SIP URL.
      Once the user has entered this information it is sent (e.g.
      by pressing a corresponding button) to the Interaction
      Manager. The Interaction Manager generates a message towards
      the CCXML event processor to create a new CCXML session and
      to initiate a phone call to the given telephone number (or
      SIP URL).</p>

      <p>As soon as the telephone connection has been established
      successfully the multimodal session is initiated. Now the
      Interaction Manager is capable of controlling the two
      modalities by sending life-cycle events.</p>
    </div>

    <div class="div1">
      <h2><a id="N10196" name="N10196"></a>5 Authoring example</h2>

      <p>This section ties together the previously described
      components to implement a sample application. The multimodal
      <em>T-Shirt example</em> contains a combined graphical and
      voice user interface and allows to fill in a form containing
      two fields (color and size) either by voice or by
      pen/keyboard.</p>

      <p>The following figure shows the corresponding state machine
      logic for this example together with the MMI life-cycle
      events.</p><img src="images/T-Shirt_StateMachine.png" alt=
      "State machine logic and MMI life-cycle events" />

      <div class="div2">
        <h3><a id="T-shirt.scxml" name="T-shirt.scxml"></a>5.1
        T-Shirt.scxml</h3>

        <p>The state machine could be represented in SCXML source
        code (T-Shirt.scxml) as follows:</p>

        <div class="exampleInner">
          <pre>
&lt;?xml version="1.0" encoding="UTF-8"?&gt;
&lt;scxml version="1.0" profile="ecmascript" initial="getColor" &gt;
  &lt;!-- we assume there is a script library which constructs MMI lifecycle events etc. --&gt;
  &lt;script src="mmi.js"/&gt;

  &lt;!-- datamodel definition --&gt;
  &lt;datamodel&gt;
    &lt;data id="color" expr=""/&gt;
    &lt;data id="size" expr=""/&gt;
    &lt;data id="received" expr="0"/&gt;
  &lt;/datamodel&gt;

  &lt;!-- state getColor --&gt;
  &lt;state id="getColor"&gt;
    &lt;onentry&gt;
      &lt;script&gt;
        mmiEvent = new mmiStartRequest();
        mmiEvent.setURL('captureColorSize.html');
      &lt;/script&gt;
      &lt;!-- issue startRequest to GUI --&gt;
      &lt;send event="mmi:startRequest" target="GUI" targetType="x-ajax" namelist="mmiEvent"/&gt;
      &lt;script&gt;
        mmiEvent = new mmiStartRequest();
        mmiEvent.setURL('getColor.vxml');
      &lt;/script&gt;
      &lt;!-- issue startRequest to VUI --&gt;
      &lt;send event="mmi:startRequest" target="VUI" targetType="basichttp" namelist="mmiEvent"/&gt;
    &lt;/onentry&gt;

    &lt;!-- handle voice input --&gt;
    &lt;transition event="mmi:done" cond="_event.data..@source.toString() == 'VUI' &amp;&amp;
         _event.data..@status.toString() == 'success'" target="echoColor"/&gt;
       &lt;!-- save color to data model --&gt;
       &lt;assign location="_data.color" expr="_event.data..color.toString()"/&gt;
       &lt;!-- send event to GUI to display information --&gt;
       &lt;script&gt;
         mmiEvent = new mmiExtension();
         // construct content of data element of extension event as XML string
         dataFieldValue = "&amp;lt;eventType&amp;gt;_check&amp;lt;/eventType&amp;gt;";
         dataFieldValue += "&amp;lt;fieldName&amp;gt;color&amp;lt;/fieldName&amp;gt;";
         dataFieldValue += "&amp;lt;fieldValue&amp;gt;" + color + "&amp;lt;/fieldValue&amp;gt;";
         mmiEvent.setDataField(dataFieldValue);
       &lt;/script&gt;
      &lt;send event="mmi:extension" target="GUI" targetType="x-ajax" namelist="mmiEvent"/&gt;
    &lt;/transition&gt;

    &lt;!-- handle GUI input --&gt;
    &lt;transition event="mmi:extension" cond="_event.data..@source.toString() == 'GUI' &amp;&amp;
         _event.data..@status.toString() == 'success'" target="echoColor"/&gt;
       &lt;!-- save color to data model --&gt;
       &lt;assign location="_data.color" expr="_event.data..color.toString()"/&gt;
    &lt;/transition&gt;

    &lt;!-- error handling --&gt;
    &lt;transition event="mmi:startResponse" cond="_event.data..@status.toString() == 'error'" target="failure"/&gt;
    &lt;transition event="mmi:done" cond="_event.data..@status.toString() == 'error'" target="failure"/&gt;
  &lt;/state&gt;

  &lt;!-- state echoColor --&gt;
  &lt;state id="echoColor"&gt;
    &lt;onentry&gt;
      &lt;!-- play back color to user --&gt;
      &lt;script&gt;
        mmiEvent = new mmiStartRequest();
        mmiEvent.setURL('echoColor.vxml');
        // construct content of data element of extension event as XML string
        dataFieldValue = "&amp;lt;color&amp;gt;" + color + "&amp;lt;/color&amp;gt;";
        mmiEvent.setDataField(dataFieldValue);
      &lt;/script&gt;
      &lt;send event="mmi:startRequest" target="VUI" targetType="basichttp" namelist="mmiEvent"/&gt;
    &lt;/onentry&gt;

    &lt;!-- play prompt done --&gt;
    &lt;transition event="mmi:done" cond="_event.data..@source.toString() == 'VUI' &amp;&amp;
         _event.data..@status.toString() == 'success'" target="getSize"/&gt;

    &lt;!-- error handling --&gt;
    &lt;transition event="mmi:startResponse" cond="_event.data..@status.toString() == 'error'" target="failure"/&gt;
    &lt;transition event="mmi:done" cond="_event.data..@status.toString() == 'error'" target="failure"/&gt;
  &lt;/state&gt;

  &lt;!-- state getSize --&gt;
  &lt;state id="getSize"&gt;
    &lt;onentry&gt;
      &lt;script&gt;
        mmiEvent = new mmiStartRequest();
        mmiEvent.setURL('getSize.vxml');
      &lt;/script&gt;
      &lt;!-- issue startRequest to VUI --&gt;
      &lt;send event="mmi:startRequest" target="VUI" targetType="basichttp" namelist="mmiEvent"/&gt;
    &lt;/onentry&gt;

    &lt;!-- handle voice input --&gt;
    &lt;transition event="mmi:done" cond="_event.data..@source.toString() == 'VUI' &amp;&amp;
         _event.data..@status.toString() == 'success'" target="echoSize"/&gt;
       &lt;!-- save color to data model --&gt;
       &lt;assign location="_data.size" expr="_event.data..size.toString()"/&gt;
       &lt;!-- send event to GUI to display information --&gt;
       &lt;script&gt;
         mmiEvent = new mmiExtension();
         // construct content of data element of extension event as XML string
         dataFieldValue = "&amp;lt;eventType&amp;gt;_check&amp;lt;/eventType&amp;gt;";
         dataFieldValue += "&amp;lt;fieldName&amp;gt;size&amp;lt;/fieldName&amp;gt;";
         dataFieldValue += "&amp;lt;fieldValue&amp;gt;" + size + "&amp;lt;/fieldValue&amp;gt;";
         mmiEvent.setDataField(dataFieldValue);
       &lt;/script&gt;
      &lt;send event="mmi:extension" target="GUI" targetType="x-ajax" namelist="mmiEvent"/&gt;
    &lt;/transition&gt;

    &lt;!-- handle GUI input --&gt;
    &lt;transition event="mmi:extension" cond="_event.data..@source.toString() == 'GUI' &amp;&amp;
         _event.data..@status.toString() == 'success'" target="echoSize"/&gt;
       &lt;!-- save size to data model --&gt;
       &lt;assign location="_data.size" expr="_event.data..size.toString()"/&gt;
    &lt;/transition&gt;

    &lt;!-- error handling --&gt;
    &lt;transition event="mmi:startResponse" cond="_event.data..@status.toString() == 'error'" target="failure"/&gt;
    &lt;transition event="mmi:done" cond="_event.data..@status.toString() == 'error'" target="failure"/&gt;
  &lt;/state&gt;

  &lt;!-- state echoSize --&gt;
  &lt;state id="echoSize"&gt;
    &lt;onentry&gt;
      &lt;!-- play back color to user --&gt;
      &lt;script&gt;
        mmiEvent = new mmiStartRequest();
        mmiEvent.setURL('echoSize.vxml');
        // construct content of data element of extension event as XML string
        dataFieldValue = "&amp;lt;size&amp;gt;" + size + "&amp;lt;/size&amp;gt;";
        mmiEvent.setDataField(dataFieldValue);
      &lt;/script&gt;
      &lt;send event="mmi:startRequest" target="VUI" targetType="basichttp" namelist="mmiEvent"/&gt;
    &lt;/onentry&gt;

    &lt;!-- play prompt done --&gt;
    &lt;transition event="mmi:done" cond="_event.data..@source.toString() == 'VUI' &amp;&amp;
         _event.data..@status.toString() == 'success'" target="endOfInteraction"/&gt;

    &lt;!-- error handling --&gt;
    &lt;transition event="mmi:startResponse" cond="_event.data..@status.toString() == 'error'" target="failure"/&gt;
    &lt;transition event="mmi:done" cond="_event.data..@status.toString() == 'error'" target="failure"/&gt;
  &lt;/state&gt;

  &lt;!-- state  endOfInteraction--&gt;
  &lt;state id="endOfInteraction"&gt;
    &lt;onentry&gt;
      &lt;!-- number of received clearContextResponse messages, we are waiting for two --&gt;
      &lt;assign location="received" expr="0"/&gt;

      &lt;!-- issue clearContextRequest messages --&gt;
      &lt;script&gt;
        mmiEvent = new mmiClearContextRequest();
      &lt;/script&gt;
      &lt;!-- issue clearContextRequest to GUI --&gt;
      &lt;send event="mmi:clearContextRequest" target="GUI" targetType="x-ajax" namelist="mmiEvent"/&gt;
      &lt;script&gt;
        mmiEvent = new mmiClearContextRequest();
      &lt;/script&gt;
      &lt;!-- issue clearContextRequest to VUI --&gt;
      &lt;send event="mmi:clearContextRequest" target="VUI" targetType="basichttp" namelist="mmiEvent"/&gt;
    &lt;/onentry&gt;
    &lt;transition event="mmi:clearContextResponse" cond="received = 0"&gt;
      &lt;!-- increase counter --&gt;
      &lt;assign location="received" expr="1"/&gt;
    &lt;/transition&gt;
    &lt;transition event="mmi:clearContextResponse" cond="received &gt; 0" target="end"/&gt;
  &lt;/state&gt;

  &lt;!-- state failure --&gt;
  &lt;state id ="failure"&gt;
    &lt;!-- simply stop interaction --&gt;
    &lt;transition target="endOfInteraction"/&gt;
  &lt;/state&gt;

  &lt;!-- final state --&gt;
  &lt;state id="end" final="true"/&gt;
&lt;/scxml&gt;
</pre>
        </div>
      </div>

      <p>In this example we assume that the Runtime Framework
      supports the <em>x-ajax</em> and <em>basichttp</em>
      targettypes for the &lt;send&gt; tag. The GUI modality
      component uses AJAX to communicate to the Runtime Framework.
      Therefore we use <em>x-ajax</em> as the targettype, whereas
      the Voice modality component is implemented using
      CCXML/VoiceXML. As the external event interface of CCXML is
      used to inject events into the CCXML session we have to make
      use of the <em>basichttp</em> targettype.</p>

      <div class="div2">
        <h3><a id="captureColorSize.html" name=
        "captureColorSize.html"></a>5.2 captureColorSize.html</h3>

        <p>The following code fragment provides the basics of the
        HTML source code for the GUI modality component (i.e.
        captureColorSize.html):</p>

        <div class="exampleInner">
          <pre>
&lt;!DOCTYPE html PUBLIC "-//W3C//DTD XHTML 1.0 Transitional//EN"
    "http://www.w3.org/TR/xhtml1/DTD/xhtml1-transitional.dtd"&gt;
&lt;html xmlns="http://www.w3.org/1999/xhtml"&gt;

&lt;head&gt;

&lt;meta name="application" content="captureColorSize" /&gt;
&lt;meta name="description" content="solicits value for size and color" /&gt;

&lt;title&gt;CaptureColorSize&lt;/title&gt;

&lt;script src="sendEvent.js" language="javascript"/&gt;
&lt;script src="evaluateResponseXML.js" language="javascript"/&gt;

&lt;!-- Event handlers that create and send external events --&gt;

&lt;script language="JavaScript" type="text/javascript"&gt;

  var _SOURCE="GUI";
  var _CONTEXT="";

  function onloadHandler()
  {
    // create new mmiLifeCycleEvent Object that requests a new Context
    event = new LifeCycleEvent("newContextRequest", "", "", "");

    // send newContextRequest lifecycle event
    sendMmiLifecycleEvent(_SOURCE, _CONTEXT, event);

    // send HTTP request to server to receive lifecycle event.
    readMmiLifecycleEvent();
  }

  /* HTML event handler. The functions makes use of the browser event object
    which holds the id and the value (in this case the color value) of the HTML object.
  */
  function eventHandler(event)
  {
    target = new Object();
    if(event.target)
    {
      target = event.target;
    }
    // Internet Explorer has no attribute target
    else if(event.srcElement)
    {
      target = event.srcElement;
    }

    event = new LifeCycleEvent("extension", event.type, target.id, target.value);

    sendMmiLifecycleEvent(_SOURCE, _CONTEXT, event);
  }

  /* initiate AJAX request to the interaction manager to read the next MMI lifecycle
     event. The returned event is handled asynchronously within handleIncomingMmiEvents().
     Finally the next MMI lifecycle event is fetched.
  */
  function readMmiLifecycleEvent()
  {
    // Start asynchronous XMLHttpRequest to receive MMI lifecycle event.
    // We assume that the the IM always returns a lifecycle event, i.e.
    // we do not handle special timeout events. Once the request returns,
    // handleIncomingMmiEvents() will be called to evaluate the xml encoded event.

    var xmlHttpRequest = new XMLHttpRequest();

    // relative url, assuming that AJAX requests go to a url
    // relative to the documents url
    var url ="./getMMILifeCycleEvent";

    xmlHttpRequest.open("GET", url, true);
    xmlHttpRequest.onreadystatechange = readyhandler;
    xmlHttpRequest.send(null);

    function readyhandler()
    {
      if (xmlHttpRequest.readyState == 4) {
        if (xmlHttpRequest.status == 200) {
          //handle lifecycle event
          handleIncomingMmiEvents(xmlHttpRequest.responseText);
        } else {
          // alert error
          alert("readMmiLifecycleEvent failure");
        }
      }
    }
  }

  // function to check elements with the given html id, e.g. radio buttons
  function _check(elementname, elementvalue)
  {
     document.getElementById(elementvalue).checked=true;
  }
&lt;/script&gt;
&lt;/head&gt;

&lt;body id="bodyId" onload="onloadHandler();"&gt;

&lt;form action="" name="Color" id="Color"&gt;T-shirt color:
&lt;table width="200"&gt;
  &lt;tr&gt;
    &lt;td&gt;
      &lt;label&gt; &lt;input type="radio" id="red"
      name="radioGroup1" value="Red" onclick="eventHandler(event);" /&gt;
      Red&lt;/label&gt;
    &lt;/td&gt;
  &lt;/tr&gt;
  &lt;tr&gt;
    &lt;td&gt;
      &lt;label&gt; &lt;input type="radio" id="green"
      name="radioGroup1" value="Green" onclick="eventHandler(event);" /&gt;
      Green&lt;/label&gt;
    &lt;/td&gt;
  &lt;/tr&gt;
  &lt;tr&gt;
    &lt;td&gt;
      &lt;label&gt; &lt;input type="radio" id="blue"
      name="radioGroup1" value="Blue" onclick="eventHandler(event);" /&gt;
      Blue&lt;/label&gt;
    &lt;/td&gt;
  &lt;/tr&gt;
&lt;/table&gt;
&lt;/form&gt;

&lt;form action="" name="Size" id="Size"&gt;T-shirt size:
&lt;table width="200"&gt;
  &lt;tr&gt;
    &lt;td&gt;&lt;label&gt; &lt;input type="radio" id="small"
      name="radioGroup2" value="Small" onclick="eventHandler(event);" /&gt;
    Small&lt;/label&gt;&lt;/td&gt;
  &lt;/tr&gt;
  &lt;tr&gt;
    &lt;td&gt;&lt;label&gt; &lt;input type="radio" id="medium"
      name="radioGroup2" value="Medium" onclick="eventHandler(event);" /&gt;
    Medium&lt;/label&gt;&lt;/td&gt;
  &lt;/tr&gt;
  &lt;tr&gt;
    &lt;td&gt;&lt;label&gt; &lt;input type="radio" id="large"
      name="radioGroup2" value="Large" onclick="eventHandler(event);" /&gt;
    Large&lt;/label&gt;&lt;/td&gt;
  &lt;/tr&gt;
&lt;/table&gt;
&lt;/form&gt;

&lt;/body&gt;
&lt;/html&gt;
</pre>
        </div>

        <p>sendEvent.js:</p>

        <div class="exampleInner">
          <pre>
/* The sendMmiLifecycleEvent() function sends the MMI lifecycle
   event, potentially containing data values like color. The implementation of
   this function is vendor specific. The function is called to send a life cycle
   event to the Runtime Framework using AJAX. The parameter "payload" contains a life
   cycle event object.
*/
function sendMmiLifecycleEvent(source, context, payload)
{
  var xmlHttpRequest = new XMLHttpRequest();

  // relative url, assuming that AJAX requests go to a url
  // relative to the documents url
  var url ="./someURL";

  var XMLpayload = payload.toXML(source, context);

  xmlHttpRequest.open("POST", url, true);
  xmlHttpRequest.onreadystatechange = readystatehandler;
  xmlHttpRequest.setRequestHeader('Content-Type', 'application/x-www-form-urlencoded');
  xmlHttpRequest.send(XMLpayload);

  function readystatehandler()
  {
    if (xmlHttpRequest.status == 200 || xmlHttpRequest.status==304) {
      // be quiet in case of success
      // alert("success");
    } else {
      // alert error
      alert("send failure");
    }
  }
}

// JavaScript Event (pseudo) object
function LifeCycleEvent(mmiEvType, eventType, fieldName, fieldValue)
{
  this.mmiEventType = mmiEvType;        // e.g. extension
  this.eventType = eventType;           // user initiated event, e.g. change
  this.fieldName = fieldName;           // e.g. HTML id of the field
  this.fieldValue = fieldValue;         // e.g. value of the field
}

// method of LifeCycleEvent object to generate XML string from its properties
LifeCycleEvent.prototype.toXML = function(source, context)
{
  var mmiLifeCycleEvent;

  mmiLifeCycleEvent  = '&amp;lt;mmi version="1.0" xmlns:mmi="http://www.w3.org/2008/04/mmi-arch"&amp;gt;';
  mmiLifeCycleEvent += '  &amp;lt;mmi:' +  this.mmiEventType + '"';
  mmiLifeCycleEvent += '  mmi:source="' +  source + '" mmi:context="' +  context + '"&amp;gt;';
  mmiLifeCycleEvent += '  &amp;lt;mmi:data&amp;gt;";
  mmiLifeCycleEvent += '    &amp;lt;eventType&amp;gt;' + this.eventType + '&amp;lt;/eventType&amp;gt;';
  mmiLifeCycleEvent += '    &amp;lt;fieldName&amp;gt;' + this.fieldName + '&amp;lt;/fieldName&amp;gt;';
  mmiLifeCycleEvent += '    &amp;lt;fieldValue&amp;gt;' + this.fieldValue + '&amp;lt;/fieldValue&amp;gt;';
  mmiLifeCycleEvent += '  &amp;lt;/mmi:data&amp;gt;';
  mmiLifeCycleEvent += '  &amp;lt;/mmi:' +  this.mmiEventType + '&amp;gt;';
  mmiLifeCycleEvent += '&amp;lt;/mmi&amp;gt;';

  return mmiLifeCycleEvent;
}
</pre>
        </div>

        <p>evaluateResponseXML.js:</p>

        <div class="exampleInner">
          <pre>
/* This function handles all incoming MMI lifecycle events. They may be fetched
   from the server side interaction manager using AJAX. The returned XML document
   is the MMI lifecycle event.
*/
function handleIncomingMmiEvents(xml)
{
  // check if incoming message is MMI lifecycle event
  // perform a very simple check:
  if(xml.match("&lt;mmi:"))
  {
    // parse incoming xml string to DOM
    parser=new DOMParser();
    doc=parser.parseFromString(xml,"text/xml");

    var element = doc.documentElement;
    if(element.childNodes[0].nodeName=="mmi:newContextResponse")
    {
      _CONTEXT = element.childNodes[0].getAttribute("mmi:context");
    }
    else if(element.childNodes[0].nodeName=="mmi:extension")
    {
      if(element.childNodes[0].childNodes[0].nodeName=="mmi:data")
       {
        // Application specific extension
        // In this example we receive the name of a function and the params.
        // This has to be evaluated locally using eval().
        var functionname = element.childNodes[0].childNodes[0].childNodes[0].childNodes[0].nodeValue;
        var elementname = element.childNodes[0].childNodes[0].childNodes[1].childNodes[0].nodeValue;
        var elementvalue = element.childNodes[0].childNodes[0].childNodes[2].childNodes[0].nodeValue;
        eval(functionname + "(elementname ,elementvalue)");
      }
    }
    else if(element.childNodes[0].nodeName=="mmi:clearContextRequest")
    {
      // create new mmiLifeCycleEvent Object that signals the removal of the Context
      event = new LifeCycleEvent("clearContextResponse", "", "", "");

      // send clearContextResponse lifecycle event
      sendMmiLifecycleEvent(_SOURCE, _CONTEXT, event);
    }
    else
    {
      // unknown lifecycle event
      alert("MMI lifecycle event not handled.");
    }

    // send HTTP request to server to receive lifecycle event.
    readMmiLifecycleEvent();
  }
  else // check if contains "&lt;mmi:"
  {
    // --&gt; it is not a valid lifecycle event!
    alert("Error: wrong message!");
  }
}
</pre>
        </div>

        <p>The ECMAScript function <em>_check(elementname,
        elementvalue)</em> within captureColorSize.html is provided
        to check a radio button. To achieve this, the Interaction
        Manager sends a mmi:extension life-cycle event where the
        (application specific) <em>eventType</em> element within
        the &lt;mmi:data&gt; element is set to <em>_check</em>. The
        <em>fieldValue</em> element contains the HTML id of the
        corresponding object. The <em>_check(...)</em> function
        therefore simply uses the DOM API to activate the radio
        button. The following example shows the MMI life-cycle
        event to activate the green color radio button.</p>

        <div class="exampleInner">
          <pre>
&lt;?xml version="1.0" encoding="UTF-8"?&gt;
&lt;mmi version="1.0" xmlns:mmi="http://www.w3.org/2008/04/mmi-arch"&gt;
  &lt;mmi:extension mmi:source="captureColorSize.html" mmi:context=""&gt;
    &lt;mmi:data&gt;
      &lt;eventType&gt;_check&lt;/eventType&gt;
      &lt;fieldName&gt;color&lt;/fieldName&gt;
      &lt;fieldValue&gt;green&lt;/fieldValue&gt;
    &lt;/mmi:data&gt;
  &lt;/mmi:extension&gt;
&lt;/mmi&gt;
</pre>
        </div>

        <p>This event is created within the SCXML script. See the
        <em>getColor</em> state of the SCXML sample code (<a href=
        "#T-shirt.scxml"><b>5.1 T-Shirt.scxml</b></a>).</p>
      </div>

      <div class="div2">
        <h3><a id="dispatcher.ccxml" name=
        "dispatcher.ccxml"></a>5.3 dispatcher.ccxml</h3>

        <p>CCXML is used as a dispatcher of events between SCXML
        and VoiceXML. The script <em>ccxml_events.js</em> contains
        a collection of support functions to create MMI life-cycle
        events or to start VoiceXML dialogs.</p>

        <p>Note that this script is written to be application
        independent.</p>

        <div class="exampleInner">
          <pre>
&lt;?xml version="1.0" encoding="UTF-8" ?&gt;
&lt;ccxml version="1.0" xmlns="http://www.w3.org/2002/09/ccxml"&gt;

  &lt;!-- we assume there is a library of functions to send data to
   the Interaction Manager --&gt;
  &lt;script src="ccxml_events.js" /&gt;

  &lt;!-- CCXML session ID --&gt;
  &lt;var name="connectionId" expr="''" /&gt;

  &lt;!-- SCXML session ID --&gt;
  &lt;var name="interactionId" expr="''" /&gt;
  &lt;!-- request ID of lifecycle event --&gt;
  &lt;var name="requestID" expr="'123456'" /&gt;
  &lt;!-- target type --&gt;
  &lt;var name="SCXML" expr="'basichttp'" /&gt;

  &lt;!-- VXML dialog ID for termination --&gt;
  &lt;var name="vxml_dialogid" expr="0" /&gt;
  &lt;!-- whether a VXML dialog is running or not --&gt;
  &lt;var name="vxml_running" expr="false" /&gt;
  &lt;!-- whether a VXML dialog is in terminating or not --&gt;
  &lt;var name="vxml_terminating" expr="false" /&gt;

  &lt;var name="prompt" expr="''" /&gt;
  &lt;var name="audio" expr="''" /&gt;
  &lt;var name="grammarUri" expr="''" /&gt;
  &lt;var name="fields" expr="''" /&gt;


  &lt;!-- Note: All events which are tagged with "(INTERNAL)" are standard events! --&gt;
  &lt;eventprocessor&gt;

    &lt;!-- ===================================================== --&gt;
    &lt;!-- SCXML events --&gt;
    &lt;!-- ===================================================== --&gt;

    &lt;!-- CCXML (INTERNAL): when CCXML is started, it throws this internal event --&gt;
    &lt;transition event="ccxml.loaded"&gt;
      &lt;script&gt;
        _ccxml.setSCXML_URI(session.values.scxml_serverip,
            session.values.scxml_serverport,
            session.values.scxml_serverpage);
      &lt;/script&gt;
      &lt;assign name="connectionId" expr="event$.connectionid" /&gt;
      &lt;assign name="interactionId" expr="session.values.interactionid" /&gt;

      &lt;!-- call SIP phone --&gt;
      &lt;var name="sipip" expr="session.values.sip_phoneprefix + '@' + session.values.sip_phoneip +
           ':' + session.values.sip_phoneport" /&gt;
      &lt;createcall dest="sipip" connectionid="connectionId" /&gt;
    &lt;/transition&gt;

    &lt;!-- CCXML: terminate --&gt;
    &lt;transition event="ccxml.terminate"&gt;
      &lt;send target="_ccxml.clearContextResponse(interactionId, requestID)"
          targettype="SCXML" name="'ccxml.external'" /&gt;
      &lt;send target="session.id" targettype="'ccxml'" name="'this.exit'" /&gt;
    &lt;/transition&gt;
    &lt;transition event="this.exit"&gt;
      &lt;log expr="'CCXML.exit'" /&gt;
      &lt;exit /&gt;
    &lt;/transition&gt;

    &lt;!-- SIP: disconnect SIP phone --&gt;
    &lt;transition event="sip.disconnect"&gt;
      &lt;disconnect connectionid="connectionId" /&gt;
      &lt;/transition&gt;

        &lt;!-- VXML: start --&gt;
    &lt;transition event="vxml.start"&gt;
      &lt;assign name="prompt" expr="event$.prompt" /&gt;
      &lt;assign name="audio" expr="event$.audio" /&gt;
      &lt;assign name="grammarUri" expr="event$.grammarUri" /&gt;
      &lt;assign name="fields" expr="event$.fields" /&gt;

      &lt;!-- If a VXML dialog is running, terminate. otherwise start --&gt;
      &lt;if cond="vxml_running == false"&gt;
        &lt;assign name="vxml_running" expr="true" /&gt;
        &lt;var name="sessionid" expr="event$.sessionid" /&gt;
        &lt;dialogstart src="_vxml.start(grammarUri, prompt, audio, fields)"
            dialogid="vxml_dialogid" connectionid="connectionId" namelist="sessionid" /&gt;
      &lt;else /&gt;
        &lt;assign name="vxml_terminating" expr="true" /&gt;
        &lt;dialogterminate dialogid="vxml_dialogid" immediate="true" /&gt;
      &lt;/if&gt;
    &lt;/transition&gt;

    &lt;!-- VXML: terminate --&gt;
    &lt;transition event="vxml.terminate"&gt;
      &lt;var name="immediate" expr="event$.immediate" /&gt;
      &lt;dialogterminate dialogid="vxml_dialogid" immediate="immediate" /&gt;
    &lt;/transition&gt;


    &lt;!-- ===================================================== --&gt;
    &lt;!-- SIP events --&gt;
    &lt;!-- ===================================================== --&gt;

    &lt;!-- SIP (INTERNAL): connection to phone completed --&gt;
    &lt;transition event="connection.connected"&gt;
      &lt;send target="_ccxml.createResponse(interactionId, requestID, session.id)"
          targettype="SCXML" name="'ccxml.external'" /&gt;
    &lt;/transition&gt;

    &lt;!-- SIP (INTERNAL): disconnected --&gt;
    &lt;transition event="connection.disconnected"&gt;
      &lt;send target="_ccxml.clearContextRequest(interactionId)"
          targettype="SCXML" name="'ccxml.external'" /&gt;
      &lt;send target="session.id" targettype="'ccxml'" name="'this.exit'" /&gt;
    &lt;/transition&gt;

    &lt;!-- SIP (INTERNAL): reject call from SIP phone --&gt;
    &lt;transition event="connection.alerting"&gt;
      &lt;reject /&gt;
    &lt;/transition&gt;


    &lt;!-- ===================================================== --&gt;
    &lt;!-- VXML events --&gt;
    &lt;!-- ===================================================== --&gt;

    &lt;!-- VXML (INTERNAL): when VXML is started, it throws this internal event --&gt;
    &lt;transition event="dialog.started"&gt;
      &lt;send target="_vxml.startResponse(interactionId, requestID, vxml_dialogid)"
          targettype="SCXML" name="'ccxml.external'" /&gt;
    &lt;/transition&gt;

    &lt;!-- VXML (INTERNAL): an exit in VXML throws this internal event
      if it was not just a prompt, get EMMA from VXML and send response to SCXML --&gt;
    &lt;transition event="dialog.exit"&gt;
      &lt;assign name="vxml_running" expr="false" /&gt;
      &lt;!-- if a VXML dialog was terminated as a cause of dialogstart until a dialog was running,
        start new dialog now (for that case we have the global _vxml.start()-parameters) --&gt;
      &lt;if cond="vxml_terminating == false"&gt;
        &lt;send target="_vxml.doneNotification(interactionId, event$.values.emma, vxml_dialogid)"
            targettype="SCXML" name="'ccxml.external'" /&gt;
      &lt;else /&gt;
        &lt;send target="_vxml.doneNotification(interactionId, '', vxml_dialogid)"
            targettype="SCXML" name="'ccxml.external'" /&gt;
        &lt;assign name="vxml_running" expr="true" /&gt;
        &lt;var name="sessionid" expr="event$.sessionid" /&gt;
        &lt;dialogstart src="_vxml.start(grammarUri, prompt, audio, fields)"
            dialogid="vxml_dialogid" connectionid="connectionId" namelist="sessionid" /&gt;
        &lt;assign name="vxml_terminating" expr="false" /&gt;
      &lt;/if&gt;
    &lt;/transition&gt;


    &lt;!-- ===================================================== --&gt;
    &lt;!-- error events --&gt;
    &lt;!-- ===================================================== --&gt;

    &lt;!-- all errors (INTERNAL) --&gt;
    &lt;transition event="error.*"&gt;
      &lt;log expr="'CCXML error'" /&gt;
      &lt;!--
      &lt;send target="_vxml.sendEvent(interactionId, _vxml.ERROR)"
          targettype="SCXML" name="'ccxml.external'" /&gt;
      --&gt;
    &lt;/transition&gt;

  &lt;/eventprocessor&gt;

&lt;/ccxml&gt;
</pre>
        </div>
      </div>

      <div class="div2">
        <h3><a id="captureColor.vxml" name=
        "captureColor.vxml"></a>5.4 captureColor.vxml</h3>

        <p>As mentioned in <a href="#VUI-MC"><b>3.3 Voice Modality
        Component</b></a> VoiceXML must return to CCXML (and hence
        exit the VoiceXML dialog) to return results (e.g.
        recognition results). Therefore the VoiceXML user
        interfaces has to be implemented as small independent
        scripts. Each script corresponds to a single action, like
        <em>play a prompt</em> or <em>start grammar and listen to
        user input</em>.</p>

        <p>The following code sample shows how the
        captureColor.vxml document could look like. The script
        <em>vxml_emma.js</em>, which is referenced in the VoiceXML
        document, contains a collection of auxiliary ECMAScript
        functions to create an <a href="#EMMA">[EMMA]</a>
        representation of the user input. See Appendix C of
        <a href="#EMMA">[EMMA]</a> for more information of how to
        map a recognition result into an EMMA representation.</p>

        <p>Note that the other VoiceXML documents are very similar
        and therefore not shown here.</p>

        <div class="exampleInner">
          <pre>
&lt;?xml version="1.0" encoding="UTF-8"?&gt;
&lt;vxml xmlns:vxml="http://www.w3.org/2001/vxml" version="2.1"&gt;
  &lt;!-- We assume that there is a script library to convert the
       recognition result into an EMMA string
       (see http://www.w3.org/TR/emma) --&gt;
  &lt;script src="vxml_emma.js" /&gt;
  &lt;form&gt;
    &lt;field name="color"&gt;
       &lt;prompt&gt;Which color?&lt;/prompt&gt;
       &lt;option&gt;red&lt;/option&gt;
       &lt;option&gt;blue&lt;/option&gt;
       &lt;option&gt;green&lt;/option&gt;
       &lt;filled&gt;
         &lt;!-- generate EMMA string from recognition result --&gt;
         &lt;var name="emma" expr="createEmma(application.lastresult$)"/&gt;
         &lt;!-- exit back to CCXML and return the recognized result --&gt;
         &lt;exit namelist="emma"/&gt;
       &lt;/filled&gt;
    &lt;/field&gt;
    &lt;catch event="help nomatch noinput"&gt;
       Your options are &lt;enumerate/&gt;
    &lt;/catch&gt;
  &lt;/form&gt;
&lt;/vxml&gt;
</pre>
        </div>
      </div>
    </div>
  </div>

  <div class="back">
    <div class="div1">
      <h2><a id="acknowledgments" name="acknowledgments"></a>A
      Acknowledgments</h2>

      <p>The editor wishes to thank Jim Larson (Intervoice) from
      the Voice Browser Working Group for his contributions to the
      writing of this document and for many helpful comments.</p>
    </div>

    <div class="div1">
      <h2><a id="references" name="references"></a>B
      References</h2>

      <dl>
        <dt class="label"><a id="apacheTomcat" name=
        "apacheTomcat"></a>Apache Tomcat</dt>

        <dd><a href="http://tomcat.apache.org"><cite>Apache Tomcat
        servlet container</cite></a>, The Apache Software
        Foundation, 2008.</dd>

        <dt class="label"><a id="commonsSCXML" name=
        "commonsSCXML"></a>Apache Commons SCXML</dt>

        <dd><a href="http://commons.apache.org/scxml"><cite>A Java
        SCXML engine</cite></a>, The Apache Software Foundation,
        2008.</dd>

        <dt class="label"><a id="CCXML" name="CCXML"></a>CCXML</dt>

        <dd><a href="http://www.w3.org/TR/ccxml/"><cite>"Voice
        Browser Call Control: CCXML Version 1.0 (Working
        Draft)"</cite></a> , R.J. Auburn, editor. CCXML, or the
        Call Control eXtensible Markup Language, is designed to
        provide telephony call control support for dialog systems,
        such as VoiceXML. World Wide Web Consortium, 2007.</dd>

        <dt class="label"><a id="COMET" name="COMET"></a>COMET</dt>

        <dd><a href=
        "http://en.wikipedia.org/wiki/Comet_(programming)"><cite>Comet
        Ajax server-push</cite></a>, Wikipedia.</dd>

        <dt class="label"><a id="EMMA" name="EMMA"></a>EMMA</dt>

        <dd><a href="http://www.w3.org/TR/emma/"><cite>"Extensible
        multimodal Annotation markup language (EMMA)"</cite></a>,
        Michael Johnson et al. editors. EMMA is an XML format for
        annotating application specific interpretations of user
        input with information such as confidence scores, time
        stamps, input modality and alternative recognition
        hypotheses. World Wide Web Consortium, 2007.</dd>

        <dt class="label"><a id="Communicator" name=
        "Communicator"></a>Galaxy</dt>

        <dd><a href=
        "http://communicator.sourceforge.net/"><cite>"Galaxy
        Communicator"</cite></a> , Galaxy Communicator is an open
        source hub and spoke architecture for constructing dialogue
        systems that was developed with funding from Defense
        Advanced Research Projects Agency (DARPA) of the United
        States Government.</dd>

        <dt class="label"><a id="MMI-ARCH" name=
        "MMI-ARCH"></a>MMI-ARCH</dt>

        <dd><a href=
        "http://www.w3.org/TR/2008/WD-mmi-arch-20080414/"><cite>"Multimodal
        Architecture and Interfaces (Working Draft)"</cite></a> ,
        Jim Barnett et al. editors. This specification describes a
        loosely coupled architecture for multimodal user
        interfaces, which allows for co-resident and distributed
        implementations, and focuses on the role of markup and
        scripting, and the use of well defined interfaces between
        its constituents. World Wide Web Consortium, 2008.</dd>

        <dt class="label"><a id="MMIF" name="MMIF"></a>MMIF</dt>

        <dd><a href=
        "http://www.w3.org/TR/mmi-framework/"><cite>"W3C Multimodal
        Interaction Framework"</cite></a> , James A. Larson, T.V.
        Raman and Dave Raggett, editors, World Wide Web Consortium,
        2003.</dd>

        <dt class="label"><a id="MMIUse" name=
        "MMIUse"></a>MMIUse</dt>

        <dd><a href=
        "http://www.w3.org/TR/mmi-use-cases/#driving-dir"><cite>"W3C
        Multimodal Interaction Use Cases"</cite></a>, Emily Candell
        and Dave Raggett, editors, World Wide Web Consortium,
        2002.</dd>

        <dt class="label"><a id="SCXML" name="SCXML"></a>SCXML</dt>

        <dd><a href=
        "http://www.w3.org/TR/2008/WD-scxml-20080516/"><cite>"State
        Chart XML (SCXML): State Machine Notation for Control
        Abstraction (Working Draft)"</cite></a> , Jim Barnett et
        al. editors. SCXML, or the "State Chart extensible Markup
        Language", provides a generic state-machine based execution
        environment based on CCXML and Harel State Tables. World
        Wide Web Consortium, 2008.</dd>

        <dt class="label"><a id="VoiceXML" name=
        "VoiceXML"></a>VoiceXML</dt>

        <dd><a href="http://www.w3.org/TR/voicexml21/"><cite>"Voice
        Extensible Markup Language (VoiceXML) Version
        2.1"</cite></a> , Matt Oshry et al. editors. VoiceXML, the
        Voice Extensible Markup Language, is designed for creating
        audio dialogs that feature synthesized speech, digitized
        audio, recognition of spoken and DTMF key input, recording
        of spoken input, telephony, and mixed initiative
        conversations. Its major goal is to bring the advantages of
        Web-based development and content delivery to interactive
        voice response applications. World Wide Web Consortium,
        2007.</dd>

        <dt class="label"><a id="XHTML" name="XHTML"></a>XHTML</dt>

        <dd><a href="http://www.w3.org/TR/xhtml1/"><cite>" XHTML
        1.0 The Extensible HyperText Markup Language (Second
        Edition)"</cite></a> , Steven Pemberton et al. editors.
        World Wide Web Consortium, 2004.</dd>
      </dl>

      <dl>
        <dt class="label"><a id="XMLHttpRequest" name=
        "XMLHttpRequest"></a>XMLHttpRequest</dt>

        <dd><a href=
        "http://www.w3.org/TR/XMLHttpRequest/"><cite>The
        XMLHttpRequest Object (Working Draft)</cite></a> , Anne van
        Kesteren. World Wide Web Consortium, 2008.</dd>
      </dl>
    </div>
  </div>
</body>
</html>