index.html 13.2 KB

Raw Blame History Permalink

<!DOCTYPE html PUBLIC "-//W3C//DTD XHTML 1.0 Strict//EN"
       "http://www.w3.org/TR/xhtml1/DTD/xhtml1-strict.dtd">
<html xmlns="http://www.w3.org/1999/xhtml" xml:lang="en" lang="en">
<head>
  <meta http-equiv="Content-Type" content="text/html; charset=urf-8" />
  <title>W3C QA - MUTAT developer information</title>
  <meta name="Keywords"
  content="qa, quality assurance, conformance, validity, test suite, test,tool,earl"
  />
  <meta name="Description" content="W3C QA - MUTAT: test automating tool" />
  <link rel="schema.DC" href="http://purl.org/dc" />
  <meta name="DC.Subject" xml:lang="en" lang="en" content="test,tool,earl" />
  <meta name="DC.Title" xml:lang="en" lang="en" content="MUTAT" />
  <meta name="DC.Description.Abstract" xml:lang="en" lang="en"
  content="MUTAT: test automating tool" />
  <meta name="DC.Date.Created" content="2002-07-26" />
  <meta name="DC.Language" scheme="RFC1766" content="en" />
  <meta name="DC.Creator" content="Olivier Thereaux" />
  <meta name="DC.Publisher"
  content="W3C - World Wide Web Consortium - http://www.w3.org" />
  <meta name="DC.Rights"
  content="http://www.w3.org/Consortium/Legal/copyright-documents-19990405" />
  <link rel="Stylesheet" href="/QA/2002/12/qa4.css" />
</head>

<body>
<!-- Header -->

<div id="Logo">
<a href="http://www.w3.org/"><img alt="W3C" src="/Icons/WWW/w3c_home" /></a>
<a href="http://www.w3.org/QA/"><img alt="QA" src="/QA/images/qa" width="161"
height="48" /></a> <!-- <div id="Header">Be strict to be cool</div> -->


<div>
<map name="introLinks" id="introLinks" title="Introductory Links">

  <div class="banner">
  <a class="bannerLink" title="W3C Activities" accesskey="A"
  href="/Consortium/Activities">Activities</a> | <a class="bannerLink"
  title="Technical Reports and Recommendations" accesskey="T"
  href="/TR/">Technical Reports</a> | <a class="bannerLink"
  title="Alphabetical Site Index" accesskey="S" href="/Help/siteindex">Site
  Index</a> | <a class="bannerLink" title="Help for new visitors"
  accesskey="N" href="/2002/03/new-to-w3c">New Visitors</a> | <a
  class="bannerLink" title="About W3C" accesskey="B"
  href="/Consortium/">About W3C</a> | <a class="bannerLink" title="Join W3C"
  accesskey="J" href="/Consortium/Prospectus/Joining">Join W3C</a></div>
</map>
</div>
</div>
<!-- menuRight -->

<div id="Menu">
<p><a href="#about">About MUTAT</a><span class="dot">·</span><a
href="#Getting">Getting the Source</a><span class="dot">·</span><a
href="#files">The Files</a><span class="dot">·</span><a
href="#changelog">Change History</a><span class="dot">·</span></p>
<hr />

<p class="navhead">Nearby:</p>

<p><a href="bin/start">MUTAT demos</a><span class="dot">·</span> <a
href="/QA/"><abbr title="Quality Assurance">QA</abbr> Homepage</a><span
class="dot">·</span> <a href="/QA/#latest">Latest News</a><span
class="dot">·</span> <a href="/QA/#resources">QA Resources</a><span
class="dot">·</span> <a href="/QA/IG/">QA <abbr
title="Interest Group">IG</abbr></a><span class="dot">·</span> <a
href="/QA/WG/">QA <abbr title="Working Group">WG</abbr></a><span
class="dot">·</span> <a href="/QA/Agenda/">QA Calendar</a><span
class="dot">·</span></p>
</div>
<!-- content -->

<div id="Content">
<h1>MUTAT - Developer information</h1>

<p>MUTAT is a simple cgi script to demonstrate possible uses of <a
href="http://www.w3.org/2001/03/earl/">EARL</a>, and also use of RDF to
configure an application. It is designed to help perform Quality Assurance
evaluation tasks.</p>

<div>
<h2 id="use">Using it</h2>

<p>You can go straight to the <a href="bin/start">demonstration page</a> of
the testing tool which has been updated to be more user-friendly.</p>

<p>You can also make a URI that calls it directly, such as <a
href="bin/mutat?script_status=1&amp;amp;TEST_url=http://www.w3.org/QA/Tools/MUTAT/wcag.n3">http://www.w3.org/QA/Tools/MUTAT/bin/mutat?script_status=1&amp;TEST_url=http://www.w3.org/QA/Tools/MUTAT/wcag.n3</a></p>
</div>

<h2 id="about">About MUTAT</h2>

<p>QA experts have come up with a variety of solutions to handle entry and
retrieval of the data they need - sometimes little more than a paper and
pencil. The idea behind MUTAT was to demonstrate automating the process for
human-centered testing, and output the results in EARL. The script presents
the user with a series of questions: first a page of fill-in information on
their identity and testing environment, and then the actual tests. The tests
can be formatted either as a single page of text-based questions and links or
a succession of framed HTML pages, with form inputs for the results. Both the
initial information questions and the test questions are configured using an
RDF file. There is also a feature that allows filtering of test questions.</p>

<p>The script uses the Ginger Alliance's <a
href="http://www.gingerall.com/charlie/ga/xml/p_rdf.xml">RDF n3 parser</a>.
@@Is that still a sensible decision?</p>

<h3 id="source"><a name="Getting" id="Getting">Getting the Source
Code</a></h3>

<p>the code is available read-only, using the CVS server at W3C.</p>
<ul>
  <li><p>Set the CVSROOT parameter:
    <code>CVSROOT=:pserver:anonymous@dev.w3.org:/sources/public</code><br />
    with a unix shell (e.g bash) type :<br />
    <code>me@myhost:~$ export
    CVSROOT=:pserver:anonymous@dev.w3.org:/sources/public</code>.<br />
    Or see the documentation for your CVS client.</p>
  </li>
  <li><p>login : <br />
    <code>me@myhost:~$ cvs login</code><br />
    (password:<code>anonymous</code>).</p>
  </li>
  <li><p>checkout the module <code>perl/modules/W3C/MUTAT</code>. <br />
    with a unix shell (e.g bash) type :<br />
    <code>me@myhost:~$ cvs checkout perl/modules/W3C/MUTAT</code></p>
  </li>
</ul>

<p>You can also browse the corresponding <a
href="http://dev.w3.org/cvsweb/perl/modules/W3C/MUTAT">CVS repository</a>
with your favorite browser.</p>

<h3 id="todo">TODO</h3>
<ul>
  <li>Update to new version of EARL as implemented by Hera, AccVerify,
    axforms.</li>
  <li>Document process for updating EARL versions in case the working group
    restarts.</li>
  <li>Restart discussion on creation and use of a public EARL store</li>
  <li>Accept XML RDF. For instance it should be able to read the description
    of WCAG at http://www.w3.org/2001/sw/Europe/200305/axforms/earlInst.rdf
    at least.</li>
  <li>Error-check on user input</li>
  <li>Allow for completion of partial reports (should be possible with
    working EARL store)</li>
  <li>Document how to describe a test set more clearly</li>
</ul>

<h3><a name="files" id="files">The files @@incomplete</a></h3>
<dl>
  <dt>mutat</dt>
    <dd>A perl file that runs the mutat thing. Asks for a test file if called
      with no parameters. There are parameters to make it ask for info like
      tool etc (needs to be fixed to work for WCAG example), or to run the
      tests themselves.</dd>
  <dt>EARL.pm</dt>
    <dd>Some PERL that allows for the creation of XML/RDF, and some specific
      stuff for EARL. Needs commenting and documenting. Not sure that it is a
      great thing to stick with - guess it's fine for PERL programmers.</dd>
  <dt>RDF/Notation3/*</dt>
    <dd>The RDF N3 parser from the Ginger Alliance. It probably makes sense
      to replace this stuff over the medium term with an RDF/XML parser such
      as Redland.</dd>
  <dt>RDFcloud.pm</dt>
    <dd>some stuff for collecting RDF assertions.</dd>
  <dt>RDFtest.pm</dt>
    <dd>some stuff for querying RDF.</dd>
  <dt>create</dt>
    <dd>A PERL script that helps you to create a new set of tests.
      Unfortunately the key step, of adding an actual test case, seems not to
      work at the moment. And it is not clear to me how it interprets the
      other stuff.</dd>
</dl>

<h3>The text file of tests</h3>

<p>The text file is now in RDF. Namespace can be found <a href="ns">here</a>.
Right now the script only accepts files in n3, but there are tools to convert
between formats. The easiest way to figure out what's going on is to look at
the example file <a href="wcag.n3">wcag.n3</a> above for examples of all the
features. There is also a <a href="bin/create">script</a> to help automate
creation of test files. @@Check out script, documentation.</p>

<p>Features to play with:</p>
<ul>
  <li><p>Add your own properties for the earl:Assertor or earl:TestSubject
    (change the domain to the proper value)</p>
    <pre>:thingy a rdf:Property;
rdfs:label "myThingy";
rdfs:domain earl:TestSubject; rdfs:range rdfs:Literal .</pre>
  </li>
  <li><p>Add default values to any property (sorry about the
    re-declaration)</p>
    <pre>earl:date a rdf:Property;
rdf:value "Wed 13 Mar 2002" .</pre>
  </li>
  <li><p>Add a limitless number of possible groupings to each TestCase to
    filter which bits you would like to see - declare like below and make up
    whatever values you'd like</p>
    <pre>:priority a td:GroupProperty;
rdfs:label "priority" .</pre>
  </li>
  <li><p>Declare testcases with any properties you'd like to be copied into
    the final report, even ones that aren't used by my script</p>
    <pre>:test01 a earl:TestCase ;
rdfs:label "Test A";
:priority "C";
earl:id &lt;HTTP: ATest.html Something www.w3.org&gt; ;
td:text "This statement is false." .
:test02 a earl:TestCase ;
rdfs:label "Test Frog";
:priority "Argh";
earl:id &lt;HTTP: somewhere some.domain&gt; ;
td:text "My feet are green." .</pre>
  </li>
  <li><p>Make up your own choices for results to override earl's
    passes/fails</p>
    <pre>:C a td:ResultProperty; rdfs:label "Complete";
earl:validity earl:Pass;
earl:confidence earl:High .</pre>
  </li>
</ul>

<h2 id="changelog">ChangeLog</h2>
<ul>
  <li>2005-01-11: Started a new test set for SpecGL. Fixed the WCAG set to
    ask for a page being tested. Tested create script and noted that it can't
    add tests.</li>
  <li>2005-01-03: reordered dates. Split off a page to start using it nicely
    from this developer info. Started commenting the code.</li>
  <li>2002-07-25 / 2002-07-26: finally moving the stuff to web/CVS space of
    the QA activity at W3C.</li>
  <li>2002-04-22: Recovered lost page from Google cache (yay!). Changed EARL
    output to reflect change proposed in <a
    href="http://lists.w3.org/Archives/Public/w3c-wai-er-ig/2002Mar/0017.html">w3c-wai-er-ig</a>
    a month ago.</li>
  <li>2002-04: Created script to automate creation of n3 test files</li>
  <li>2002-03-12: Changed test format again... hopefully it's legal. Added
    ability to read beginning questions in from earl spec, td spec, and test
    file. Added ability to set default values with above. Added ability to
    override earl's existing ResultProperties. Added ability to carry all
    test information on so it can be printed in final earl report. Added
    ability to create your own grouping properties for tests so they can be
    filtered on any number of criteria. Abstracted a lot of stuff away from
    mutat script and into odd data structures to holddata.</li>
  <li>2002-03-04: Changed test format again in attempt to make it more
    earl-like, added "group" property for tests that allows them to be
    filtered, added ability to post to annotea database (but it'll give an
    error if you do), added ability to post proper incomplete tests and added
    a quick-out button for the frames view so you don't have to do all tests
    until the end</li>
  <li>2002-02-27: Made perl module to simplify EARL creation, added ability
    to make comments</li>
  <li>2002-02-25: Changed test question format to n3 (and use someone's n3
    RDF parser to parse), changed script's data structure for tests and then
    added object wrapper with pretty access methods</li>
  <li>2002-02-18: Added access to <a href="http://jibbering.com/">Jim
    Ley's</a> EARL database.</li>
  <li>2002-02-15: Made source code public</li>
  <li>2002-02-13: Created!</li>
</ul>
</div>
<!-- Footer -->
<hr />

<div class="disclaimer">
<a href="http://validator.w3.org/check/referer"><img
src="http://validator.w3.org/images/vxhtml10" alt="Valid XHTML 1.0!"
height="31" width="88" /></a>
<address class="author">
  Document created Date: 2002-04-22 by <a
  href="mailto:nadia@barbwired.com">Nadia Heninger</a><br />
  Document moved to location Date: 2002-07-26 by <a
  href="mailto:ot@w3.org">Olivier Thereaux</a><br />
  Last modified $Date: 2008/01/14 15:55:07 $ by $Author:
  charles $
</address>

<p class="policyfooter"><a rel="Copyright"
href="/Consortium/Legal/ipr-notice#Copyright">Copyright</a> © 2000-2003 <a
href="/"><acronym
title="World Wide Web Consortium">W3C</acronym></a><sup>®</sup> (<a
href="http://www.lcs.mit.edu/"><acronym
title="Massachusetts Institute of Technology">MIT</acronym></a>, <a
href="http://www.ercim.org/"><acronym
title="European Research Consortium for Informatics and Mathematics">ERCIM</acronym></a>,
<a href="http://www.keio.ac.jp/">Keio</a>), All Rights Reserved. W3C <a
href="/Consortium/Legal/ipr-notice#Legal_Disclaimer">liability</a>, <a
href="/Consortium/Legal/ipr-notice#W3C_Trademarks">trademark</a>, <a
rel="Copyright" href="/Consortium/Legal/copyright-documents">document use</a>
and <a rel="Copyright" href="/Consortium/Legal/copyright-software">software
licensing</a> rules apply. Your interactions with this site are in accordance
with our <a href="/Consortium/Legal/privacy-statement#Public">public</a> and
<a href="/Consortium/Legal/privacy-statement#Members">Member</a> privacy
statements.</p>
</div>
</body>
</html>