index.html 33 KB

Raw Blame History Permalink

<!DOCTYPE html PUBLIC "-//W3C//DTD XHTML 1.0 Strict//EN"
    "http://www.w3.org/TR/xhtml1/DTD/xhtml1-strict.dtd">
<html xmlns="http://www.w3.org/1999/xhtml" lang="en" xml:lang="en">
<head>
    <meta http-equiv="content-type" content="text/html; charset=utf-8" />
    <title>Test Metadata</title>
    <link rel="stylesheet" type="text/css" href="guideline.css" />
    <link rel="stylesheet" type="text/css" href="http://www.w3.org/StyleSheets/TR/W3C-WG-NOTE" />
    <meta name="Keywords" content="qa, quality assurance, test, test suite" />
    <meta name="Description" content="The definition and provision of metadata has proved helpful in a variety of ways during the test development and test execution processes. This document defines a minimal set of metadata elements that can usefully be applied to tests that are intended for publication within a test suite." />
    <link rel="schema.DC" href="http://dublincore.org/" />
    <meta name="DC.Subject" xml:lang="en" lang="en" content="qa, quality assurance, test, test suite" />
    <meta name="DC.Title" xml:lang="en" lang="en" content="The QA Handbook" />
    <meta name="DC.Description.Abstract" xml:lang="en" lang="en" content="The definition and provision of metadata has proved helpful in a variety of ways during the test development and test execution processes. This document defines a minimal set of metadata elements that can usefully be applied to tests that are intended for publication within a test suite." />
    <meta name="DC.Language" scheme="RFC1766" content="en" />
    <meta name="DC.Publisher" content="W3C - World Wide Web Consortium - http://www.w3.org" />
    <meta name="DC.Rights" content="http://www.w3.org/Consortium/Legal/ipr-notice#Copyright" />
</head>
<body>
    <div class="head">
        <p><a href="http://www.w3.org/"><img height="48" width="72" alt="W3C" src="http://www.w3.org/Icons/w3c_home" /></a> </p>
        <h1 id="spec-title">Test Metadata</h1>
        <h2 id="dated-subtitle">W3C Working Group Note 14 September 2005</h2>
        <dl>
            <dt>This version:</dt>
            <dd><a href="http://www.w3.org/TR/2005/NOTE-test-metadata-20050914/">http://www.w3.org/TR/2005/NOTE-test-metadata-20050914/</a> </dd>
            <dt>Latest version:</dt>
            <dd><a href="http://www.w3.org/TR/test-metadata/">http://www.w3.org/TR/test-metadata/</a> </dd>
            <dt>Editors:</dt>
            <dd>Patrick Curran, Sun Microsystems</dd>
            <dd>Karl Dubost, W3C</dd>
            <dt>Contributors:</dt>
            <dd><a shape="rect" href="#acknowledgments">See Acknowledgments.</a> </dd>
        </dl>
        <p class="copyright"><a href="http://www.w3.org/Consortium/Legal/ipr-notice#Copyright">Copyright</a>  © 2004-2005 <a href="http://www.w3.org/"><acronym title="World Wide Web Consortium">W3C</acronym></a> <sup>®</sup> (<a href="http://www.csail.mit.edu/"><acronym title="Massachusetts Institute of Technology">MIT</acronym></a> , <a href="http://www.ercim.org/"><acronym title="European Research Consortium for Informatics and Mathematics">ERCIM</acronym></a> , <a href="http://www.keio.ac.jp/">Keio</a> ), All Rights Reserved. W3C <a href="http://www.w3.org/Consortium/Legal/ipr-notice#Legal_Disclaimer">liability</a> , <a href="http://www.w3.org/Consortium/Legal/ipr-notice#W3C_Trademarks">trademark</a>  and <a href="http://www.w3.org/Consortium/Legal/copyright-documents">document use</a>  rules apply.</p>
        <hr />
    </div>
    <h2 id="abstract">Abstract</h2>

	<p>The definition and provision of metadata has proved helpful in a variety of ways during the test development and test execution processes. This document defines a minimal set of metadata elements that can usefully be applied to tests that are intended for publication within a test suite.</p>

    <h2 id="status">Status of this document</h2>

<p><em>This section describes the status of this document at the time of its publication. Other documents may supersede this document. A list of current W3C publications and the latest revision of this technical report can be found in the <a href="http://www.w3.org/TR/">W3C technical reports index</a> at http://www.w3.org/TR/.</em></p>

<p>This document is a W3C Working Group Note. It has been produced by the <a href="http://www.w3.org/QA/WG/">Quality Assurance Working Group</a>, which is part of the Quality Assurance Activity. It has been made available for discussion by W3C members and other interested parties. For more information about the QA Activity, please see the <a href="http://www.w3.org/QA/Activity">QA Activity statement</a>. <a href="http://www.w3.org/2003/03/Translations/byTechnology?technology=test-metadata">Translations</a> of this document may be available.</p>

<p>This is the first published version of this document. It has been written along with the QA Framework by collecting the test development experience of W3C Working Groups and summarizing the work done on the Wiki about <a href="http://esw.w3.org/topic/TestCaseMetadata">tests and metadata</a>.</p>

<p>Publication as a Working Group Note does not imply endorsement by the W3C Membership. This is a draft document and may be updated, replaced or obsoleted by other documents at any time. It is inappropriate to cite this document as other than work in progress.</p>

<p>The QA Working Group does not expect this document to become a Recommendation. Rather, after further development, review and refinement, it may be updated and maintained as an Interest Group Note.</p>

<p>You may email comments on this document to <a href="mailto:www-qa@w3.org">www-qa@w3.org</a>, the <a href="http://lists.w3.org/Archives/Public/www-qa/">publicly archived</a> list of the QA Interest Group.</p>

    <h2 id="contents">Table of contents</h2>

    <!--index-->
<ul>
	<li><a href="#intro">1. Introduction</a></li>
	<li><a href="#elements">2. Metadata elements</a>
	<ul>
	<li><a href="#identifier-def">2.1  Identifier</a></li>
	<li><a href="#title-def">2.2  Title</a></li>
	<li><a href="#purpose-def">2.3  Purpose</a></li>
	<li><a href="#description-def">2.4  Description</a></li>
	<li><a href="#status-def">2.5  Status</a></li>
	<li><a href="#specref-def">2.6  SpecRef</a></li>
	<li><a href="#preconditions-def">2.7  Preconditions</a></li>
	<li><a href="#inputs-def">2.8  Inputs</a></li>
	<li><a href="#expectedresults-def">2.9  ExpectedResults</a></li>
	<li><a href="#version-def">2.10 Version</a></li>
	<li><a href="#contributor-def">2.11 Contributor</a></li>
	<li><a href="#rights-def">2.12 Rights</a></li>
	<li><a href="#grouping-def">2.13 Grouping</a></li>
	<li><a href="#seealso-def">2.14 SeeAlso</a></li>
	</ul>
	</li>
	<li><a href="#acknowledgments">3. Acknowledgments</a></li>
	<li><a href="#references">4. References</a></li>
</ul>
<!--/index-->

    <hr />
    <h2 id="intro">1. Introduction</h2>

    <p>To be truly useful, a test suite should consist of more than a simple collection of tests. Additional information is typically required to help users understand how to execute the tests and how to interpret the results. Much of this information should be provided in the test suite documentation but some is more appropriately provided in the form of <em>metadata</em> about the tests themselves. Well-defined metadata can help in:</p>

<ul>
	<li>tracking tests during the development and review process</li>
	<li>filtering tests according to a variety of criteria (for example, whether or not they are applicable for a particular profile or optional feature)</li>
	<li>identifying the area of the specification that is tested by the tests</li>
	<li>constructing a test harness to automatically execute the tests</li>
	<li>formatting test results so that they are easily understood</li>
</ul>

<p>Most test suites provided by W3C Working Groups make use of some form of metadata. However, the extent of metadata usage and the forms and syntax in which metadata elements are defined varies widely from Group to Group. This document defines a minimal set of metadata elements that have proved useful in practice and attempts to standardize their names, syntax, and usage. If the use of standard metadata elements is adopted within the W3C it is likely that standardized tools will be developed to facilitate the tasks listed above.</p>

<p>Please note that these metadata elements are simply suggestions. Additional elements may be appropriate for your implementation. It is also likely that you will also want to define metadata elements that are associated with the entire test suite rather than with individual tests, and it may even make sense to define metadata about metadata (for example, the Dublin Core elements <code><a href="http://dublincore.org/documents/dces/#date">Date</a></code> [<a href="#DCES">DCES</a>] and <code><a href="http://dublincore.org/documents/dces/#language">Language</a></code> [<a href="#DCES">DCES</a>] may usefully be applied to several of the elements defined below.)</p>

<p>This document re-uses the Dublin Core elements where it seems appropriate. <cite><a href="http://dublincore.org/documents/2005/06/13/dcmi-terms/">DCMI Metadata Terms</a></cite>  [<a href="#DCMI">DCMI</a>] gives an introduction to the use of metadata. For background information on the test development process and additional rationale for the the use of metadata, the QA Working Group has created a <cite><a href="http://www.w3.org/QA/WG/2005/01/test-faq">Test FAQ</a></cite> [<a href="#TEST-FAQ">TEST-FAQ</a>]. A practical implementation of some of these principles is available in <cite>XML Query Test Suite</cite> defined by the XML Query and XSL Working Groups. (Their <cite><a href="http://www.w3.org/XML/Query/test-suite/Guidelines%20for%20Test%20Submission.html">Guidelines for Test Submission</a></cite> [<a href="#XQUERY-TEST">XQUERY-TEST</a>] and <cite><a href="http://www.w3.org/XML/Query/test-suite/Guidelines%20for%20Running%20the%20XML%20Query%20Test%20Suite.html">Guidelines for Test Execution</a></cite> [<a href="#XQUERY-TEST">XQUERY-TEST</a>] provide explanations and examples.)</p>


<h2 id="elements">2. Metadata elements</h2>


    <h3 id="identifier-def">2.1 Identifier</h3>
    <table>
        <tr>
            <th>Name</th>
            <td><em>Identifier</em></td>
        </tr>
        <tr>
            <th>Description</th>
            <td>An unambiguous reference to the test.</td>
        </tr>
        <tr>
            <th>Rationale</th>
            <td>Enables the test to be referenced and identified.</td>
        </tr>
        <tr>
            <th>Required?</th>
            <td>Yes.</td>
        </tr>
        <tr>
            <th>Syntax</th>
            <td>Free-form (URI recommended).</td>
        </tr>
        <tr>
            <th>Example</th>
            <td>CSS 2.1 Test Suite: <a href="http://www.w3.org/Style/CSS/Test/CSS2.1/current/t0510-c25-pseudo-elmnt-00-c.htm">Pseudo-elements in Selectors</a> [<a href="#CSS-TEST">CSS-TEST</a>]</td>
        </tr>
        <tr>
            <th>See Also</th>
            <td>
            <ul>
	<li>Dublin Core: <code><a href="http://dublincore.org/documents/dces/#identifier"> Identifier</a></code> [<a href="#DCES">DCES</a>]</li>
	<li>Test Metadata: <code><a href="#title-def">Title</a></code> </li>
</ul>
</td>
        </tr>
        <tr>
            <th>Comments</th>
            <td>Most W3C test suites use URIs as identifiers.</td>
        </tr>
    </table>

    <h3 id="title-def">2.2 Title</h3>
    <table>
        <tr>
            <th>Name</th>
            <td><em>Title</em></td>
        </tr>
        <tr>
            <th>Description</th>
            <td>The name by which the test is formally known.</td>
        </tr>
        <tr>
            <th>Rationale</th>
            <td>It may be advantageous to provide a more human-oriented alternative to the <em><code>Identifier</code></em>.</td>
        </tr>
        <tr>
            <th>Required?</th>
            <td>No.</td>
        </tr>
        <tr>
            <th>Syntax</th>
            <td>Text.</td>
        </tr>
        <tr>
            <th>Example</th>
            <td>The <cite><a href="http://www.w3.org/Style/CSS/Test/CSS2.1/current/">CSS Test Suite</a></cite> [<a href="#CSS-TEST">CSS-TEST</a>] uses the term <em><code>Description</code></em>for human readable names that more properly match this <em>Title</em> element. (For example, <em>CSS 2.1 Test Suite: Co&gt;.)</em></td>
        </tr>
        <tr>
            <th>See Also</th>
            <td><ul>
	<li>Dublin Core: <a href="http://dublincore.org/documents/dces/#title">title</a> [<a href="#DCES">DCES</a>]</li>
	<li>Test Metadata: <a href="#identifier-def">Identifier</a></li>
</ul>
 </td>
        </tr>
        <tr>
            <th>Comments</th>
            <td>This element is optional because the <em><code><a href="#identifier-def">Identifier</a></code></em> will often be sufficient. Note also that in very large test suites it may be impractical to define unambiguous <em><code>Title</code>.</em></td>
        </tr>
    </table>

    <h3 id="purpose-def">2.3 Purpose</h3>
    <table>
        <tr>
            <th>Name</th>
            <td><em>Purpose</em></td>
        </tr>
        <tr>
            <th>Description</th>
            <td>A brief explanation of the reason the test was developed.</td>
        </tr>
        <tr>
            <th>Rationale</th>
            <td>Provides additional context for the test.</td>
        </tr>
        <tr>
            <th>Required?</th>
            <td>Yes.</td>
        </tr>
        <tr>
            <th>Syntax</th>
            <td>Free-form (hypertext) or constrained plain text</td>
        </tr>
        <tr>
            <th>Example</th>
            <td>The <a href="http://www.w3.org/Style/CSS/Test/CSS1/current/sec545.htm">CSS1 test suite</a> [<a href="#CSS-TEST">CSS-TEST</a>] provides such explanations. For example: “This page tests the 'text-transform' property of CSS1”.</td>
        </tr>
        <tr>
            <th>See Also</th>
            <td><ul>
	<li>Test Metadata: <code><a href="#description-def">Description</a></code> </li>
	<li>Test Metadata: <code><a href="#specref-def">SpecRef</a></code></li>
</ul>
 </td>
        </tr>
        <tr>
            <th>Comments</th>
            <td>The <em><code>Purpose</code></em> should typically be kept short enough (a 'one liner') to be displayed in tabular format (other constraints, for example a prohibition on embedded newlines, may also be required to ensure that this element can be processed by various tools). The <em><code><a href="#description-def">Description</a></code></em> element should be used where a more detailed explanation is required.</td>
        </tr>
    </table>

    <h3 id="description-def">2.4 Description</h3>
    <table>
        <tr>
            <th>Name</th>
            <td><em>Description</em></td>
        </tr>
        <tr>
            <th>Description</th>
            <td>A representation in words of the nature and characteristics of the test.</td>
        </tr>
        <tr>
            <th>Rationale</th>
            <td>A detailed explanation can help reviewers to understand the purpose of a test and may also provide those executing the test with information that will be helpful if the test fails.</td>
        </tr>
        <tr>
            <th>Required?</th>
            <td>No.</td>
        </tr>
        <tr>
            <th>Syntax</th>
            <td>Free-form (hypertext).</td>
        </tr>
        <tr>
            <th>Example</th>
            <td>The <a href="http://www.cgmopen.org/resources/webcgm-test-suite.html">WebCGM test suite</a> [<a href="#WEBCGM-TEST">WEBCGM-TEST</a>] incorporates detailed descriptions. For example: "Test basic CGM-to-HTML link, from a Application Structure with linkuri APS Attribute within the CGM, to a whole HTML document, with no behaviors or fragments associated with the link."</td>
        </tr>
        <tr>
            <th>See Also</th>
            <td><ul>
	<li>Dublin Core: <code><a href="http://dublincore.org/documents/dces/#description">Description</a></code> [<a href="#DCES">DCES</a>]</li>
	<li>            Test Metadata: <code><a href="#purpose-def">Purpose</a></code></li>
</ul>
 </td>
        </tr>
        <tr>
            <th>Comments</th>
            <td>This element can be used to provide more detailed information than can be provided in the relatively short <em><code><a href="#purpose-def">Purpose</a></code></em> element.</td>
        </tr>
    </table>

    <h3 id="status-def">2.5 Status</h3>
    <table>
        <tr>
            <th>Name</th>
            <td><em>Status</em></td>
        </tr>
        <tr>
            <th>Description</th>
            <td>One of an enumerated list of values that can be used to track the state of a test at a given time.</td>
        </tr>
        <tr>
            <th>Rationale</th>
            <td>It can be helpful to track the status of a test during the development process.</td>
        </tr>
        <tr>
            <th>Required?</th>
            <td>No.</td>
        </tr>
        <tr>
            <th>Syntax</th>
            <td>Constrained choice from an enumerated list</td>
        </tr>
        <tr>
            <th>Example</th>
            <td>One of: <em>unconfirmed</em>, <em>new</em>, <em>assigned</em>, <em>pending</em>, <em>accepted</em>, <em>rejected</em>, <em>holding</em> (see <a href="http://www.w3.org/WAI/GL/WCAG20/tests/ctprocess.html">Conformance Test Process For WCAG 2.0</a> [<a href="#WCAG2-CONF-PROCESS">WCAG2-CONF-PROCESS</a>] and their <a href="http://www.w3.org/WAI/GL/WCAG20/tests/checkstatus.html">test Status document</a>).</td>
        </tr>
        <tr>
            <th>See Also</th>
            <td>&nbsp;</td>
        </tr>
        <tr>
            <th>Comments</th>
            <td>This metadata element may also be useful after test suite publication, when it could be used to record the state of challenges to the validity of the test.</td>
        </tr>
    </table>

    <h3 id="specref-def">2.6 SpecRef</h3>
    <table>
        <tr>
            <th>Name</th>
            <td><em>SpecRef</em></td>
        </tr>
        <tr>
            <th>Description</th>
            <td>Identification of the portion of the specification tested by this test.</td>
        </tr>
        <tr>
            <th>Rationale</th>
            <td>Traceability back to the specification.</td>
        </tr>
        <tr>
            <th>Required?</th>
            <td>Yes.</td>
        </tr>
        <tr>
            <th>Syntax</th>
            <td>Freeform hypertext or more usefully a link to markup or anchors in the text of the specification.</td>
        </tr>
        <tr>
            <th>Example</th>
            <td>The HTML 4.1 test suite provides a list of <em><a href="http://www.w3.org/MarkUp/Test/HTML401/current/assertions/assertions_toc.html">Testable Assertions</a> </em> that associates individual tests with assertions in the specification.</td>
        </tr>
        <tr>
            <th>See Also</th>
            <td>&nbsp;</td>
        </tr>
        <tr>
            <th>Comments</th>
            <td>The more specific this reference is the better. (Simply pointing to the beginning of a large sub-section of the spec is not helpful. Identifying the exact string containing the requirement to be tested is ideal.)</td>
        </tr>
    </table>

    <h3 id="preconditions-def">2.7 Preconditions</h3>
    <table>
        <tr>
            <th>Name</th>
            <td><em>Preconditions</em></td>
        </tr>
        <tr>
            <th>Description</th>
            <td>Conditions that must be met before this test is executed.</td>
        </tr>
        <tr>
            <th>Rationale</th>
            <td>Any such conditions must be understood and met before the test can be successfully executed.</td>
        </tr>
        <tr>
            <th>Required?</th>
            <td>No.</td>
        </tr>
        <tr>
            <th>Syntax</th>
            <td>Free-form (hypertext).</td>
        </tr>
        <tr>
            <th>Example</th>
            <td>It might be necessary that a network connection be available or a server process be running before the test is executed.</td>
        </tr>
        <tr>
            <th>See Also</th>
            <td>Test Metadata: <code><a href="#inputs-def">Inputs</a></code> </td>
        </tr>
        <tr>
            <th>Comments</th>
            <td></td>
        </tr>
    </table>

    <h3 id="inputs-def">2.8 Inputs</h3>
    <table>
        <tr>
            <th>Name</th>
            <td><em>Inputs</em></td>
        </tr>
        <tr>
            <th>Description</th>
            <td>Parameters or data that are needed for the test execution.</td>
        </tr>
        <tr>
            <th>Rationale</th>
            <td>Must be understood and supplied for test execution.</td>
        </tr>
        <tr>
            <th>Required?</th>
            <td>No.</td>
        </tr>
        <tr>
            <th>Syntax</th>
            <td>Implementation dependent.</td>
        </tr>
        <tr>
            <th>Example</th>
            <td><a href="http://www.w3.org/XML/Query/test-suite/Guidelines%20for%20Test%20Submission.html">The XML Query Test Suite metadata</a> [<a href="#XQUERY-TEST">XQUERY-TEST</a>] defines an <em>input_file</em> element.</td>
        </tr>
        <tr>
            <th>See Also</th>
            <td>Test Metadata: <code><a href="#preconditions-def">Preconditions</a></code> </td>
        </tr>
        <tr>
            <th>Comments</th>
            <td>&nbsp;</td>
        </tr>
    </table>

    <h3 id="expectedresults-def">2.9 ExpectedResults</h3>
    <table>
        <tr>
            <th>Name</th>
            <td><em>ExpectedResults</em></td>
        </tr>
        <tr>
            <th>Description</th>
            <td>
                <p>The results that a conformant implementation is expected to produce when this test is executed.</p>
            </td>
        </tr>
        <tr>
            <th>Rationale</th>
            <td>If these are not defined, it will be impossible to determine whether the test passed or failed.</td>
        </tr>
        <tr>
            <th>Required?</th>
            <td>Yes.</td>
        </tr>
        <tr>
            <th>Syntax</th>
            <td>Implementation dependent.</td>
        </tr>
        <tr>
            <th>Example</th>
            <td>tests within the CSS Test Suite embed their expected results within the test's output. See <a href="http://www.w3.org/Style/CSS/Test/CSS2.1/current/t0510-c25-pseudo-elmnt-00-c.htm">this example</a> [<a href="#CSS-TEST">CSS-TEST</a>]. <a href="http://www.w3.org/XML/Query/test-suite/Guidelines%20for%20Test%20Submission.html">The XML Query Test Suite metadata</a> [<a href="#XQUERY-TEST">XQUERY-TEST</a>] defines an <em>output_file</em> element.</td>
        </tr>
        <tr>
            <th>See Also</th>
            <td>&nbsp;</td>
        </tr>
        <tr>
            <th>Comments</th>
            <td>&nbsp;</td>
        </tr>
    </table>

    <h3 id="version-def">2.10 Version</h3>
    <table>
        <tr>
            <th>Name</th>
            <td><em>Version</em></td>
        </tr>
        <tr>
            <th>Description</th>
            <td>An identifier that allows one to distinguish between different revisions of test.</td>
        </tr>
        <tr>
            <th>Rationale</th>
            <td>tests often evolve over time, and it is important to maintain a history and to be able to identify and distinguish between different revisions.</td>
        </tr>
        <tr>
            <th>Required?</th>
            <td>Yes.</td>
        </tr>
        <tr>
            <th>Syntax</th>
            <td>
                <p>Implementation dependent.</p>
            </td>
        </tr>
        <tr>
            <th>Example</th>
            <td>This will often be generated by a source-code control system such as <a href="http://www.nongnu.org/cvs/">CVS</a> .</td>
        </tr>
        <tr>
            <th>See Also</th>
            <td>&nbsp;</td>
        </tr>
        <tr>
            <th>Comments</th>
            <td><span class="issue">ISSUE: It is unclear whether it is necessary to apply version numbers to individual tests. More common practice is to version the entire test suite. Ideally tests will not need to be revised as the specification evolves. However, this is sometimes necessary, and at a minimum each Working Group should define <code><a href="#specref-def">SpecRef</a></code> and possibly other metadata such as <code><a href="#grouping-def">Grouping</a></code> elements to indicate which versions of the specification an individual test supports.</span></td>
        </tr>
    </table>

    <h3 id="contributor-def">2.11 Contributor</h3>
    <table>
        <tr>
            <th>Name</th>
            <td><em>Contributor</em></td>
        </tr>
        <tr>
            <th>Description</th>
            <td>The individual or organization that contributed this test.</td>
        </tr>
        <tr>
            <th>Rationale</th>
            <td>It may be necessary to contact the contributor to ask for information about the test or to request an update.</td>
        </tr>
        <tr>
            <th>Required?</th>
            <td>Yes.</td>
        </tr>
        <tr>
            <th>Syntax</th>
            <td>Free-form.</td>
        </tr>
        <tr>
            <th>Example</th>
            <td>The <a href="http://www.w3.org/XML/Test/#releases">XML Conformance Test Suite</a> [<a href="#XML-TEST">XML-TEST</a>] encodes the contributor in the directory structure used to store the tests.</td>
        </tr>
        <tr>
            <th>See Also</th>
            <td>Dublin Core: <code><a href="http://dublincore.org/documents/dces/#description">Contributor</a></code> [<a href="#DCES">DCES</a>]</td>
        </tr>
        <tr>
            <th>Comments</th>
            <td>&nbsp;</td>
        </tr>
    </table>

    <h3 id="rights-def">2.12 Rights</h3>
    <table>
        <tr>
            <th>Name</th>
            <td><em>Rights</em></td>
        </tr>
        <tr>
            <th>Description</th>
            <td>Information about rights held in and over the test.</td>
        </tr>
        <tr>
            <th>Rationale</th>
            <td>Publishers and users of the test need to understand the associated Intellectual Property Rights (IPR).</td>
        </tr>
        <tr>
            <th>Required?</th>
            <td>Yes.</td>
        </tr>
        <tr>
            <th>Syntax</th>
            <td>Ask your lawyer.</td>
        </tr>
        <tr>
            <th>Example</th>
            <td>&nbsp;</td>
        </tr>
        <tr>
            <th>See Also</th>
            <td>Dublin Core: <code><a href="http://dublincore.org/documents/dces/#rights">Rights</a></code> [<a href="#DCES">DCES</a>]</td>
        </tr>
        <tr>
            <th>Comments</th>
            <td>This will often be simply a pointer to a copyright notice contained within the source code.</td>
        </tr>
    </table>

    <h3 id="grouping-def">2.13 Grouping</h3>
    <table>
        <tr>
            <th>Name</th>
            <td><em>Grouping</em></td>
        </tr>
        <tr>
            <th>Description</th>
            <td>A mechanism for classifying tests into groups.</td>
        </tr>
        <tr>
            <th>Rationale</th>
            <td>To enable the selection of subsets of tests that share certain characteristics.</td>
        </tr>
        <tr>
            <th>Required?</th>
            <td>No.</td>
        </tr>
        <tr>
            <th>Syntax</th>
            <td>Implementation dependent. Possibilities include naming conventions and enumerated lists.</td>
        </tr>
        <tr>
            <th>Example</th>
            <td>Tests may be classified as interactive or automated, positive or negative.</td>
        </tr>
        <tr>
            <th>See Also</th>
            <td>&nbsp;</td>
        </tr>
        <tr>
            <th>Comments</th>
            <td>An important use of grouping techniques is to classify tests that belong to particular profiles, modules, or other discretionary groupings (see <cite><a href="http://www.w3.org/TR/2005/NOTE-spec-variability-20050831/">Variability in Specifications</a></cite> [<a href="#SPEC-VARIABILITY">SPEC-VARIABILITY</a>] for a detailed discussion of these issues). The <cite><a href="http://www.w3.org/Graphics/SVG/Test/">SVG test suite</a></cite> [<a href="#SVG-TEST">SVG-TEST</a>], for example, uses naming conventions to distinguish between tests targeted at different profiles (full, basic, or tiny).</td>
        </tr>
    </table>

    <h3 id="seealso-def">2.14 SeeAlso</h3>
    <table>
        <tr>
            <th>Name</th>
            <td><em>SeeAlso</em></td>
        </tr>
        <tr>
            <th>Description</th>
            <td>A list of references to relevant materials.</td>
        </tr>
        <tr>
            <th>Rationale</th>
            <td>Can help to clarify the intent or usefulness of the test.</td>
        </tr>
        <tr>
            <th>Required?</th>
            <td>No.</td>
        </tr>
        <tr>
            <th>Syntax</th>
            <td>Free-form (hypertext).</td>
        </tr>
        <tr>
            <th>Example</th>
            <td>A pointer to an item in an issue-tracking system such as <cite><a href="http://www.bugzilla.org/">Bugzilla</a></cite> [<a href="#BUGZILLA">BUGZILLA</a>] or to a mailing list thread in which the justification for this test is discussed.</td>
        </tr>
        <tr>
            <th>See Also</th>
            <td>Dublin Core: <code><a href="http://dublincore.org/documents/dces/#relation">Relation</a></code> [<a href="#DCES">DCES</a>]</td>
        </tr>
        <tr>
            <th>Comments</th>
            <td>&nbsp;</td>
        </tr>
    </table>
    <hr />
    <h2 id="acknowledgments">3. Acknowledgments</h2>
    <p>The following QA Working Group and Interest Group participants have contributed significantly to this document:</p>
    <ul>
        <li>Tim Boland (NIST),</li>
        <li>Dimitris Dimitriadis (Ontologicon),</li>
        <li>Dominique Hazaël-Massieux (W3C),</li>
        <li>Lofton Henderson (CGM Open),</li>
        <li>Richard T. Kennedy (Boeing),</li>
        <li>David Marston (IBM Research),</li>
        <li>Patrick Curran (Sun Microsystems),</li>
        <li>Lynne Rosenthal (NIST),</li>
        <li>Mark Skall (NIST).</li>
    </ul>
    <h2 id="references">4. References</h2>

<dl>
<dt id="BUGZILLA">BUGZILLA</dt>
<dd><cite><a href="http://www.bugzilla.org/">Bugzilla</a></cite> http://www.bugzilla.org/ . </dd>

<dt id="CSS-TEST">CSS-TEST</dt>
<dd><cite><a href="http://www.w3.org/Style/CSS/Test/">Cascading Style Sheets test suites</a></cite>, CSS Working Group, http://www.w3.org/Style/CSS/Test/ . </dd>

<dt id="DCES">DCES</dt>
<dd><cite><a href="http://dublincore.org/documents/2004/12/20/dces/">Dublin Core Metadata Element Set, Version 1.1: Reference Description</a></cite>, Dublin Core Metadata Initiative, DCMI Recommendation, 20 December 2004, http://dublincore.org/documents/2004/12/20/dces/ . <a href="http://dublincore.org/documents/dces/" title="Latest version of Dublin Core Metadata Element Set, Version 1.1: Reference Description">Latest version</a> available at http://dublincore.org/documents/dces/ .</dd>

<dt id="DCMI">DCMI</dt>
<dd><cite><a href="http://dublincore.org/documents/2005/06/13/dcmi-terms/">DCMI Metadata Terms</a></cite>, Dublin Core Metadata Initiative, DCMI Recommendation, 13 June 2005, http://dublincore.org/documents/2005/06/13/dcmi-terms/ . <a href="http://dublincore.org/documents/dcmi-terms/" title="Latest version of DCMI Metadata Terms">Latest version</a> available at http://dublincore.org/documents/dcmi-terms/ .</dd>

<dt id="HTML4-TEST">HTML4-TEST</dt>
<dd><cite><a href="http://www.w3.org/MarkUp/Test/HTML401/current/">HTML4 Test Suite</a></cite>, W3C, http://www.w3.org/MarkUp/Test/HTML401/current/ . </dd>

<dt id="SPEC-VARIABILITY">SPEC-VARIABILITY</dt>
<dd><cite><a href="http://www.w3.org/TR/2005/NOTE-spec-variability-20050831/">Variability in Specifications</a></cite>, D. Hazaël-Massieux, L. Rosenthal,  Editors, W3C Working Group Note, 31 August 2005, http://www.w3.org/TR/2005/NOTE-spec-variability-20050831/ . <a href="http://www.w3.org/TR/spec-variability/" title="Latest version of Variability in Specifications">Latest version</a> available at http://www.w3.org/TR/spec-variability/ .</dd>

<dt id="SVG-TEST">SVG-TEST</dt>
<dd><cite><a href="http://www.w3.org/Graphics/SVG/Test/">W3C Scalable Vector Graphics (SVG) 1.1 Test Suite</a></cite>, SVG Working Group, http://www.w3.org/Graphics/SVG/Test/ . </dd>


<dt id="TEST-FAQ">TEST-FAQ</dt>
<dd><cite><a href="http://www.w3.org/QA/WG/2005/01/test-faq">Test Development FAQ</a></cite>, QA Working Group, http://www.w3.org/QA/WG/2005/01/test-faq . </dd>

<dt id="WCAG2-CONF-PROCESS">WCAG2-CONF-PROCESS</dt>
<dd><cite><a href="http://www.w3.org/WAI/GL/WCAG20/tests/ctprocess.html">Conformance Test Process For WCAG 2.0</a></cite>, WCAG WG, 23 February 2005, http://www.w3.org/WAI/GL/WCAG20/tests/ctprocess.html . </dd>

<dt id="WEBCGM-TEST">WEBCGM-TEST</dt>
<dd><cite><a href="http://www.cgmopen.org/resources/webcgm-test-suite.html">WebCGM Test Suite</a></cite>, OASIS CGM Open, http://www.cgmopen.org/resources/webcgm-test-suite.html . </dd>

<dt id="XML-TEST">XML-TEST</dt>
<dd><cite><a href="http://www.w3.org/XML/Test/">Extensible Markup Language (XML) Conformance Test Suites</a></cite>, W3C, NIST, http://www.w3.org/XML/Test/ . </dd>

<dt id="XQUERY-TEST">XQUERY-TEST</dt>
<dd><cite><a href="http://www.w3.org/XML/Query/test-suite/">XML Query Test Suite</a></cite>, W3C XML Query Working Group, W3C XSL Working Group, http://www.w3.org/XML/Query/test-suite/ . </dd>

</dl>

</body>
</html>