how_do_we_test_a_web_browser_o.html
17.6 KB
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
139
140
141
142
143
144
145
146
147
148
149
150
151
152
153
154
155
156
157
158
159
160
161
162
163
164
165
166
167
168
169
170
171
172
173
174
175
176
177
178
179
180
181
182
183
184
185
186
187
188
189
190
191
192
193
194
195
196
197
198
199
200
201
202
203
204
205
206
207
208
209
210
211
212
213
214
215
216
217
218
219
220
221
222
223
224
225
226
227
228
229
230
231
232
233
234
235
236
237
238
239
240
241
242
243
244
245
246
247
248
249
250
251
252
253
254
255
256
257
258
259
260
261
262
263
264
265
266
267
268
269
270
271
272
273
274
275
276
277
278
279
280
281
282
283
284
285
286
287
288
289
290
291
292
293
294
295
296
297
298
299
300
301
302
303
304
305
306
307
308
309
310
311
312
313
314
315
316
317
318
319
320
321
322
323
324
325
326
327
328
329
<?xml version="1.0"?>
<!DOCTYPE html PUBLIC "-//W3C//DTD XHTML 1.0 Strict//EN"
"http://www.w3.org/TR/xhtml1/DTD/xhtml1-strict.dtd">
<html xmlns="http://www.w3.org/1999/xhtml" xml:lang="en" lang="en">
<head>
<meta http-equiv="Content-Type" content="text/html; charset=utf-8" />
<style type="text/css" media="all">
@import "/QA/2006/01/blogstyle.css";
</style>
<meta name="keywords" content='css, html, svg, test, test suite' />
<meta name="description" content="The situation with regards to testing at W3C is improving a bit but is still far from ideal. Various groups are different ways to test implementations and are all lacking resources to improve their test suites. We need your help now to build the next open Web platform and make HTML5 a real success!" />
<meta name="revision" content="$Id: how_do_we_test_a_web_browser_o.html,v 1.31 2011/12/15 23:20:59 mirror Exp $" />
<link rel="alternate" type="application/atom+xml" title="Atom" href="http://www.w3.org/QA/atom.xml" />
<link rel="alternate" type="application/rss+xml" title="RSS 1.0" href="http://www.w3.org/QA/news.rss" />
<title>How do we test a Web browser? (one year after) - W3C Blog</title>
<link rel="start" href="http://www.w3.org/QA/" title="Home" />
<link rel="prev" href="http://www.w3.org/QA/2010/09/html5_getting_to_last_call.html" title="HTML5: Getting to Last Call" />
<link rel="next" href="http://www.w3.org/QA/2010/09/one_web_day_and_w3c_community.html" title="One Web Day and W3C Community Groups" />
<!--
<rdf:RDF xmlns:rdf="http://www.w3.org/1999/02/22-rdf-syntax-ns#"
xmlns:trackback="http://madskills.com/public/xml/rss/module/trackback/"
xmlns:dc="http://purl.org/dc/elements/1.1/">
<rdf:Description
rdf:about="http://www.w3.org/QA/2010/09/how_do_we_test_a_web_browser_o.html"
trackback:ping="http://www.w3.org/QA/sununga/mt-tb.cgi/350"
dc:title="How do we test a Web browser? (one year after)"
dc:identifier="http://www.w3.org/QA/2010/09/how_do_we_test_a_web_browser_o.html"
dc:subject="HTML"
dc:description="The situation with regards to testing at W3C is improving a bit but is still far from ideal. Various groups are different ways to test implementations and are all lacking resources to improve their test suites. We need your help now to build the next open Web platform and make HTML5 a real success!"
dc:creator="Philippe Le Hégaret"
dc:date="2010-09-15T13:06:00+00:00" />
</rdf:RDF>
-->
<!-- <script type="text/javascript" src="http://www.w3.org/QA/mt.js"></script>-->
</head>
<body class="layout-one-column">
<div id="banner">
<h1 id="title">
<a href="http://www.w3.org/"><img height="48" alt="W3C" id="logo" src="http://www.w3.org/Icons/WWW/w3c_home_nb" /></a>
W3C Blog
</h1>
</div>
<ul class="navbar" id="menu">
<li><strong><a href="/QA/" title="W3C Blog Home">[ W3C Blog ]</a></strong></li>
<li><a href="/QA/Library/" title="Documents and Publications on Web and Quality">Documents</a></li>
<li><a href="/QA/Tools/" accesskey="3" title="Validators and other Tools">Tools</a></li>
<li><a href="/2007/12/qa-blog-help/index#feedback">Feedback</a></li>
</ul>
<div id="searchbox">
<form method="get" action="http://www.google.com/custom" enctype="application/x-www-form-urlencoded">
<p id="formbox"><input type="text" size="15" class="textfield" name="q" accesskey="E" maxlength="255" /> <input type="submit" class="submitfield" value="Search" id="goButton" name="sa" accesskey="G" /> <input type="hidden" name="cof" value="T:black;LW:72;ALC:#ff3300;L:http://www.w3.org/Icons/w3c_home;LC:#000099;LH:48;BGC:white;AH:left;VLC:#660066;GL:0;AWFID:0b9847e42caf283e;" /><input type="hidden" id="searchW3C" name="sitesearch" checked="checked" value="www.w3.org/QA" /><input type="hidden" name="domains" value="www.w3.org/QA" /></p>
</form>
</div>
<div id="main"><!-- This DIV encapsulates everything in this page - necessary for the positioning -->
<p class="content-nav">
<a href="http://www.w3.org/QA/2010/09/html5_getting_to_last_call.html">« HTML5: Getting to Last Call</a> |
<a href="http://www.w3.org/QA/">Main</a>
| <a href="http://www.w3.org/QA/2010/09/one_web_day_and_w3c_community.html">One Web Day and W3C Community Groups »</a>
</p>
<h2 class="entry-header">How do we test a Web browser? (one year after)</h2>
<div class="entry-body">
<p>
Almost one year ago, I wrote about browser testing in <cite><a href='http://www.w3.org/QA/2009/09/how_do_we_test_a_web_browser.html'>How do we test a Web browser?</a></cite>. Since then, the situation improved a bit but we're still far from reaching an appropriate comfort level.
</p>
<p>
There is now a new mercurial server and youl'll find tests related to <a href='http://dvcs.w3.org/hg/html/'>HTML5</a> or <a href='http://dvcs.w3.org/hg/webapps/'>Web Applications</a>. All of those tests are automatically mirrored on <a href='http://test.w3.org/'>test.w3.org</a>. The <a href='http://test.w3.org/m/'>Mobile tests</a> are also linked from there. The <a href='http://www.w3.org/Style/CSS/Test/'>CSS</a> and <a href="http://www.w3.org/Math/testsuite/">MathML</a> test suites are still in a separate space. The <a href='http://www.w3.org/Graphics/SVG/WG/wiki/Test_Suite_Overview'>SVG Group</a> and <a href="http://www.w3.org/International/tests/">Internationalization effort</a> are in a transition phase.
</p>
<p>
A few of our Groups have documented how to contribute tests:
</p>
<ul>
<li><a href='http://wiki.csswg.org/test'>CSS Working Group</a></li>
<li><a href='http://www.w3.org/html/wg/wiki/Testing/Submission/'>HTML Working Group</a></li>
<li><a href='http://dev.w3.org/SVG/docs/SVGTestSuite-howto.html'>SVG Working Group</a></li>
</ul>
<p>The Internationalization effort has been reporting on test results for while, such as for <a href='http://www.w3.org/International/tests/tests-html-css/tests-language-declarations/results-language-declarations#results'>language declarations</a>.</p>
<p>
The HTML Working Group, working on HTML5, also started to publish <a href='http://test.w3.org/html/tests/reporting/report.htm'>ongoing test results</a>. Thanks to a few contributions, we started to test a some features, including video and canvas:
</p>
<table summary='This table indicates the results for the HTML5 test suite. The f
irst column is the test category.'>
<caption>HTML5 Test Suite results</caption>
<thead>
<tr><th>Feature</th><th>Internet Explorer 9 Platform Preview 4</th><th>Chrome 5.0.375.125</th><th>Firefox 4 Beta 2</th><th>Opera 10.60</th><th>Safari 5.0.7533.16</th></tr>
</thead>
<tbody>
<tr><th>attributes</th><td>100%</td><td>100%</td><td>100%</td><td>100%</td><td>100%</td></tr>
<tr><th>audio</th><td>100%</td><td>100%</td><td>100%</td><td>100%</td><td>100%</td></tr>
<tr><th>canvas</th><td>85%</td><td>86.67%</td><td>76.67%</td><td>85%</td><td>88.33%</td></tr>
<tr><th>foreigncontent</th><td>100%</td><td>10%</td><td>100%</td><td>10%</td><td>10%</td></tr>
<tr><th>video</th><td>84.62%</td><td>69.23%</td><td>61.54%</td><td>61.54%</td><td>61.54%</td></tr>
<tr><th>xhtml5</th><td>100%</td><td>22.22%</td><td>66.67%</td><td>100%</td><td>33.33%</td></tr>
</tbody></table>
<p>
The HTML test suite only contains 97 approved tests for the moment so don't draw too many conclusion from the result table. The number of tests needs to increase significantly if we want to test HTML5 properly. Around 900 tests are waiting to be approved within the task force but we're lacking participants. Help in identifying more test sets and submitting them to the group would also be appreciated.
</p>
<p>
<strong>The Web browsers of tomorrow are being developed and tested today, so don't wait and help us make the Web a better place!</strong> So pick your favorite HTML, CSS, SVG, MathML, or API feature, write as many tests as you can on it, and submit those tests to us. Before sending bug reports to various browser vendors, that's your best chance to get your favorite feature properly implemented and the browser developers will even thank you.
</p>
</div>
<div id="more" class="entry-more">
</div>
<p class="postinfo">Filed by <a href="http://www.w3.org/People/LeHegaret/">Philippe Le Hégaret</a> on September 15, 2010 1:06 PM in <a href="http://www.w3.org/QA/archive/technology/css/">CSS</a>, <a href="http://www.w3.org/QA/archive/technology/html/">HTML</a>, <a href="http://www.w3.org/QA/archive/technology/svg/">SVG</a><br />
<span class="separator">|</span> <a class="permalink" href="http://www.w3.org/QA/2010/09/how_do_we_test_a_web_browser_o.html">Permalink</a>
| <a href="http://www.w3.org/QA/2010/09/how_do_we_test_a_web_browser_o.html#comments">Comments (9)</a>
| <a href="http://www.w3.org/QA/2010/09/how_do_we_test_a_web_browser_o.html#trackback">TrackBacks (0)</a>
</p>
<h3 class="comments-header" id="comments">Comments</h3>
<div class="comment" id="comment-200829">
<p class="comment-meta" id="c200829">
<span class="comment-meta-author"><strong>monty </strong></span>
<span class="comment-meta-date"><a href="#c200829">#</a> 2010-09-18</span>
</p>
<div class="comment-bulk">
<p>Why are you comparing unstable versions (IE and Firefox) with stable releases (Chrome, Opera, Safari)? There's a some new HTML5 releated features in both Chrome 7 and Opera 10.70.</p>
</div>
</div>
<div class="comment" id="comment-200840">
<p class="comment-meta" id="c200840">
<span class="comment-meta-author"><strong>Philippe Le Hegaret </strong></span>
<span class="comment-meta-date"><a href="#c200840">#</a> 2010-09-18</span>
</p>
<div class="comment-bulk">
<p>Hi Monty,</p>
<p>
Given the low level of resources we have in the HTML testing group, I take what I can get in terms of test results at the moment. Microsoft and Opera are actually the only two browser vendors active in the effort, and Opera didn't judge necessary to update the results so far. They're both active at updating the test harness, which is more important than updating test results for the moment. Join the effort and you could help us maintaining more comprehensive results.
</p>
</div>
</div>
<div class="comment" id="comment-200869">
<p class="comment-meta" id="c200869">
<span class="comment-meta-author"><strong>Ms2ger </strong></span>
<span class="comment-meta-date"><a href="#c200869">#</a> 2010-09-19</span>
</p>
<div class="comment-bulk">
<p>Philippe, it isn't helpful to post results of running a very small number of not at all representative tests (most of which were written by Microsoft), especially as they aren't very high quality. We are expecting hundreds of thousands of tests to be submitted, and I hope they will be more useful than the ones we have seen so far.</p>
</div>
</div>
<div class="comment" id="comment-200881">
<p class="comment-meta" id="c200881">
<span class="comment-meta-author"><strong>Philippe Le Hegaret </strong></span>
<span class="comment-meta-date"><a href="#c200881">#</a> 2010-09-19</span>
</p>
<div class="comment-bulk">
<p>Hi Ms2ger,</p>
<p>I want to bring attention to the fact that W3C will publish ongoing test results on Web technologies, especially HTML5. In addition, a good fraction of those tests were not written by Microsoft: Philipp Taylor wrote the canvas tests for example (and we still need to go through most of them). Regarding the usefulness or quality of the tests, I wouldn't be so affirmative: some of tests were written at the very beginning and the test harness is still evolving. So it's not a surprise that they aged quickly and we already thinking about rewriting them. Like you, I do hope that we'll have more tests and that's why I want to bring attention to the effort. </p>
</div>
</div>
<div class="comment" id="comment-201783">
<p class="comment-meta" id="c201783">
<span class="comment-meta-author"><strong>Javaer </strong></span>
<span class="comment-meta-date"><a href="#c201783">#</a> 2010-09-30</span>
</p>
<div class="comment-bulk">
<p>Im wnt to show a button in a html.in the Firefox or IE is no problem,but in Safari not showed.
why?</p>
</div>
</div>
<div class="comment" id="comment-201808">
<p class="comment-meta" id="c201808">
<span class="comment-meta-author"><strong>Philippe Le Hégaret <a class="commenter-profile" href="http://www.w3.org/People/LeHegaret/"><img alt="Author Profile Page" src="http://www.w3.org/QA/sununga/mt-static/images/comment/mt_logo.png" width="16" height="16" /></a></strong></span>
<span class="comment-meta-date"><a href="#c201808">#</a> 2010-09-30</span>
</p>
<div class="comment-bulk">
<p>Hi Javaer,</p>
<p>If you're willing to write a test that contains the bug you're running into, I'm willing to take your test and submit to the HTML or CSS Working Groups.</p>
</div>
</div>
<div class="comment" id="comment-204549">
<p class="comment-meta" id="c204549">
<span class="comment-meta-author"><strong>Chris </strong></span>
<span class="comment-meta-date"><a href="#c204549">#</a> 2010-10-29</span>
</p>
<div class="comment-bulk">
<p>I have written a CSS3 test which shows a rounding error with background-size in Opera:
<a href="http://winner.cspsx.de/background-size-rounding.zip" rel="nofollow">http://winner.cspsx.de/background-size-rounding.zip</a></p>
</div>
</div>
<div class="comment" id="comment-205165">
<p class="comment-meta" id="c205165">
<span class="comment-meta-author"><strong>Igor </strong></span>
<span class="comment-meta-date"><a href="#c205165">#</a> 2010-11-04</span>
</p>
<div class="comment-bulk">
<p>Chrome 5? Current stable is 7.0</p>
</div>
</div>
<div class="comment" id="comment-205166">
<p class="comment-meta" id="c205166">
<span class="comment-meta-author"><strong>Philippe Le Hegaret </strong></span>
<span class="comment-meta-date"><a href="#c205166">#</a> 2010-11-04</span>
</p>
<div class="comment-bulk">
<p>
@Chris, I forwarded your test to the editor of the CSS border and background editor. He'll have a look at it and see if it's relevant for them.
</p>
<p>
@Igor, other folks have noticed that and submitted test results for Chrome 7. The table in this blog entry is static but we're maintaining the one in the testing report.
</p>
</div>
</div>
<div class="comments-open" id="comments-open">
<h3 class="comments-open-header">Leave a comment</h3>
<div class="comments-open-moderated">
<p>
Note: this blog is intended to foster <strong>polite
on-topic discussions</strong>. Comments failing these
requirements and spam will not get published. Please,
enter your real name and email address. Every
individual comment is reviewed by the W3C staff.
This may take some time, thank you for your patience.
</p>
<p>
You can use the following HTML markup (a href, b, i,
br/, p, strong, em, ul, ol, li, blockquote, pre)
and/or <a href="http://daringfireball.net/projects/markdown/syntax">Markdown syntax</a>.</p>
</div>
<div id="comments-open-data">
<form method="post" action="http://www.w3.org/QA/sununga/beach.pl" id="comments-form">
<h4>Your comment</h4>
<div id="comments-open-text">
<textarea id="comment-text" name="text" rows="20" cols="100"></textarea><br />
<label for="comment-text">Write your comment text here. Remember, keep the discussion on topic and courteous.</label>
</div>
<h4>About you</h4>
<div id="comment-form-name">
<input type="hidden" name="static" value="1" />
<input type="hidden" name="entry_id" value="8896" />
<input type="hidden" name="__lang" value="en" />
<label for="comment-author">Your Name</label>
<input id="comment-author" name="author" size="30" value="" />
</div>
<div id="comment-form-email">
<label for="comment-email">Your Email Address</label>
<input id="comment-email" name="email" size="30" value="" />
</div>
<div id="comments-open-footer">
<input type="submit" accesskey="s" name="post" id="comment-submit" value="Submit" />
</div>
</form>
</div>
</div>
<p id="gentime">This page was last generated on $Date: 2011/12/15 23:20:59 $</p>
</div><!-- End of "main" DIV. -->
<address>
This blog is written by W3C staff and working group participants,<br />
and maintained by <a href="/People/CMercier/">Coralie Mercier</a>.<br />
Authorized parties may <a href="/QA/new">log in</a> to create a new entry.<br/>
<span id="poweredby">Powered by Movable Type, magpierss and a lot of Web Technology</span>
</address>
<p class="copyright">
<a rel="Copyright" href="http://www.w3.org/Consortium/Legal/ipr-notice#Copyright">Copyright</a> © 1994-2011
<a href="http://www.w3.org/"><acronym title="World Wide Web Consortium">W3C</acronym></a>®
(<a href="http://www.csail.mit.edu/"><acronym title="Massachusetts Institute of Technology">MIT</acronym></a>,
<a href="http://www.ercim.eu/"><acronym title="European Research Consortium for Informatics and Mathematics">ERCIM</acronym></a>,
<a href="http://www.keio.ac.jp/">Keio</a>),
All Rights Reserved.
W3C <a href="http://www.w3.org/Consortium/Legal/ipr-notice#Legal_Disclaimer">liability</a>,
<a href="http://www.w3.org/Consortium/Legal/ipr-notice#W3C_Trademarks">trademark</a>,
<a rel="Copyright" href="http://www.w3.org/Consortium/Legal/copyright-documents">document use</a>
and <a rel="Copyright" href="http://www.w3.org/Consortium/Legal/copyright-software">software licensing</a>
rules apply. Your interactions with this site are in accordance
with our <a href="http://www.w3.org/Consortium/Legal/privacy-statement#Public">public</a> and
<a href="http://www.w3.org/Consortium/Legal/privacy-statement#Members">Member</a> privacy
statements.
</p>
</body>
</html>