HTML5 Web Browsers:Benchmarks and Market Share

HTML5 is rapidly coming of age as a)the major browser vendors move to implement more of HTML5 and b)Web developers, especially for mobiles, add more HTML5 features. So theOpenSourcery will be producing a quarterly report on the latest HTML5 Web Browser performance and market share numbers. Also commentary on Web Development related to the browsers will be added as required.

Meanwhile, there is lots happening in the world of HTML5 web browsers; so lets get right to the details. First the new browser usage reports have come out and Firefox is seeing continuing decline for the first time ever:

See details at ConceivableTech
In a vigorous  response to its small  downturn in usage, Firefox has been updated to version 5 not more than 4 months after version 4. And version 6 is due in 6-8 weeks time. Mozilla says that they are responding to competitive pressure from Google Chrome and demands in the marketplace. There are a number of  improvements most notably in PDF rendering quality and speed, improved Javascript speed and HTML5 compliance, plus the promise of major improvements in memory utilization. Finally Firefox 7, which is due before the year end, sees substantial improvements in graphics, HTML5 compliance and 3D rendering. So clearly Firefox has gotten the Chrome message and is responding with some major improvements of its own.

Opera followed suit a week later[June 28th 2011] with the significant 11.50 upgrade. Try it, Opera is now very fast and feature packed. Meanwhile,  Chrome has been showing an Agile Development schedule updating every 3-4 months as Google’s commitment to a ChromeOS. This browser based OS depends on superior performance and capabilities in Chrome the browser.

Microsoft, which just revealed a similar move to development based on the Web, will be hard pressed to retain its leisurely  once-every-18-months-to-two-years cycle of updates to IE. So IE 10 is due in 2012 with the  Windows Phone 7 and Windows 8 OS updates. But as can be seen below there is a lot of work for the IE team to catch up on HTML5 + CS3 + JavaScript – IE9 is running worst of the browsers by far in many  test categories.

Apple appears to be splitting development – more for Safari on the iDevices, less on Safari for Macs and Windows. This seems to be a growing Apple trend as Mac OS Lion still will not have multi-touch screen operations and of course many of the sensor based features of the iDevices. Also  HTML5  carries the burden of being the only cross platform development tool for iOS; so developers have to watch Apple’s compliance with Web standards extra closely.

HTML5 + CSS3 + JavaScript Tests

Given all the new browsers out, it is helpful to apply them against a range of benchmarks and tests. There are a number of HTML5  and CSS3 tests available in the table below with links provided so users can try out the tests on their browsers.

Surpringly, the organization lagging the furthest behind in providing tests is W3C. They have a poorly automated test harness for HTML5 here with 1281 tests but it only covers three of 8 major new technologies in HTML5 – audio/visual, canvas,  and some specialized HTML/HTML5 syntax. This  leaves a large swath of HTML5 capabilities completely untested by the standards setter. W3C is promising  more comprehensive HTML5 testing suite  for the 2013-2014 time frame making a mockery of the trend in development towards test first and often [and you thought Washington had a lock on budget muddling].

So developers will have to look elsewhere for a more broad sets  of HTML5 tests. But this is the old ANSI SQL  standard problem where so many exceptions and standard versions were created that no vendor met the stanadrds – and huge incompatible database silos resulted. The resulting data incompatibilities still plague IT and have given rise to new open database standards like Hadoop and its Huge database variants. The split or poorly administered standards produce a race to the bottom: Software vendors and special interest organizations then fill in the gaps with “free proprietary” extensions. This fragmentation in Web testing is then reflected with tests like ACID which are incomplete or sometimes skewed to show off a particular vendors competence.

Fortunately a few test have been developed that appear to have a)a much broader cross section of  HTML5 features tested and b)are open and subject to review of the process. For example,  HTML5test.com released on June 22nd its latest generation of HTML5 Testing. The number of indiviual tests was increased by 150 in all categories to a total of 450.  Right now HTML5Test is running  a close match with Just go to the site and let your browser show off its HTML5 prowess. HTML5 is becoming ever more important as the only cross platform glue for the proliferation of mobile devices and operating systems as Apple, the leader has thwarted all efforts to deliver cross platform  Flash, Java, and Program generatorthat all the major browser vendors

So note the following table which summarizes the HTML5and other  test results using several measures[all of which can be referenced directly]:

HTML5 Test of Latest Browsers
Test Description	Chrome	Firefox	IE 9	Opera	Safari
Browser Version	13.0.782	5.0	9.0.8	11.50	5.0.5
Acid3	100%	97%	95%	100%	100%
CanIuseJS	93%	82%	41%	63%	71%
GoogleV8.6JS	5680gm	2618gm	378gm	2266gm	1757gm
SunSpiderJS	754ms	404ms	2341ms	417ms	599ms
KrakenJS	7,956ms	12,779ms	128,262ms	20,517ms	29,057ms
CanIuseSVG	97%	88%	59%	84%	72%
GUImark2B	14.71fps	27.17fps	29.92fps	11.08fps	6.38fps
GUIMark2V	11.85fps	7.79fps	8.17fps	18.26fps	3.35fps
WebGL	100	67	NS	NS	NS
WebViz	100	129	469	NA	NA
SenchaTouch	Most	NoLoad	Blank	NoLoad	Most
W3C HTML5	93%	NA	NA	NA	NA
CanIuseHTML5	88%	78%	52%	73%	63%
HTML5Demos	58%	64%	36%	58%	55%
HTML5Test	73%+13	64%+9	31%+5	64%+7	56%+5
CanIuseCSS3	88%	88%	68%	79%	86%
FindbymeCSS3	97%	92%	70%	89%	97%
NA-Not Available,  NS-Not Supported

 

Conclusions

The bencnhmarks for compliance and speed/performance of the browsers are fairly straight forward. First the Acid 3 test results which are primarly CSS 2.x, HTML4 or less, and JavaScript/DOM oriented show that all the browsers are finally pretty compliant with the old standards. IE which had the most improvements to make  is still lowest in Acid compliance. But the improvements over IE7 and IE8 [to say nothing of IE6 that lingering Bill Gates “stunted-growth gift” to Web Developers] are major.

Compliance Tests

For the compliance with  HTML5, new DOM/JS, and CSS3 standards, one finds that  IE9 consistently scores poorest and by a wide margin. Next worst is surprisingly Safari given that Steve Jobs has declared that HTML5/CSS3/JavaScript is the future for cross platform development for mobile and PC systems. Safari via webkit does score some high compliance scores; but generally Safari trails notably. Some have raised questions of Apple’s commitment to Safari on other than iOS.

In contrast, Firefox and Opera vie for most improved with Firefox’s latest versions providing the promise of even faster improvements in compliance and performance. SVG developers will not be surprised by Opera’s good CanIuseSVG compliance and GUIMark2 Vector performance scores. And Firefox users can see that their browser consistently scores second in most compliance tests and achives top ratings for the CanIuseCSS3 and HTML5 demos test.

Chrome consistently scores highest in compliance test  taking top spot  in the HTML5Test, Sencha Touch, CanIuseHTML5, and  in both CSS3 compliance tests. With ChromeOS and ChromeBook being Web devoted systems, Google has every incentive to make their browser work well with W3C  standards being led by Googlite Ian Hickson.

Performance Tests

 

Chrome does well in the  speed/performance tests especially for JavaScript but notably less so for the Webvis, GuiMark2, and other graphics processing tests. .This is where IE is placing all its marbles – not speed of operation of IE’s JavaScript engine where it continues to trail badly all the other browsers.Rather Microsoft is placing all its emphasis on high performance in  graphics and media operations. There is talk of GPU assist for IE9 and above. Clearly IE9 scores tops in the GUIMark2 Bitmap and WebViz tests that capitalize on Directx and 3D graphics processing.

But with WebGL and WebCL rapidly  coming down the pike for Chrome, Firefox, and Safari, their performance in 3D, games and media  graphics processing will undoubtedly improve dramatically. In fact, for Firefox 7 just moving to its own new Azure engine, takes the lead in WebViz graphics performance results.  Samsung and others are making WebCL available to show the large speedup available with the open code. The result is that by Fall or  Christmas time some dandy browsers should be appearing on the market.

Summary

Clearly browser wars are on again and all the major players are investing in Web Browsers as a main interface to the Web Cloud and even in the mobile domain. Google and Microsoft and Apple are committed to browsers [or say they are in the case of Apple] for some of their major PC, Cloud, and mobile/tablet products. But clearly there is some “embrace and extend” dangers here. Witness the battles in HTML5 over Web Database, Multi-touch, Web Workers, Web Offline and Web 3D standards. Of even more concern is the Webkit only philosophy  embraced by Apple, Google, RIM, and others . Also CSS3 is seeing a proliferation of special properties for _moz, _wb, _ie etc. Finally W3C is moving slowly on HTML5 in some of the key areas noted above. Even more critical is less than Agile approach to design and testing adopted by W3C – it is reminiscent of the “Fire, Ready Aim” approach to development.

So clearly Web browser development is in for some rapid change. Consider Tom’s Hardware rating of the best browser. That honor is based on an impressive list of tests and  has flitted among several browser over the past 2 1/2 years. Currently, it is Chrome [ye Editor agrees]but that honor could change again by Fall.