jQuery vs. My Library
in [JavaScript]
|
From: Scott Sauyet on 12 Feb 2010 12:21 Based on the SlickSpeed tests John-David Dalton recently demonstrated, I've created my own new version of SlickSpeed. It uses essentially the same timer-loop method as he did, but then calculates and reports the time per iteration in microseconds, totaling them in milliseconds so that the final report looks more like the original one: smaller is better again! :-) I've chosen to run the test repeatedly until it's taken over 250 ms, which seems to give a reasonably good balance between accuracy and performance; the original 40 selectors take about 10 seconds per library tested. There is still one flaw that Richard Cornford pointed out [1]: the loop also includes repeated calls to "new Date()". This will serve to slow down all libraries and somewhat downplay the differences between them. I'm considering ways to fix it, but for the moment it shouldn't affect rankings, only the speed ratios between the libraries. My first pass at this is at: http://scott.sauyet.com/Javascript/Test/slickspeed/2010-02-12a/ There is a link in the footer to a zip file containing the PHP code used. My raw results are below, run against recent versions of the major browsers on a powerful Windows XP machine. While this is still not a home run for My Library, it's definitely getting to be a closer contest on my developer's machine, at least with QSA. While I understand and partially agree with John-David's objections to the current My Library QSA code [2], I think it's good enough for this sort of testing at the moment, and I doubt the fixes to it will change much in the way of speed. The big news is that in these tests, in all browsers, except IE6 and IE8 in compatibility mode, My Library (QSA) was the fastest for a majority of the selectors. But in none was it the overall fastest. JQuery was the fastest in everything but IE6, where it came in third behind Dojo and MooTools. In many browsers, if two of the selectors were optimized to match the speed of the competition, My Library (QSA) would have been about the fastest overall library. Those were the two selectors with ":contains": "h1[id]:contains(Selectors)" and "p:contains(selectors)". In the various IE's there was a different issue, "p:nth-child(even/odd)" were wrong in both versions of My Library, and were significantly slower, too. One other place where My Library might be able to do a little catch-up is with some of the most commonly used selectors; jQuery is fastest, and, in some environments, significantly faster than the competition, at "tag" and "#id", which probably account for a large portion of the selectors used in daily practice. The other point to make is that we've pretty much hit the point where all the libraries are fast enough for most DOM tasks needed, and especially for querying. So although there will always be some bragging rights over fastest speed, the raw speeds are likely to become less and less relevant. In any case, nice job, David! (results below.) -- Scott ____________________ [1] http://groups.google.com/group/comp.lang.javascript/msg/44cf1a85fe8075c0 [2] http://groups.google.com/group/comp.lang.javascript/browse_thread/thread/7ee2e996c3fe952b ================================================================= Resultss on Windows XP SP2 with dual 3.0 GHz CPUs, 3.25 GB RAM ================================================================= dj = Dojo 1.4.0 jq = jQuery 1.4.1 mt = MooTools 1.2.4 ml = My Library, downloaded 2010-02-11 mlqsa = My Library with QuerySelectorAll, downloaded 2010-02-11 pr = Prototype 1.6.1 yui = Yahoo User Interface 3.0 Chrome 3.0.195.27: ------------------ dj:9 jq:6 mt:38 ml:61 mlqsa:12 pr:9 yui:12 Firefox 3.5.7: -------------- dj:24 jq:16 mt:45 ml:94 mlqsa:22 pr:32 yui:20 IE6: ---- dj:871 jq:1295 mt:1004 ml:4856 mlqsa:4815 pr:2665 yui:1731 IE8 compat: ----------- dj:188 jq:294 mt:588 ml:2926 mlqsa:1691 pr:1952 yui:707 IE8: ---- dj:81 jq:71 mt:227 ml:689 mlqsa:366 pr518: yui:142 Opera 10.10: ------------- dj:19 jq:17 mt:58 ml:340 mlqsa:261 pr:27 yui:18 Safari 4.0.4: ------------- dj:8 jq:7 mt:26 ml:42 mlqsa:9 pr:9 yui:8 =================================================================
From: Lasse Reichstein Nielsen on 12 Feb 2010 15:04 Scott Sauyet <scott.sauyet(a)gmail.com> writes: > There is still one flaw that Richard Cornford pointed out [1]: the > loop also includes repeated calls to "new Date()". This will serve to > slow down all libraries and somewhat downplay the differences between > them. I'm considering ways to fix it, but for the moment it shouldn't > affect rankings, only the speed ratios between the libraries. One approach to minimizing the impact of new Date computations is to ensure that you don't do them too often. If the run() function called in the loop is fast, it's better to do a bunch of them between creating a new date. I.e., instead of do { comps++; run(); } while ((time = (new Date() - start)) < 250); maybe do do { for (var i = 0; i < 1000; i++) { run(); } comps += 1000; } while ((time = (new Date() - start)) < 250); The exact number of rounds ofcourse depends on how long the run function takes. If a single call to run takes more than, say, 100 ms, then 250 ms, 3 rounds, isn't really enough to do statistics on anyway. Looks nice, btw. /L -- Lasse Reichstein Holst Nielsen 'Javascript frameworks is a disruptive technology'
From: Scott Sauyet on 12 Feb 2010 16:12 On Feb 12, 3:04 pm, Lasse Reichstein Nielsen <lrn.unr...(a)gmail.com> wrote: > Scott Sauyet <scott.sau...(a)gmail.com> writes: > One approach to minimizing the impact of new Date computations is to > ensure that you don't do them too often. > If the run() function called in the loop is fast, it's better to do a > bunch of them between creating a new date. I.e., instead of > > do { > comps++; > run(); > } while ((time = (new Date() - start)) < 250); > > maybe do > do { > for (var i = 0; i < 1000; i++) { > run(); > } > comps += 1000; > } while ((time = (new Date() - start)) < 250); That's one of the approaches I'm considering. But there are two concerns I have with it. First, if one of them *does* take 100 ms, then running it 1000 times will really slow down the tests. As it is, I get a reasonable average if it runs at least a dozen or so times, which happens in the vast majority of tests, but if it takes a comparatively long time, then I'm still running at least once, and it it takes that long once, chances are good that an average of a number of runs won't change it by an order of magnitude, probably much less. My second concern is that this still adds overhead to the raw iteration of the tests. I'm sure the loop is not as bad as the Date creation, but it's still something to think about. The other approach I'm thinking about runs times the equivalent loop without the selector test and then subtracts it from the time, something like this: do { comps++; run(); } while ((time = (new Date() - start)) < 250); var i =0, time2, start2 = new Date(); do { i++; time2 = (new Date() - start2); } while (i < comps); time = time - time2; This *seems* to make sense, but I'm afraid I'm missing something fundamental. I'll probably at least try this. > Looks nice, btw. Thanks. -- Scott
From: Hans-Georg Michna on 13 Feb 2010 07:42 Scott, run your tests twice. First you get a raw measurement with, say, 100 iterations. From that you extrapolate the number of repetitions you need for some desired total time and run the test again with that number. This way the new Date() calls will have no effect on your measurements, because there will be none inside the loop. To deduct the loop counter and test time, you can determine and subtract the time needed for an empty loop. Hans-Georg
From: Ivan S on 13 Feb 2010 07:55
On Feb 12, 6:21 pm, Scott Sauyet <scott.sau...(a)gmail.com> wrote: > Based on the SlickSpeed tests John-David Dalton recently demonstrated, > I've created my own new version of SlickSpeed. It uses essentially > the same timer-loop method as he did, but then calculates and reports > the time per iteration in microseconds, totaling them in milliseconds > so that the final report looks more like the original one: smaller is > better again! :-) I've chosen to run the test repeatedly until it's > taken over 250 ms, which seems to give a reasonably good balance > between accuracy and performance; the original 40 selectors take about > 10 seconds per library tested. > > There is still one flaw that Richard Cornford pointed out [1]: the > loop also includes repeated calls to "new Date()". This will serve to > slow down all libraries and somewhat downplay the differences between > them. I'm considering ways to fix it, but for the moment it shouldn't > affect rankings, only the speed ratios between the libraries. > > My first pass at this is at: > > http://scott.sauyet.com/Javascript/Test/slickspeed/2010-02-12a/ > > There is a link in the footer to a zip file containing the PHP code > used. > > My raw results are below, run against recent versions of the major > browsers on a powerful Windows XP machine. While this is still not a > home run for My Library, it's definitely getting to be a closer > contest on my developer's machine, at least with QSA. While I > understand and partially agree with John-David's objections to the > current My Library QSA code [2], I think it's good enough for this > sort of testing at the moment, and I doubt the fixes to it will change > much in the way of speed. > > The big news is that in these tests, in all browsers, except IE6 and > IE8 in compatibility mode, My Library (QSA) was the fastest for a > majority of the selectors. > > But in none was it the overall fastest. JQuery was the fastest in > everything but IE6, where it came in third behind Dojo and MooTools. > In many browsers, if two of the selectors were optimized to match the > speed of the competition, My Library (QSA) would have been about the > fastest overall library. Those were the two selectors with > ":contains": "h1[id]:contains(Selectors)" and > "p:contains(selectors)". In the various IE's there was a different > issue, "p:nth-child(even/odd)" were wrong in both versions of My > Library, and were significantly slower, too. > > One other place where My Library might be able to do a little catch-up > is with some of the most commonly used selectors; jQuery is fastest, > and, in some environments, significantly faster than the competition, > at "tag" and "#id", which probably account for a large portion of the > selectors used in daily practice. > > The other point to make is that we've pretty much hit the point where > all the libraries are fast enough for most DOM tasks needed, and > especially for querying. So although there will always be some > bragging rights over fastest speed, the raw speeds are likely to > become less and less relevant. > > In any case, nice job, David! > > (results below.) > > -- Scott > ____________________ > [1]http://groups.google.com/group/comp.lang.javascript/msg/44cf1a85fe8075c0 > [2]http://groups.google.com/group/comp.lang.javascript/browse_thread/thr.... > > ================================================================= > Resultss on Windows XP SP2 with dual 3.0 GHz CPUs, 3.25 GB RAM > ================================================================= > dj = Dojo 1.4.0 > jq = jQuery 1.4.1 > mt = MooTools 1.2.4 > ml = My Library, downloaded 2010-02-11 > mlqsa = My Library with QuerySelectorAll, downloaded 2010-02-11 > pr = Prototype 1.6.1 > yui = Yahoo User Interface 3.0 > > Chrome 3.0.195.27: > ------------------ > dj:9 jq:6 mt:38 ml:61 mlqsa:12 pr:9 yui:12 > > Firefox 3.5.7: > -------------- > dj:24 jq:16 mt:45 ml:94 mlqsa:22 pr:32 yui:20 > > IE6: > ---- > dj:871 jq:1295 mt:1004 ml:4856 mlqsa:4815 pr:2665 yui:1731 > > IE8 compat: > ----------- > dj:188 jq:294 mt:588 ml:2926 mlqsa:1691 pr:1952 yui:707 > > IE8: > ---- > dj:81 jq:71 mt:227 ml:689 mlqsa:366 pr518: yui:142 > > Opera 10.10: > ------------- > dj:19 jq:17 mt:58 ml:340 mlqsa:261 pr:27 yui:18 > > Safari 4.0.4: > ------------- > dj:8 jq:7 mt:26 ml:42 mlqsa:9 pr:9 yui:8 > ================================================================= Latest Opera (10.10) has a problem with Mylib without QSA and Dojo (all tests returned error). |