Prev: Presentation of a new native JavaScript database
Next: wordpress and modal commentary window
From: Scott Sauyet on 15 Feb 2010 15:56
On Feb 14, 11:45 pm, David Mark <dmark.cins...(a)gmail.com> wrote:
> David Mark wrote:
>> I've updated the TaskSpeed test functions to improve performance.  This
>> necessitated some minor additions (and one change) to the OO interface
>> as well.  I am pretty happy with the interface at this point, so will
>> set about properly documenting it in the near future.

So you've learned that test-driven development is not an
oxymoron? :-)

I actually am a fan of test-driven design, but I don't do it with
performance tests; that scares me.

>> [http://www.cinsoft.net/taskspeed.html]

> Opera 10.10, Windows XP on a very busy and older PC:-
>
> 2121     18624   9000    5172    22248   4846    4360    1109    1266    1189  
> 6140     1876    843*    798*
>
> I ran it a few times.  This is representative.  The two versions flip
> flop randomly.  Usually around a third of the purer tests.  :)

I can confirm similar rankings (with generally faster speeds, of
course) on my modern machine in most recent browsers, with two
exceptions: First, in Firefox and IE, PureDOM was faster than My
Library. Second, in IE6, several tests fail in My Library ("setHTML",
"insertBefore", "insertAfter".) Also note that the flip-flopping of
the two versions might have to do with the fact that they are pointing
at the same exact version of My Library (the one with QSA) and the
same test code. You're running the same infrastructure twice! :-)

This is great work, David. I'm very impressed.

But I do have some significant caveats. Some of the tests seem to me
to be cheating, especially when it comes to the loops. For instance,
here is one of the functions specifications:

"append" : function(){
// in a 500 iteration loop:
// create a new <div> with the same critera as 'create'
// - NOTE: rel needs to be == "foo2"
// then append to body element (no caching)
//
// return then length of the matches from the selector
"div[rel^='foo2']"
},

My Library's implementation looks like this:

"append" : function() {
var myEl = E().loadNew('div', { 'rel':'foo2' }), body =
document.body;
for (var i = 500; i--;) {
myEl.loadClone().appendTo(body);
}
return $("div[rel^=foo2]").length;
},

This definitely involves caching some objects outside the loop. There
are a number of such instances of this among the test cases. My
Library is not alone in this, but most of the other libraries mainly
just have variable declarations outside the loop, not initialization.
In part, this is a problem with the Task Speed design and
specification. It would have been much better to have the testing
loop run each library's tests the appropriate number of times rather
than including the loop count in the specification. But that error
should not be an invitation to abuse. I ran a version with such
initializations moved inside the loop, and my tests average about a
15% performance drop for My Library, in all browsers but Opera, where
it made no significant difference.

But that is just one instance of a general problem. My Library is not
alone in coding its tests to the performance metrics. The spec has
this:

"bind" : function(){
// connect onclick to every first child li of ever ul
(suggested: "ul > li")
//
// return the length of the connected nodes
},

but the YUI3 tests perform this with event delegation:

"bind" : function(){
Y.one('body').delegate('click', function() {}, 'ul > li');
return Y.all('ul > li').size();
},

This might well be the suggested way to attach a behavior to a number
of elements in YUI. There's much to be said for doing it in this
manner. And yet it is pretty clearly not what was intended in the
specification; if nothing else, it's an avoidance of what was
presumably intended to be a loop. There's a real question of doing
things the appropriate way.

To test this, I limited myself to 15 minutes of trying to optimize the
jQuery tests in the same manner. I moved initialization outside the
loop and switched to event delegation. After this brief attempt, I
achieved speed gains between 54% and 169% in the various browsers.
And I did this without any changes to the underlying library. I'm
sure I could gain reasonable amounts of speed in some of the other
libraries as well, but this sort of manipulation is wrong-headed.

Perhaps an updated version of TaskSpeed is in order, but it's hard to
design a system that can't be gamed in this manner.

Does your host have PHP? It would suggest it would be better to host
a dynamic version of this, and not rely on static files. It's easy to
set up, and that also makes it almost trivial to add and remove
libraries from your tests.

Finally, the matter of IE6 is disappointing. This is still a widely
used browser; I'm surprised you didn't test there before releasing the
code. You've often pointed out how well My Library performed without
change when IE8 came out. Well the flip side is that it needs to keep
doing well at least in environments that are widely used, even as you
make changes. All the other libraries except qoodoox did fine in IE6,
even if all of them were ungodly slow.

So, overall, I'd give this a B+. It's solid work, but there are some
significant issues that still need to be resolved.

Cheers,

-- Scott

From: Richard Cornford on 15 Feb 2010 19:21
Scott Sauyet wrote:
<snip>
> "bind" : function(){
> // connect onclick to every first child li of ever ul
>(suggested: "ul > li")
<snip> ^^^^^^^

Surely that is not the CSS selector for "every first child li of ever
ul". The specs for these tests are really bad, so it is not surprising
that the implementations of the tests are all over the place.

> // return the length of the connected nodes

You see, what is "the length of the connected nodes"? Is that "length"
in terms of something like the pixel widths/height of the displayed
nodes, or is "length" intended to imply the length of some sort of
'collected' array of nodes (i.e. some sort of 'query', and if so why are
we timing that in a context that is not a test of 'selector'
performance), or does the spec just call for *number* of nodes modified
by the task to be returned?

Richard.

From: Peter Michaux on 15 Feb 2010 20:22
On Feb 15, 12:19 am, Cody Haines <chaine...(a)gmail.com> wrote:

> P.S. I'm sending this from an iPhone app, so apologies if the formatting
> is screwy

Which app?

Peter
From: Scott Sauyet on 15 Feb 2010 23:20
On Feb 15, 7:21 pm, "Richard Cornford" <Rich...(a)litotes.demon.co.uk>
wrote:
> Scott Sauyet wrote:
>
> <snip>>    "bind" : function(){
> >        //  connect onclick to every first child li of ever ul
> >(suggested: "ul > li")
>
> <snip>        ^^^^^^^
>
> Surely that is not the CSS selector for "every first child li of ever
> ul". The specs for these tests are really bad, so it is not surprising
> that the implementations of the tests are all over the place.

And of course, if you do what's suggested by the text, you'll look
wrong, because all the other libraries are taking the "ul > li"
literally. I meant to mention this too in my last posting. There is
definitely some room for improvement to the test specs as well as to
the infrastructure.

> >         //  return the length of the connected nodes
>
> You see, what is "the length of the connected nodes"? Is that "length"
> in terms of something like the pixel widths/height of the displayed
> nodes, or is "length" intended to imply the length of some sort of
> 'collected' array of nodes (i.e. some sort of 'query', and if so why are
> we timing that in a context that is not a test of 'selector'
> performance), or does the spec just call for *number* of nodes modified
> by the task to be returned?

This one does not bother me too much. In the context of the
collection of tests [1], it's fairly clear that they want the number
of nodes modified. Presumably they assume that the tools will collect
the nodes in some array-like structure with a "length" property.


-- Scott
___________________
[1] http://dante.dojotoolkit.org/taskspeed/tests/sample-tests.js
From: Cody Haines on 16 Feb 2010 01:08
Peter Michaux <petermichaux(a)gmail.com> wrote:
> On Feb 15, 12:19 am, Cody Haines <chaine...(a)gmail.com> wrote:
>
> > P.S. I'm sending this from an iPhone app, so apologies if the
> > formatting
> > is screwy
>
> Which app?
>
> Peter
>

NewsTap
--
--Cody Haines
First  |  Prev  |  Next  |  Last
Pages: 1 2 3 4 5 6 7 8
Prev: Presentation of a new native JavaScript database
Next: wordpress and modal commentary window