From: David Mark on
On Jan 7, 12:20 am, Garrett Smith <dhtmlkitc...(a)gmail.com> wrote:
> David Mark wrote:
> > On Jan 6, 5:12 pm, Garrett Smith <dhtmlkitc...(a)gmail.com> wrote:
> >> David Mark wrote:
> >>> On Jan 6, 6:36 am, Matt Sach <matts...(a)gmail.com> wrote:
> >>>> On Jan 5, 10:57 pm, David Mark <dmark.cins...(a)gmail.com> wrote:
> >>>>> As for the "goals":-
> >>>>>     * Mobile is more important than Internet Explorer
>
> [...]
>
> > I think not.
>
> >> Those could all be improved by automation. Significantly
>
> >> Please don't make me click through to every example and don't make
> >> complicated test cases (like an functional/example page that has every
> >> usage of the property/method).
>
> > No clue what that means.  :)
>
> By functional test, here,  I am referring to here is a static demo page,
> like any of the Hixie or Juriy tests.
>
> Another sort of functional test is use-case functionality. Selenium is
> useful for that.
>
> Clicking each example is tiresome and doesn't give a comprehensive view
> of what works and what fails.
>
> A functional test that tries to test everything about one specific
> property is worse because it is not focused on one thing.
>
> Unit tests are the well-suited for testing browser features and dom
> features because each aspect can be tested.
>
> insertBefore suite might contain: testReturnsNode, testRefNodeNull,
> testHeirarchyRequestErr, testWrongDocumentErr, testNoModificationErr,
> testNotFoundErr.
>
> A unit test page can do those things and separate out the test cases to
> test functions, where each function tests one specific thing, using
> maybe 4-8 lines of code for each test function.

Like this:-

http://www.cinsoft.net/attributes.html

>
> >> For such a test suite, the relevance of specification is important.
>
> > Or that.
>
> Specifications should provide guidance for the outcome of assertions and
> also can help outline an API.

Okay.

>
> For example, any of the DOM APIs could be tested, ECMA specs can be
> tested. Such tests would be valuable to me. I believe they could
> potentially contribute usefull to the w3c APIs. I doubt that anyone in
> the w3c would be able to see it. I might have had better luck if I had
> shown them actual test cases. That takes a bit of time to set up. You
> can see my input to that here:
>
> http://lists.w3.org/Archives/Public/www-dom/2009JulSep/0013.html
>
> I'm no sure what the status is; I can't follow up because I was banned
> from the list.

You can't even read it? That's insane, but GG does the same thing.
Of course, you just create a new account. ;)

>
> >> Writing a testrunner will take time, but the investment will pay off
> >> with managing complexity of testing variance across browsers.
>
> > I don't need to write one.  I have what I need for now.
>
> That's what Hixie said. I still think he's missing something valuable in
> that. He doesn't (or didn't) see value in it. His acid tests, to me, are
> not valuable tools that I can refer to to see what works where, etc. His
> functional tests require a click through to each one are not even
> indexed that well. Having to click through does not provide a
> comprehensive view of what fails and what passes because you have to
> click through to each one, and in each browser, to see those results.
>
> What is valuable for me, is to see the appendChild method, see where it
> fails, and see an example of it.

Yes, I can't imagine having to click to see each result.

>
> >> Also it would be
> >> useful to have similar to Selenium `waitForCondition` (so the test can
> >> complete before the allotted expiration time).
>
> > You are getting way ahead of me.
>
> For example, say you ahve an XHR adapter. The adapter makes a call to
> the server, and when the response comes, you need to verify that response..

Right.

>
> A test case wants to test the success of a response.
>
> To test that, the test case gets a connection. Opens, sends, and then
> *waits* for the response. The `success` callback fires asynchronously;
> it cannot be tested until the xhr is complete. The problem is, we don't
> know when that will happen.

Well, I think that's simple enough.

>
> What YUI Test provides is a `wait` method. `wait` takes two arguments:
> callback, and timeout millis. So what you can do is to wait for a while.

I get it.

>
> The assertions that verify the message of the ajax call are performed in
> the `wait` callback.
>
> test case:-
>   this.wait(function(){
>     Assert.areSame("a", xhr.responseText);
>   }, 4000);
>
> I would like a `waitForCondition` method, so I could, say:-
>
>   this.waitForCondition(function(){
>     Assert.areSame("a", xhr.responseText);
>   }, condition);
>
>   function condition(){
>     xhr.complete == true;
>   }
>
> I don't really like that order, but it is consistent with the `wait`
> approach.
>
> Also, I don't like how most x unit frameworks have expected result
> first. TestNG (Java) is switched, which makes sense, but inconsistent.
>
> >> Also in YUI Test, it would be nice t ohave smaller methods. Not only
> >> would this help make YUI Test more readable, but when debugging, the
> >> developer should not have to step through a 300 line function just to
> >> get to where YUI Test has dispatched and event. Oh, it is so painful my
> >> fingers hurt just thinking about clicking debugger "step over" that many
> >> times. Ouch.
>
> > Whatever YUI is doing with their stuff is irrelevant.
>
> If you try the test runner, you might get a feel for the API and about
> testing. You'll find what you don't like and maybe you might find things
> that are valuable in it. It is a lot better than JS Unit.

I've been all through Dojo's, which I imagine is similar.

>
> The problems in the design of the YUI Test are solvable; it's not like
> he painted himself into a corner.

Who?
From: Garrett Smith on
David Mark wrote:
> On Jan 7, 12:20 am, Garrett Smith <dhtmlkitc...(a)gmail.com> wrote:
>> David Mark wrote:
>>> On Jan 6, 5:12 pm, Garrett Smith <dhtmlkitc...(a)gmail.com> wrote:
>>>> David Mark wrote:
>>>>> On Jan 6, 6:36 am, Matt Sach <matts...(a)gmail.com> wrote:
>>>>>> On Jan 5, 10:57 pm, David Mark <dmark.cins...(a)gmail.com> wrote:
[...]

> Like this:-
>
> http://www.cinsoft.net/attributes.html
>

That is a combination of an article and a series of assertions in the
source.

A test case is code only.

A separate article can refer to the test case.

You might consider breaking up the assertions into groups of tests, or
test cases. One for the wrapper, one for properties, and one for
attributes. I would probably consider putting each test case into a
separate document, so that each document is simpler.

A test case has a setup, a teardown, and one or more named test methods.
Each test function tests one specific thing and has one assertion per test.

Tests (test functions) do not affect other tests.

An HTML template can be used to prevent tests from affecting other
tests. The way it works is a setUp/tearDown "sandbox" resets the
innerHTML of the template div. This allow for each test do to anything
in there without affecting other tests. A template might not work for
your specific "attributes" test, but can be useful for most things.

<div id="template">
<a id="testLink">a</a>
</div>

var template = document.getElementById("template"),
templateHTML = template.innerHTML,
testLink;

var myTestCase = {
setUp : function(){
testLink= document.getElementById("testLink");
},

tearDown : function(){
template.innerHTML = templateHTML;
},

testGetHref : function(){
Assert.areSame(null, testLink.getAttribute("href"));
}
};

Resetting the innerHTML (in tearDown) means that test1 can do anything
inside that template without affecting test2. Tests should never affect
each other.

[...]

>> The problems in the design of the YUI Test are solvable; it's not like
>> he painted himself into a corner.
>
> Who?

Nicholas wrote that.
--
Garrett
comp.lang.javascript FAQ: http://jibbering.com/faq/
From: Garrett Smith on
David Mark wrote:
> On Jan 7, 3:46 am, David Mark <dmark.cins...(a)gmail.com> wrote:
>> On Jan 7, 1:24 am, Garrett Smith <dhtmlkitc...(a)gmail.com> wrote:
>>
>>
>>
>>> David Mark wrote:
>>>> On Jan 7, 12:20 am, Garrett Smith <dhtmlkitc...(a)gmail.com> wrote:
>>>>> David Mark wrote:
>>>>>> On Jan 6, 5:12 pm, Garrett Smith <dhtmlkitc...(a)gmail.com> wrote:
>>>>>>> David Mark wrote:
>>>>>>>> On Jan 6, 6:36 am, Matt Sach <matts...(a)gmail.com> wrote:
>>>>>>>>> On Jan 5, 10:57 pm, David Mark <dmark.cins...(a)gmail.com> wrote:
>>> [...]
>>>> Like this:-
>>>> http://www.cinsoft.net/attributes.html
>>> That is a combination of an article and a series of assertions in the
>>> source.
>>> A test case is code only.
>>> A separate article can refer to the test case.
>> You are splitting hairs. Sure it could be broken up into two pages
>> (and likely will be in the near future).
>>
You can do whatever you like, but I'm telling you that when I see that
test, it is hard to follow. The test aassertions shouldbe broken up into
test functions with one assertion per test. The test functions should be
organized into test cases. You might even want to organize suites, too,
but a good first step would be test functions and test cases.

[...]
>
> Oops, apparently that wasn't him. Must have gotten my bookmarks mixed
> up.

YUI Test does not use browser sniffing. There was one, about a year ago
or so, but he took it out after I showed him a different way. The test
runner runs in IE5.5 and Blackberry, and, despite what the author
continues to misinform about Opera mini, setTimeout does work there
(though apparently the duration is limited to 2000 ms). I have tested
Palm Pre Webkit, Opera Mini, Blackberry9000, IE5.5. The async stuff is
troubled in BB9k because of the timing strategy (should have used a
Timing manager)

YUI Test has other shortcomings.

Regardless, the aspects of YUI Test that I outlined are valuable design
decisions. I can and do point those out to Nicholas (though a good
number remain unfixed).

Separate test cases with simple test functions, setUp, tearDown, go a
long way toward making the test code easy to read and failures easy to
spot.
--
Garrett
comp.lang.javascript FAQ: http://jibbering.com/faq/
From: David Mark on
Garrett Smith wrote:
> David Mark wrote:
>> On Jan 7, 3:46 am, David Mark <dmark.cins...(a)gmail.com> wrote:
>>> On Jan 7, 1:24 am, Garrett Smith <dhtmlkitc...(a)gmail.com> wrote:
>>>
>>>
>>>
>>>> David Mark wrote:
>>>>> On Jan 7, 12:20 am, Garrett Smith <dhtmlkitc...(a)gmail.com> wrote:
>>>>>> David Mark wrote:
>>>>>>> On Jan 6, 5:12 pm, Garrett Smith <dhtmlkitc...(a)gmail.com> wrote:
>>>>>>>> David Mark wrote:
>>>>>>>>> On Jan 6, 6:36 am, Matt Sach <matts...(a)gmail.com> wrote:
>>>>>>>>>> On Jan 5, 10:57 pm, David Mark <dmark.cins...(a)gmail.com> wrote:
>>>> [...]
>>>>> Like this:-
>>>>> http://www.cinsoft.net/attributes.html
>>>> That is a combination of an article and a series of assertions in the
>>>> source.
>>>> A test case is code only.
>>>> A separate article can refer to the test case.
>>> You are splitting hairs. Sure it could be broken up into two pages
>>> (and likely will be in the near future).
>>>
> You can do whatever you like, but I'm telling you that when I see that
> test, it is hard to follow.

What are you talking about? The tests couldn't be more straightforward.

> The test aassertions shouldbe broken up into
> test functions with one assertion per test.

There is only one assertion per test.

> The test functions should be
> organized into test cases. You might even want to organize suites, too,
> but a good first step would be test functions and test cases.

There's nothing more that needs to be done to those very simple tests.
There's just nothing to them (e.g. read an attribute, assert it equals
the expected value, repeat ad nauseam).

>
> [...]
>>
>> Oops, apparently that wasn't him. Must have gotten my bookmarks mixed
>> up.
>
> YUI Test does not use browser sniffing.

Good.

> There was one, about a year ago
> or so, but he took it out after I showed him a different way.

Good.

> The test
> runner runs in IE5.5 and Blackberry, and, despite what the author
> continues to misinform about Opera mini, setTimeout does work there
> (though apparently the duration is limited to 2000 ms).

It would seem a prerequisite for tests to run in virtually any browser.
God knows, I don't need that guy to show me how to write cross-browser
scripts (asynchronous or not). ;)

> I have tested
> Palm Pre Webkit, Opera Mini, Blackberry9000, IE5.5. The async stuff is
> troubled in BB9k because of the timing strategy (should have used a
> Timing manager)

I don't find that particularly intriguing. :)

>
> YUI Test has other shortcomings.

I bet. :)

>
> Regardless, the aspects of YUI Test that I outlined are valuable design
> decisions. I can and do point those out to Nicholas (though a good
> number remain unfixed).

Why am I not surprised?

>
> Separate test cases with simple test functions, setUp, tearDown, go a
> long way toward making the test code easy to read and failures easy to
> spot.

They would only make my test code _more_ complicated. So I don't see
the point.
From: Garrett Smith on
David Mark wrote:
> Garrett Smith wrote:
>> David Mark wrote:
>>> On Jan 7, 3:46 am, David Mark <dmark.cins...(a)gmail.com> wrote:
>>>> On Jan 7, 1:24 am, Garrett Smith <dhtmlkitc...(a)gmail.com> wrote:
>>>>
>>>>
>>>>
>>>>> David Mark wrote:
>>>>>> On Jan 7, 12:20 am, Garrett Smith <dhtmlkitc...(a)gmail.com> wrote:
>>>>>>> David Mark wrote:
>>>>>>>> On Jan 6, 5:12 pm, Garrett Smith <dhtmlkitc...(a)gmail.com> wrote:
>>>>>>>>> David Mark wrote:
>>>>>>>>>> On Jan 6, 6:36 am, Matt Sach <matts...(a)gmail.com> wrote:
>>>>>>>>>>> On Jan 5, 10:57 pm, David Mark <dmark.cins...(a)gmail.com> wrote:
>>>>> [...]
>>>>>> Like this:-
>>>>>> http://www.cinsoft.net/attributes.html
>>>>> That is a combination of an article and a series of assertions in the
>>>>> source.
>>>>> A test case is code only.
>>>>> A separate article can refer to the test case.
>>>> You are splitting hairs. Sure it could be broken up into two pages
>>>> (and likely will be in the near future).
>>>>
>> You can do whatever you like, but I'm telling you that when I see that
>> test, it is hard to follow.
>
> What are you talking about? The tests couldn't be more straightforward.
>

There is no test function, just a long series of assertions.

| if (typeof assertEquals == 'function' && typeof prop == 'function') {
|
| var el, elTest, elProgress, count, set, skips, failures, errors,
|
| domSkips, domFailures, hostAttrTotals, hostPropTotals,
| wrapperAttrTotals,
|
| wrapperPropTotals, doc = window.document, html = doc.documentElement;
|
|
| var startTime = new Date().getTime();
| elTest = doc.getElementById('test1');
| elProgress = doc.getElementById('progress1')
| count = failures = skips = errors = 0;
| hostAttrTotals = { failures: 0, skips:0, errors:0 };
| set = 1;
|
| el = elTest.getElementsByTagName('a')[0];
| docWrite('<h3>Anchor<\/h3>');
|
| assertEquals(el.getAttribute('tabindex'), null,
| 'Anchor tab index missing');
| el.setAttribute('tabindex', '0');
| assertEquals(el.getAttribute('tabindex'), '0',
| 'Anchor tab index added');
|
| if (assertNotEquals(el.getAttribute('href'), null,
| 'Anchor reference present')) {
| assertEquals(el.getAttribute('href'), 'images/cinsoft.gif',
| 'Anchor reference unresolved');
} else {
|
| skipTest('Anchor reference unresolved');
}

It goes on for several hundred lines.

You have there an assertion in an if statement. Tests should never
affect other tests. Well, you don't actually have *tests*, just a mile
of assertions and if-else statements.

The code would be easier to follow if you break it up into test functions.

var uriTestCase = {
testAHref : function(){

},
testLinkHref : function() {

}

// etc.
};


MyTestRunner.addTest( uriTestCase );


When the test runner runs, it reports something like:-

testLinkHref: PASS

or

testLinkHref: FAIL: expected "foo" (string), found 12 (number).

or

testLinkHref: FAIL. ERROR. Stack trace: (line 12 object expected).

That way you can run all the tests and see what fails, and then focus on
what fails and see how it failed, then go straight to the function and
see exactly what the test function is doing.

A two line function is great for that, because that way anybody can look
at your two-liner and see what is going on straight away.

It might be hard to see the value in that for someone who has never
tried doing that.

[...]

>
>> Regardless, the aspects of YUI Test that I outlined are valuable design
>> decisions. I can and do point those out to Nicholas (though a good
>> number remain unfixed).
>
> Why am I not surprised?

Surprised at what? The fact t hat YUI Test author knows principles of
unit testing? Or surprised that things haven't been fixed?

A lot of my older tests are complicated, though, and you'll notice that
as I've gotten more experience, the test have gotten very short.

Here is an example of a good test I wrote the other day:-

| testNamespaceOther : function() {
| APE.namespace("other");
| YAHOO.util.Assert.isObject(window.other);
| }

Two lines of code and one assertion. I already made enough suggestions
for how to use that approach for testing DOM stuff. Maybe even too much.

Now if I look at tests I wrote last year, oh no. Twenty line unit test
functions. Blah. What was I thinking? Very hard to follow.

>
>> Separate test cases with simple test functions, setUp, tearDown, go a
>> long way toward making the test code easy to read and failures easy to
>> spot.
>
> They would only make my test code _more_ complicated. So I don't see
> the point.

I tried to show you what could be improved but you don't see it. You
have a long series of assertions and not contained test functions.

You probably won't see much value in testing until you get some
experience doing it.

I learned unit testing from writing JUnit and TestNG tests. I also had
pain using that JSUnit, which is a good example of what a js test
framework should not be.

I still recommend at least download and try it out. I would like to have
Nicholas fix his bugs and I would like the Timing manager added (would
greatly improve efficiency and avoid BB9k crashes), and I would like YUI
Test to not have 200 LOC methods that are so painful to "step over"
(debugger), and it should have stack traces like my hacked version does.

All shortcomings aside, there are good aspects of YUI Test that you can
learn about unit testing (so you don't have to use JUnit or TestNG).
--
Garrett
comp.lang.javascript FAQ: http://jibbering.com/faq/