From: David Mark on 7 Jan 2010 00:29 On Jan 7, 12:20 am, Garrett Smith <dhtmlkitc...(a)gmail.com> wrote: > David Mark wrote: > > On Jan 6, 5:12 pm, Garrett Smith <dhtmlkitc...(a)gmail.com> wrote: > >> David Mark wrote: > >>> On Jan 6, 6:36 am, Matt Sach <matts...(a)gmail.com> wrote: > >>>> On Jan 5, 10:57 pm, David Mark <dmark.cins...(a)gmail.com> wrote: > >>>>> As for the "goals":- > >>>>> * Mobile is more important than Internet Explorer > > [...] > > > I think not. > > >> Those could all be improved by automation. Significantly > > >> Please don't make me click through to every example and don't make > >> complicated test cases (like an functional/example page that has every > >> usage of the property/method). > > > No clue what that means. :) > > By functional test, here, I am referring to here is a static demo page, > like any of the Hixie or Juriy tests. > > Another sort of functional test is use-case functionality. Selenium is > useful for that. > > Clicking each example is tiresome and doesn't give a comprehensive view > of what works and what fails. > > A functional test that tries to test everything about one specific > property is worse because it is not focused on one thing. > > Unit tests are the well-suited for testing browser features and dom > features because each aspect can be tested. > > insertBefore suite might contain: testReturnsNode, testRefNodeNull, > testHeirarchyRequestErr, testWrongDocumentErr, testNoModificationErr, > testNotFoundErr. > > A unit test page can do those things and separate out the test cases to > test functions, where each function tests one specific thing, using > maybe 4-8 lines of code for each test function. Like this:- http://www.cinsoft.net/attributes.html > > >> For such a test suite, the relevance of specification is important. > > > Or that. > > Specifications should provide guidance for the outcome of assertions and > also can help outline an API. Okay. > > For example, any of the DOM APIs could be tested, ECMA specs can be > tested. Such tests would be valuable to me. I believe they could > potentially contribute usefull to the w3c APIs. I doubt that anyone in > the w3c would be able to see it. I might have had better luck if I had > shown them actual test cases. That takes a bit of time to set up. You > can see my input to that here: > > http://lists.w3.org/Archives/Public/www-dom/2009JulSep/0013.html > > I'm no sure what the status is; I can't follow up because I was banned > from the list. You can't even read it? That's insane, but GG does the same thing. Of course, you just create a new account. ;) > > >> Writing a testrunner will take time, but the investment will pay off > >> with managing complexity of testing variance across browsers. > > > I don't need to write one. I have what I need for now. > > That's what Hixie said. I still think he's missing something valuable in > that. He doesn't (or didn't) see value in it. His acid tests, to me, are > not valuable tools that I can refer to to see what works where, etc. His > functional tests require a click through to each one are not even > indexed that well. Having to click through does not provide a > comprehensive view of what fails and what passes because you have to > click through to each one, and in each browser, to see those results. > > What is valuable for me, is to see the appendChild method, see where it > fails, and see an example of it. Yes, I can't imagine having to click to see each result. > > >> Also it would be > >> useful to have similar to Selenium `waitForCondition` (so the test can > >> complete before the allotted expiration time). > > > You are getting way ahead of me. > > For example, say you ahve an XHR adapter. The adapter makes a call to > the server, and when the response comes, you need to verify that response.. Right. > > A test case wants to test the success of a response. > > To test that, the test case gets a connection. Opens, sends, and then > *waits* for the response. The `success` callback fires asynchronously; > it cannot be tested until the xhr is complete. The problem is, we don't > know when that will happen. Well, I think that's simple enough. > > What YUI Test provides is a `wait` method. `wait` takes two arguments: > callback, and timeout millis. So what you can do is to wait for a while. I get it. > > The assertions that verify the message of the ajax call are performed in > the `wait` callback. > > test case:- > this.wait(function(){ > Assert.areSame("a", xhr.responseText); > }, 4000); > > I would like a `waitForCondition` method, so I could, say:- > > this.waitForCondition(function(){ > Assert.areSame("a", xhr.responseText); > }, condition); > > function condition(){ > xhr.complete == true; > } > > I don't really like that order, but it is consistent with the `wait` > approach. > > Also, I don't like how most x unit frameworks have expected result > first. TestNG (Java) is switched, which makes sense, but inconsistent. > > >> Also in YUI Test, it would be nice t ohave smaller methods. Not only > >> would this help make YUI Test more readable, but when debugging, the > >> developer should not have to step through a 300 line function just to > >> get to where YUI Test has dispatched and event. Oh, it is so painful my > >> fingers hurt just thinking about clicking debugger "step over" that many > >> times. Ouch. > > > Whatever YUI is doing with their stuff is irrelevant. > > If you try the test runner, you might get a feel for the API and about > testing. You'll find what you don't like and maybe you might find things > that are valuable in it. It is a lot better than JS Unit. I've been all through Dojo's, which I imagine is similar. > > The problems in the design of the YUI Test are solvable; it's not like > he painted himself into a corner. Who?
From: Garrett Smith on 7 Jan 2010 01:24 David Mark wrote: > On Jan 7, 12:20 am, Garrett Smith <dhtmlkitc...(a)gmail.com> wrote: >> David Mark wrote: >>> On Jan 6, 5:12 pm, Garrett Smith <dhtmlkitc...(a)gmail.com> wrote: >>>> David Mark wrote: >>>>> On Jan 6, 6:36 am, Matt Sach <matts...(a)gmail.com> wrote: >>>>>> On Jan 5, 10:57 pm, David Mark <dmark.cins...(a)gmail.com> wrote: [...] > Like this:- > > http://www.cinsoft.net/attributes.html > That is a combination of an article and a series of assertions in the source. A test case is code only. A separate article can refer to the test case. You might consider breaking up the assertions into groups of tests, or test cases. One for the wrapper, one for properties, and one for attributes. I would probably consider putting each test case into a separate document, so that each document is simpler. A test case has a setup, a teardown, and one or more named test methods. Each test function tests one specific thing and has one assertion per test. Tests (test functions) do not affect other tests. An HTML template can be used to prevent tests from affecting other tests. The way it works is a setUp/tearDown "sandbox" resets the innerHTML of the template div. This allow for each test do to anything in there without affecting other tests. A template might not work for your specific "attributes" test, but can be useful for most things. <div id="template"> <a id="testLink">a</a> </div> var template = document.getElementById("template"), templateHTML = template.innerHTML, testLink; var myTestCase = { setUp : function(){ testLink= document.getElementById("testLink"); }, tearDown : function(){ template.innerHTML = templateHTML; }, testGetHref : function(){ Assert.areSame(null, testLink.getAttribute("href")); } }; Resetting the innerHTML (in tearDown) means that test1 can do anything inside that template without affecting test2. Tests should never affect each other. [...] >> The problems in the design of the YUI Test are solvable; it's not like >> he painted himself into a corner. > > Who? Nicholas wrote that. -- Garrett comp.lang.javascript FAQ: http://jibbering.com/faq/
From: Garrett Smith on 7 Jan 2010 12:43 David Mark wrote: > On Jan 7, 3:46 am, David Mark <dmark.cins...(a)gmail.com> wrote: >> On Jan 7, 1:24 am, Garrett Smith <dhtmlkitc...(a)gmail.com> wrote: >> >> >> >>> David Mark wrote: >>>> On Jan 7, 12:20 am, Garrett Smith <dhtmlkitc...(a)gmail.com> wrote: >>>>> David Mark wrote: >>>>>> On Jan 6, 5:12 pm, Garrett Smith <dhtmlkitc...(a)gmail.com> wrote: >>>>>>> David Mark wrote: >>>>>>>> On Jan 6, 6:36 am, Matt Sach <matts...(a)gmail.com> wrote: >>>>>>>>> On Jan 5, 10:57 pm, David Mark <dmark.cins...(a)gmail.com> wrote: >>> [...] >>>> Like this:- >>>> http://www.cinsoft.net/attributes.html >>> That is a combination of an article and a series of assertions in the >>> source. >>> A test case is code only. >>> A separate article can refer to the test case. >> You are splitting hairs. Sure it could be broken up into two pages >> (and likely will be in the near future). >> You can do whatever you like, but I'm telling you that when I see that test, it is hard to follow. The test aassertions shouldbe broken up into test functions with one assertion per test. The test functions should be organized into test cases. You might even want to organize suites, too, but a good first step would be test functions and test cases. [...] > > Oops, apparently that wasn't him. Must have gotten my bookmarks mixed > up. YUI Test does not use browser sniffing. There was one, about a year ago or so, but he took it out after I showed him a different way. The test runner runs in IE5.5 and Blackberry, and, despite what the author continues to misinform about Opera mini, setTimeout does work there (though apparently the duration is limited to 2000 ms). I have tested Palm Pre Webkit, Opera Mini, Blackberry9000, IE5.5. The async stuff is troubled in BB9k because of the timing strategy (should have used a Timing manager) YUI Test has other shortcomings. Regardless, the aspects of YUI Test that I outlined are valuable design decisions. I can and do point those out to Nicholas (though a good number remain unfixed). Separate test cases with simple test functions, setUp, tearDown, go a long way toward making the test code easy to read and failures easy to spot. -- Garrett comp.lang.javascript FAQ: http://jibbering.com/faq/
From: David Mark on 7 Jan 2010 13:31 Garrett Smith wrote: > David Mark wrote: >> On Jan 7, 3:46 am, David Mark <dmark.cins...(a)gmail.com> wrote: >>> On Jan 7, 1:24 am, Garrett Smith <dhtmlkitc...(a)gmail.com> wrote: >>> >>> >>> >>>> David Mark wrote: >>>>> On Jan 7, 12:20 am, Garrett Smith <dhtmlkitc...(a)gmail.com> wrote: >>>>>> David Mark wrote: >>>>>>> On Jan 6, 5:12 pm, Garrett Smith <dhtmlkitc...(a)gmail.com> wrote: >>>>>>>> David Mark wrote: >>>>>>>>> On Jan 6, 6:36 am, Matt Sach <matts...(a)gmail.com> wrote: >>>>>>>>>> On Jan 5, 10:57 pm, David Mark <dmark.cins...(a)gmail.com> wrote: >>>> [...] >>>>> Like this:- >>>>> http://www.cinsoft.net/attributes.html >>>> That is a combination of an article and a series of assertions in the >>>> source. >>>> A test case is code only. >>>> A separate article can refer to the test case. >>> You are splitting hairs. Sure it could be broken up into two pages >>> (and likely will be in the near future). >>> > You can do whatever you like, but I'm telling you that when I see that > test, it is hard to follow. What are you talking about? The tests couldn't be more straightforward. > The test aassertions shouldbe broken up into > test functions with one assertion per test. There is only one assertion per test. > The test functions should be > organized into test cases. You might even want to organize suites, too, > but a good first step would be test functions and test cases. There's nothing more that needs to be done to those very simple tests. There's just nothing to them (e.g. read an attribute, assert it equals the expected value, repeat ad nauseam). > > [...] >> >> Oops, apparently that wasn't him. Must have gotten my bookmarks mixed >> up. > > YUI Test does not use browser sniffing. Good. > There was one, about a year ago > or so, but he took it out after I showed him a different way. Good. > The test > runner runs in IE5.5 and Blackberry, and, despite what the author > continues to misinform about Opera mini, setTimeout does work there > (though apparently the duration is limited to 2000 ms). It would seem a prerequisite for tests to run in virtually any browser. God knows, I don't need that guy to show me how to write cross-browser scripts (asynchronous or not). ;) > I have tested > Palm Pre Webkit, Opera Mini, Blackberry9000, IE5.5. The async stuff is > troubled in BB9k because of the timing strategy (should have used a > Timing manager) I don't find that particularly intriguing. :) > > YUI Test has other shortcomings. I bet. :) > > Regardless, the aspects of YUI Test that I outlined are valuable design > decisions. I can and do point those out to Nicholas (though a good > number remain unfixed). Why am I not surprised? > > Separate test cases with simple test functions, setUp, tearDown, go a > long way toward making the test code easy to read and failures easy to > spot. They would only make my test code _more_ complicated. So I don't see the point.
From: Garrett Smith on 7 Jan 2010 14:15
David Mark wrote: > Garrett Smith wrote: >> David Mark wrote: >>> On Jan 7, 3:46 am, David Mark <dmark.cins...(a)gmail.com> wrote: >>>> On Jan 7, 1:24 am, Garrett Smith <dhtmlkitc...(a)gmail.com> wrote: >>>> >>>> >>>> >>>>> David Mark wrote: >>>>>> On Jan 7, 12:20 am, Garrett Smith <dhtmlkitc...(a)gmail.com> wrote: >>>>>>> David Mark wrote: >>>>>>>> On Jan 6, 5:12 pm, Garrett Smith <dhtmlkitc...(a)gmail.com> wrote: >>>>>>>>> David Mark wrote: >>>>>>>>>> On Jan 6, 6:36 am, Matt Sach <matts...(a)gmail.com> wrote: >>>>>>>>>>> On Jan 5, 10:57 pm, David Mark <dmark.cins...(a)gmail.com> wrote: >>>>> [...] >>>>>> Like this:- >>>>>> http://www.cinsoft.net/attributes.html >>>>> That is a combination of an article and a series of assertions in the >>>>> source. >>>>> A test case is code only. >>>>> A separate article can refer to the test case. >>>> You are splitting hairs. Sure it could be broken up into two pages >>>> (and likely will be in the near future). >>>> >> You can do whatever you like, but I'm telling you that when I see that >> test, it is hard to follow. > > What are you talking about? The tests couldn't be more straightforward. > There is no test function, just a long series of assertions. | if (typeof assertEquals == 'function' && typeof prop == 'function') { | | var el, elTest, elProgress, count, set, skips, failures, errors, | | domSkips, domFailures, hostAttrTotals, hostPropTotals, | wrapperAttrTotals, | | wrapperPropTotals, doc = window.document, html = doc.documentElement; | | | var startTime = new Date().getTime(); | elTest = doc.getElementById('test1'); | elProgress = doc.getElementById('progress1') | count = failures = skips = errors = 0; | hostAttrTotals = { failures: 0, skips:0, errors:0 }; | set = 1; | | el = elTest.getElementsByTagName('a')[0]; | docWrite('<h3>Anchor<\/h3>'); | | assertEquals(el.getAttribute('tabindex'), null, | 'Anchor tab index missing'); | el.setAttribute('tabindex', '0'); | assertEquals(el.getAttribute('tabindex'), '0', | 'Anchor tab index added'); | | if (assertNotEquals(el.getAttribute('href'), null, | 'Anchor reference present')) { | assertEquals(el.getAttribute('href'), 'images/cinsoft.gif', | 'Anchor reference unresolved'); } else { | | skipTest('Anchor reference unresolved'); } It goes on for several hundred lines. You have there an assertion in an if statement. Tests should never affect other tests. Well, you don't actually have *tests*, just a mile of assertions and if-else statements. The code would be easier to follow if you break it up into test functions. var uriTestCase = { testAHref : function(){ }, testLinkHref : function() { } // etc. }; MyTestRunner.addTest( uriTestCase ); When the test runner runs, it reports something like:- testLinkHref: PASS or testLinkHref: FAIL: expected "foo" (string), found 12 (number). or testLinkHref: FAIL. ERROR. Stack trace: (line 12 object expected). That way you can run all the tests and see what fails, and then focus on what fails and see how it failed, then go straight to the function and see exactly what the test function is doing. A two line function is great for that, because that way anybody can look at your two-liner and see what is going on straight away. It might be hard to see the value in that for someone who has never tried doing that. [...] > >> Regardless, the aspects of YUI Test that I outlined are valuable design >> decisions. I can and do point those out to Nicholas (though a good >> number remain unfixed). > > Why am I not surprised? Surprised at what? The fact t hat YUI Test author knows principles of unit testing? Or surprised that things haven't been fixed? A lot of my older tests are complicated, though, and you'll notice that as I've gotten more experience, the test have gotten very short. Here is an example of a good test I wrote the other day:- | testNamespaceOther : function() { | APE.namespace("other"); | YAHOO.util.Assert.isObject(window.other); | } Two lines of code and one assertion. I already made enough suggestions for how to use that approach for testing DOM stuff. Maybe even too much. Now if I look at tests I wrote last year, oh no. Twenty line unit test functions. Blah. What was I thinking? Very hard to follow. > >> Separate test cases with simple test functions, setUp, tearDown, go a >> long way toward making the test code easy to read and failures easy to >> spot. > > They would only make my test code _more_ complicated. So I don't see > the point. I tried to show you what could be improved but you don't see it. You have a long series of assertions and not contained test functions. You probably won't see much value in testing until you get some experience doing it. I learned unit testing from writing JUnit and TestNG tests. I also had pain using that JSUnit, which is a good example of what a js test framework should not be. I still recommend at least download and try it out. I would like to have Nicholas fix his bugs and I would like the Timing manager added (would greatly improve efficiency and avoid BB9k crashes), and I would like YUI Test to not have 200 LOC methods that are so painful to "step over" (debugger), and it should have stack traces like my hacked version does. All shortcomings aside, there are good aspects of YUI Test that you can learn about unit testing (so you don't have to use JUnit or TestNG). -- Garrett comp.lang.javascript FAQ: http://jibbering.com/faq/ |