Assembler/Compiler testing
in [ASM]
|
From: Betov on 12 Mar 2006 14:54 "santosh" <santosh.k83(a)gmail.com> ?crivait news:1142186294.455607.317920 @e56g2000cwe.googlegroups.com: > Err...Betov, I think you're confusing sevagK to be Randy Hyde here. > :) Are you sure it makes some difference? They are both medical studies' cases, you know... :)) Betov. < http://rosasm.org >
From: Chewy509 on 12 Mar 2006 19:37 T933191278 wrote: > Chewy509 wrote: > > > How do you guys do testing on your assemblers and/or libraries and > > applications? > I use various methods, first i do some small tests when programing to > see if the program > works as expected, second i assemble a sample program with all the > instrucctions and i compare the output with others assemblers y also > make some test with incorrect sources to test error handling ,and third > i use this assembler (octasm) myself ,the test suite is all my programs > and also wait that others users will report bugs. Thanks... I'm doing something similar. Just wanted to know if there were any other techniques that are suitable for assembler/compiler testing... -- Darran (aka Chewy509) brought to you by Google Groups!
From: Chewy509 on 12 Mar 2006 20:20 randyhyde wrote: > Chewy509 wrote: > > Hi Guys, > > > > This one is really for assembler/compiler authors, as well as some of > > the other guys that are more evolved in testing. It's not really > > related to assembly, but more about the actual assemblers themselves. > > > > How do you guys do testing on your assemblers and/or libraries and > > applications? Do you use test suites or test harnesses, scripts with > > good/bad input and comparing to known good output, or are their other > > methods that you use, eg release often and let the users find the > > bugs**? > > > > What about things like complete regression testing, out-of-band testing > > (supplying complete garbage input to see how the > > assembler/compiler/libraries handle it), source-code auditing, formal > > validations, etc... > > Many years ago, I did a fairly complete code coverage test on HLA v1.x. > As time passed (and the source code and language changed), the test > suite became less and less useful. It got to the point where half the > code either didn't compile (because of language changes) or wasn't > testing anything useful (because of source code changes). Ultimately, I > stopped using that suite because it wasn't achieving anything (i.e., > finding defects in the code). After that point, I used a set of > specific test files for the features I was working on, but no formal > test suite for the entire assembler. > > Of course, for each release I usually (unless I'm in a hurry or forget) > compile every HLA sample source file I have on Webster to see if I've > broken anything major. While this doesn't test the *entire* language or > compiler by any stretch of the imagination, it does catch stupid stuff. > And making sure that the example programs still compile is kind of > important. If someone is reading AoA and the sample programs don't > compile, this is a bit of a problem. > > For HLA v2.x, I have a very formal regression test suite that I'm > using. Each module has two sets of test files -- one set tests code > coverage, the other checks error conditions. I compile each file and > compare its output against a version that I've checked by hand. That > way I can run these thousands of tests in an automated fashion in about > five minutes (well, so far; as the assembler gets larger I expect that > time to expand). > > The code coverage test programs tend to be rather long, trying to cover > as many paths as possible in a single source file (it is not possible > to achieve code coverage with a single source file, however, so there > are several such files per module). The code that tests error > conditions usually tests only *one* condition per source file. Once > you've had one error, any further results become suspect because of > cascading errors. It would be nice to test combinations of errors, but > you get a combinatorial explosion when you try this, and furthermore, > cascading errors with a *single* error condition often produce multiple > error messages, which change as the code is modified; maintaining > single error conditions in all these files is bad enough. > > Ultimately, code coverage testing isn't *that* good. But several books > estimate that about 50% of the bugs in common applications exist in > code that has never before executed until the bug was discovered. So if > you really do achieve code coverage (a feat in itself), you'll find > about 50% of the outstanding defects in the code (in theory, at least). Hi Randy, I'm doing something similar, just a basic script that compiles all the examples and compares to the previous version, as well as additional files that check for error/fail conditions and out-of-band syntax (eg symbols are the too large to fit into the internal buffers (more than 256 characters), nesting too many files (32 files deep), or overly complex preprocessor nesting (256 levels deep of #ifdef/#else/#endif), etc). Obvisously, like yourself each fail-condition is a separate file. But the one thing that has helped enormously, at least in my case is rewritting the compiler in it's own language... (As this has tested roughly 99% of the code path, and dealing with a cross-platform product it gets a good workout). I'm assuming your finding that experience similar with rewritting HLA v2 in HLA? The thing was, I was wondering if there was a better or alternate way of testing... At least for my purpose, there doesn't appear to be. Thanks to everyone for their feedback! :) PS. Good to see Rene is back in fine form! ;) -- Darran (aka Chewy509) brought to you by Google Groups!
From: Chewy509 on 12 Mar 2006 20:24 o//annabee wrote: > > LOL. I recompile EVERY thing in my code, maybe 100 times a day :)) > Thats maybe 1000 routines. Impressive, got a download link to your assembler? Any notable features, say above RosAsm? (I know that you're secretly conspiring against Rene, to overthrow his RosAsm empire with your own RosAsm syntax compatible assembler). -- Darran (aka Chewy509) brought to you by Google Groups!
From: o//annabee on 12 Mar 2006 20:47
P? Mon, 13 Mar 2006 02:24:26 +0100, skrev <Chewy509(a)austarnet.com.au>: > o//annabee wrote: >> >> LOL. I recompile EVERY thing in my code, maybe 100 times a day :)) >> Thats maybe 1000 routines. > > Impressive, got a download link to your assembler? Its just Randall writing to confuse. I use the RosAsm assembler to write on various hobby projects in assembly. I actually never counted my routines. But there are 12000 code labels in one of my apps. (RosAsm statistics). > Any notable features, say above RosAsm? (I know that you're secretly > conspiring against Rene, to overthrow his RosAsm empire with your own > RosAsm syntax compatible assembler). :)) Heh? What ever gave you such an idea? Here is a link to what I work on : < http://szmyggenpv.com/ > In addition, I have started rewriting now. To do all drawing to a memory buffer, manually. > -- > Darran (aka Chewy509) brought to you by Google Groups! > -- Bonnsky. |