Prev: ActionController::AbstractRequest.relative_url_root = "/rotspecs"
Next: Now Win a laptop computer from (EZ laptop) free of charge
From: Yotta Meter on 22 Feb 2010 23:04
With the following example:

#---------------------------------------------------------
#!/usr/bin/ruby

require "test/unit"

class TestImplArchFile < Test::Unit::TestCase

def test_loop

[0,1,2].each do |i|
assert_equal(i, 6, "Duh, #{i} is not equal to 6.")
end
end

end
#---------------------------------------------------------
I get the following output:
#---------------------------------------------------------

Loaded suite ./t
Started
F
Finished in 0.00527 seconds.

1) Failure:
test_loop(TestImplArchFile)
[./t.rb:10:in `test_loop'
./t.rb:9:in `each'
./t.rb:9:in `test_loop']:
Duh, 0 is not equal to 6.
<0> expected but was
<6>.
#---------------------------------------------------------

Is there any way I can see three errors, ie the exception
would create a failure, but not exit the def? I'd like to see
all errors in the loop. I'm not sure
#---------------------------------------------------------

1) Failure:
test_loop(TestImplArchFile)
[./t.rb:10:in `test_loop'
./t.rb:9:in `each'
./t.rb:9:in `test_loop']:
Duh, 0 is not equal to 6.
<0> expected but was
<6>.

2) Failure:
test_loop(TestImplArchFile)
[./t.rb:10:in `test_loop'
./t.rb:9:in `each'
./t.rb:9:in `test_loop']:
Duh, 1 is not equal to 6.
<1> expected but was
<6>.

3) Failure:
test_loop(TestImplArchFile)
[./t.rb:10:in `test_loop'
./t.rb:9:in `each'
./t.rb:9:in `test_loop']:
Duh, 2 is not equal to 6.
<2> expected but was
<6>.
--
Posted via http://www.ruby-forum.com/.

From: Brian Candler on 23 Feb 2010 11:32
Yotta Meter wrote:
> Is there any way I can see three errors, ie the exception
> would create a failure, but not exit the def?

No, I don't think so. A failed assertion raises AssertionFailedError
which terminates your method. There are no restarts in Ruby.

You can accumulate the errors yourself:

errs = []
[0,1,2].each do |i|
errs << "Duh, #{i} is not equal to 6" unless i == 6
end
assert errs.empty?, errs.join("; ")

You can use a bit of metaprogramming to create three separate test
methods in a loop.

require 'test/unit'
class TestBase < Test::Unit::TestCase
(0..2).each do |i|
define_method("test_loop#{i}") do
assert_equal 6, i, "Duh, #{i} is not equal to 6."
end
end
end

Or for fun, even three separate test classes:

require 'test/unit'
class TestBase < Test::Unit::TestCase
CHECK = 0
def test_it
assert_equal 6, self.class::CHECK, "Duh, #{self.class::CHECK} !=
6"
end
end

klasses = []
(1..2).each do |n|
klass = Class.new(TestBase)
klass.const_set(:CHECK, n)
klasses << klass # for GC
end

That's not very pretty though.
--
Posted via http://www.ruby-forum.com/.

From: Yotta Meter on 23 Feb 2010 12:04
This is really the great idea I was looking for, thanks. Obviously I'm
not comparing integers, I'm actually iterating over verilog modules in
an ASIC, and do a test for each verilog module. Most verilog modules
will pass, but there will be a few that don't, so I'd like the test to
go through all verilog modules.

Another question, is my philosophy off? I'm a little confused by the
amount of test options out there, rspec, cucumber, etc. Is there another
testing strategy I should be working with in this instance where I want
to iterate through an array of complex objects?

Thanks for not telling me to rtfm or google it.


>
> require 'test/unit'
> class TestBase < Test::Unit::TestCase
> (0..2).each do |i|
> define_method("test_loop#{i}") do
> assert_equal 6, i, "Duh, #{i} is not equal to 6."
> end
> end
> end
>
--
Posted via http://www.ruby-forum.com/.

From: Brian Candler on 23 Feb 2010 13:32
Yotta Meter wrote:
> Another question, is my philosophy off? I'm a little confused by the
> amount of test options out there, rspec, cucumber, etc.

Well, they're apples and oranges to some extent. rspec is similar in
concept to Test::Unit but with a very different syntax (which I
personally dislike, so I stick to Test::Unit). But as far as I know,
assertion failures in rspec also raise exceptions and so can't continue.
There are a whole bunch of alternative test frameworks out there, but I
don't know enough about any of them to say whether they would fit your
needs better.

Cucumber works at a different layer. It sits on top of either rspec or
test/unit and lets you write tests in plain language, using regexps to
match that plain language and turn it into actual test code. But again,
if one step fails within a scenario, the remaining steps are skipped.

> Is there another
> testing strategy I should be working with in this instance where I want
> to iterate through an array of complex objects?

Apart from the ideas I gave before, you could try rescuing
Test::Unit::AssertionFailedError explicitly inside your loop, to allow
it to continue (but making a note of the failure).

Indeed, the core code for Test::Unit which handles this looks very
simple:

# Runs the individual test method represented by this
# instance of the fixture, collecting statistics, failures
# and errors in result.
def run(result)
yield(STARTED, name)
@_result = result
begin
setup
__send__(@method_name)
rescue AssertionFailedError => e
add_failure(e.message, e.backtrace)
rescue Exception
raise if PASSTHROUGH_EXCEPTIONS.include? $!.class
add_error($!)
ensure
begin
teardown
rescue AssertionFailedError => e
add_failure(e.message, e.backtrace)
rescue Exception
raise if PASSTHROUGH_EXCEPTIONS.include? $!.class
add_error($!)
end
end
result.add_run
yield(FINISHED, name)
end

So it looks like you could just call add_failure yourself and continue.
This is a private method/undocumented API, but I'd happily do that if it
gets the job done.

Here's a proof-of-concept:

require "test/unit"

class TestImplArchFile < Test::Unit::TestCase

def no_stop
yield
rescue Test::Unit::AssertionFailedError => e
add_failure(e.message, e.backtrace)
rescue Exception
raise if PASSTHROUGH_EXCEPTIONS.include? $!.class
add_error($!)
end

def test_loop
[0,1,2].each do |i|
no_stop do
assert_equal(i, 6, "Duh, #{i} is not equal to 6.")
end
end
end

end
--
Posted via http://www.ruby-forum.com/.

From: Yotta Meter on 23 Feb 2010 14:20
I'm seeing some performance issues I can't seem to get around. Your
solution for creating a method was great though. I'd try the last one,
but I think I'm seeing a fundamental problem with Test::Unit.

Using the following code as the base reference:

#!/usr/bin/ruby

require "test/unit"

class TestImplArchFile < Test::Unit::TestCase
(0..10).each do |i|
(0..10).each do |j|
(0..100).each do |k|
define_method("test_i#{i}__j#{j}_k#{k}") do
flunk
end
end
end
end
end


When 'flunk' is set to flunk
Finished in 5.586734 seconds.
12221 tests, 12221 assertions, 12221 failures, 0 errors

When 'flunk' is replaced with true:
Finished in 0.482389 seconds.
12221 tests, 12221 assertions, 12221 failures, 0 errors

If I manually generate all 12k tests as individual tests in a new file,
with each one flunking:
Finished in 5.631014 seconds.
12221 tests, 12221 assertions, 12221 failures, 0 errors

The above would indicate that the problem is not with define_method, but
rather the ability of Test::Unit to handle a large number of test cases.

As a comparison, if I just print out results in a log file, I get the
awesome result:

Just using a class, no Unit::Test, not creating method, just print
method name
Finished in 0.102 seconds.

I couldn't get define_method going in a class to compare, it's a private
method in Method, and I really don't understand the Method rdoc example
using send.

Is there some sort of optimization in Test::Unit that I need to set?
Seems like something is really wrong, where is all this time going?
--
Posted via http://www.ruby-forum.com/.

 |  Next  |  Last
Pages: 1 2 3
Prev: ActionController::AbstractRequest.relative_url_root = "/rotspecs"
Next: Now Win a laptop computer from (EZ laptop) free of charge