File::Slurp/IO::String/wantarray interaction bug
in [Perl]
|
Prev: FAQ 4.69 How can I make the Perl equivalent of a C structure/C++ class/hash or array of hashes or arrays?
Next: Converting HTML to PDF with Webkit
From: kj on 11 Jun 2010 18:57 Consider the following demo script: # first line # second line use File::Slurp 'slurp'; use IO::String; { my $string = slurp( $0 ); my $io_string = IO::String->new( $string ); printf ">>>%s<<<\n", scalar $io_string->READLINE; printf ">>>%s<<<\n", scalar $io_string->READLINE; } { my $io_string = IO::String->new( slurp( $0 ) ); printf ">>>%s<<<\n", scalar $io_string->READLINE; printf ">>>%s<<<\n", scalar $io_string->READLINE; } __END__ The only difference between the two blocks is that the first one assigns the value returned by slurp( $0 ) to the intermediate lexical variable $io_string, while the second one uses this value directly. As far as I can tell, the output of both blocks should be identical, but they are not even close. The output I get for the script above is: >>># first line <<< >>># second line <<< >>># first line <<< >>><<< Note that in the second block, the second line never gets printed; it is treated as an empty string. Stepping through the code with the debugger, I narrowed down the problem to the first line in the definition of IO::String::READLINE: sub READLINE { goto &getlines if wantarray; goto &getline; } Somehow, in the failing cases, the wantarray in the first line of READLINE evaluates to true, even though the original calling environment clearly specifies the scalar keyword. Therefore getlines gets inappropriately called, rather than the correct getline. How can the *really* force a scalar environment? (Clearly, the scalar keyword is not doing the job here.) FWIW, I'm using perl v5.10.0 on Ubuntu Linux. TIA! ~K
From: kj on 11 Jun 2010 19:34 In <huuf1m$9kt$1(a)reader1.panix.com> kj <no.email(a)please.post> writes: >Consider the following demo script: ># first line ># second line >use File::Slurp 'slurp'; >use IO::String; >{ > my $string = slurp( $0 ); > my $io_string = IO::String->new( $string ); > printf ">>>%s<<<\n", scalar $io_string->READLINE; > printf ">>>%s<<<\n", scalar $io_string->READLINE; >} >{ > my $io_string = IO::String->new( slurp( $0 ) ); > printf ">>>%s<<<\n", scalar $io_string->READLINE; > printf ">>>%s<<<\n", scalar $io_string->READLINE; >} >__END__ >The only difference between the two blocks is that the first one >assigns the value returned by slurp( $0 ) to the intermediate >lexical variable $io_string, while the second one uses this value >directly. >As far as I can tell, the output of both blocks should be identical, >but they are not even close. The output I get for the script above >is: >>>># first line ><<< >>>># second line ><<< >>>># first line ><<< >>>><<< >Note that in the second block, the second line never gets printed; >it is treated as an empty string. >Stepping through the code with the debugger, I narrowed down the >problem to the first line in the definition of IO::String::READLINE: >sub READLINE >{ > goto &getlines if wantarray; > goto &getline; >} >Somehow, in the failing cases, the wantarray in the first line of >READLINE evaluates to true, even though the original calling >environment clearly specifies the scalar keyword. Therefore getlines >gets inappropriately called, rather than the correct getline. >How can the *really* force a scalar environment? (Clearly, the >scalar keyword is not doing the job here.) OK, I found out one more detail. If I replace the original second block { my $io_string = IO::String->new( slurp( $0 ) ); printf ">>>%s<<<\n", scalar $io_string->READLINE; printf ">>>%s<<<\n", scalar $io_string->READLINE; } with { my $io_string = IO::String->new( scalar slurp( $0 ) ); # <- only this line changed printf ">>>%s<<<\n", scalar $io_string->READLINE; printf ">>>%s<<<\n", scalar $io_string->READLINE; } now the output is identical for both blocks. Still, this is quite a curveball. I see that in the original version, the call to slurp was happenning in a list context, and this probably messes up the string that actually gets used to define the IO::String object. But still, this does not explain why READLINE believes its in a list context. (I'm sorry that I can't show this easily; one needs to step there with the debugger). ~K
From: Uri Guttman on 11 Jun 2010 21:22 >>>>> "k" == kj <no.email(a)please.post> writes: k> In <huuf1m$9kt$1(a)reader1.panix.com> kj <no.email(a)please.post> writes: >> Consider the following demo script: >> # first line >> # second line >> use File::Slurp 'slurp'; most people use the read_file sub. slurp is just an alias to it for backwards compatability. >> use IO::String; why do you play with io::string? perl's open can do that builtin these days. it does require a scalar var and can't do it on data but that isn't much of an issue. and a general question, what are you trying to do here?? >> { >> my $string = slurp( $0 ); >> my $io_string = IO::String->new( $string ); >> printf ">>>%s<<<\n", scalar $io_string->READLINE; >> printf ">>>%s<<<\n", scalar $io_string->READLINE; >> } >> { >> my $io_string = IO::String->new( slurp( $0 ) ); all sub/method calls pass list context to their arguments. remember, those args get set into the @_ array so that makes sense. otherwise you couldn't pass in an array and it would be converted to its count which is usually not wanted in @_. >> printf ">>>%s<<<\n", scalar $io_string->READLINE; >> printf ">>>%s<<<\n", scalar $io_string->READLINE; >> } printf also like other funcs which can take a list of args, has list context for everything past the format arg. k> OK, I found out one more detail. If I replace the original second block k> { k> my $io_string = IO::String->new( slurp( $0 ) ); k> printf ">>>%s<<<\n", scalar $io_string->READLINE; k> printf ">>>%s<<<\n", scalar $io_string->READLINE; k> } k> with k> { k> my $io_string = IO::String->new( scalar slurp( $0 ) ); # <- only this line changed well, now it slurps the file to a scalar and not a list of lines. and the IO::Scalar docs disagree with your call: new [ARGS...] Class method. Return a new, unattached scalar handle. If any arguments are given, they're sent to open(). open [SCALARREF] Instance method. Open the scalar handle on a new scalar, pointed to by SCALARREF. If no SCALARREF is given, a "private" scalar is created to hold the file data. Returns the self object on success, undefined on error. so you aren't even calling it correctly. you need to pass it a scalar ref and you are passing it a scalar value or a scalar. maybe it can handle that but it doesn't say that in the docs (at least what i read). k> Still, this is quite a curveball. I see that in the original k> version, the call to slurp was happenning in a list context, and k> this probably messes up the string that actually gets used to define k> the IO::String object. But still, this does not explain why READLINE k> believes its in a list context. (I'm sorry that I can't show this k> easily; one needs to step there with the debugger). as i said, see the docs for printf. after the format it expects a list so that call will be in list context. uri -- Uri Guttman ------ uri(a)stemsystems.com -------- http://www.sysarch.com -- ----- Perl Code Review , Architecture, Development, Training, Support ------ --------- Gourmet Hot Cocoa Mix ---- http://bestfriendscocoa.com ---------
From: Willem on 12 Jun 2010 04:45 kj wrote: ) OK, I found out one more detail. If I replace the original second block ) ) { ) my $io_string = IO::String->new( slurp( $0 ) ); ) printf ">>>%s<<<\n", scalar $io_string->READLINE; ) printf ">>>%s<<<\n", scalar $io_string->READLINE; ) } ) ) with ) ) { ) my $io_string = IO::String->new( scalar slurp( $0 ) ); # <- only this line changed ) printf ">>>%s<<<\n", scalar $io_string->READLINE; ) printf ">>>%s<<<\n", scalar $io_string->READLINE; ) } ) ) now the output is identical for both blocks. ) ) Still, this is quite a curveball. I see that in the original ) version, the call to slurp was happenning in a list context, and ) this probably messes up the string that actually gets used to define ) the IO::String object. But still, this does not explain why READLINE ) believes its in a list context. (I'm sorry that I can't show this ) easily; one needs to step there with the debugger). When I saw your first post, it was immediately apparent to me that the call to slurp was being evaluated in list context, and that caused it to pass only the first line to the IO::String object. The READLINE context is a red herring. Maybe you misinterpreted the debugger output, maybe the debugger did something strange, maybe something else happened. In any case, it's quite clear that it *is* being called in scalar context, because that final fix works like it should. SaSW, Willem -- Disclaimer: I am in no way responsible for any of the statements made in the above text. For all I know I might be drugged or something.. No I'm not paranoid. You all think I'm paranoid, don't you ! #EOT
From: kj on 14 Jun 2010 11:49
In <slrni16ia2.250t.willem(a)turtle.stack.nl> Willem <willem(a)turtle.stack.nl> writes: >...Maybe you misinterpreted the >debugger output, maybe the debugger did something strange, maybe something >else happened. Well, it looks that it's one of those. To determine the calling context I was halting the execution within the READLINE method, and then printing the value of wantarray from the debugger prompt like this: DB<1> p wantarray 1 Apparentely, in the debugger "p wantarray" always produces this result, irrespective of the value of wantarray in the executing program. (That's quite the "banana peel", IMHO.) Poking around in the perldebug man page I discovered that the right way to determine the calling context from within the debugger is to use the T command: DB<4> T $ = IO::String::READLINE(ref(IO::String)) called from file `t/wantarraybug.pl' line 28 The leading '$' is shorthand for "called in scalar context". Thanks for your post! ~K |