Prev: Thank you Rakudo-Star (and question about SciTE for Perl 6)
Next: Can this be done (by a noob :))
From: John Kelly on 30 Jul 2010 10:39

The Camel book, 16.3.1. Anonymous Pipes says:

> Perl uses your default system shell (/bin/sh on Unix) whenever a pipe
> command contains special characters that the shell cares about. If
> you're only starting one command, and you don't need--or don't want--to
> use the shell, you can use the multi-argument form of a piped open ...

> ... But then you don't get I/O redirection, wildcard expansion, or
> multistage pipes, since Perl relies on your shell to do those.

and 29.2.104. open says

> Any pipe command containing shell metacharacters such as wildcards or
> I/O redirections is passed to your system's canonical shell (/bin/sh on
> Unix), so those shell-specific constructs can be processed first. If no
> metacharacters are found, Perl launches the new process itself without
> calling the shell.


I have a script that traps the standard output of any command passed in
as args to the script. My piped open uses 2>&1 to grab stderr as well
as stdout. I thought > was a shell metacharacter so I expected to see
/bin/sh between my script and the trapped command when doing ps ax. But
in many cases, Perl runs the trapped command directly, without needing
/bin/sh.

You can see that by running the script like this:

../myscript sleep 10

and then doing a ps ax before the sleep ends. Is the book wrong, or I
am I missing something? Here is the script:






#!/usr/bin/perl

# Define author
# John Kelly, July 28, 2010

# Define copyright
# Copyright John Kelly, 2010. All rights reserved.

# Define license
# Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this work except in compliance with the License.
# You may obtain a copy of the License at:
# http://www.apache.org/licenses/LICENSE-2.0

# Define symbols and (words)
# OT ........... Output Trap
# bas0 ......... basename of $0
# binx ......... binary executable
# tt ........... temporary time




use strict;
use FileHandle;
use File::Basename;
use POSIX qw (strftime);

STDOUT->autoflush (1);
STDERR->autoflush (1);

my $bas0 = basename ($0);
my $binx;

unless ($binx = shift @ARGV) {
print "Usage: ", $bas0, " binary.executable [args]\n";
exit 1;
}

my $basx = basename ($binx);

if (!defined (open OT, "$binx @ARGV 2>&1 |")) {
printf "%s -> %s: failure starting %s: $!\n", &tt, $bas0, $binx;
exit 1;
}
while (<OT>) {
/^\s*$/ && next;
printf "%s -> %s: ", &tt, $basx;
print $_;
}
if (!(close OT) && $!) {
printf "%s -> %s: failure closing OT: $!\n", &tt, $bas0;
} else {
if ($? & 127) {
printf "%s -> %s: %s signal %d, %s coredump\n", &tt, $bas0, $basx,
($? & 127), ($? & 128) ? 'with' : 'without';
} else {
printf "%s -> %s: %s exit value %d\n", &tt, $bas0, $basx, $? >> 8;
}
}

sub tt {
strftime "%a %b %e %H:%M:%S %Z %Y", localtime;
}
--
Web mail, POP3, and SMTP
http://www.beewyz.com/freeaccounts.php

From: Ilya Zakharevich on 30 Jul 2010 12:38
On 2010-07-30, John Kelly <jak(a)isp2dial.com> wrote:
> as args to the script. My piped open uses 2>&1 to grab stderr as well
> as stdout. I thought > was a shell metacharacter so I expected to see
> /bin/sh between my script and the trapped command when doing ps ax.

At least on Unix and OS/2, I implemented shell-less "2>&1". (For a
decade already I'm also sitting on code which does shell-less "FOO"
and 'bar', but have no time to run a test suite - so I never send it
to p5p.)

[As my other post this week shows, after I fixed the bug (in fact 2)
in the OS/2 branch I suspected for several years, my "stress test" now
passes on OS/2. On other architectures, Perl's pipe open() (and maybe
system() too???) is/are seriously buggy...]

Hope this helps,
Ilya
From: John Kelly on 30 Jul 2010 13:40
On Fri, 30 Jul 2010 16:38:40 +0000 (UTC), Ilya Zakharevich
<nospam-abuse(a)ilyaz.org> wrote:

> on Unix and OS/2, I implemented shell-less "2>&1"

That's what I wondered. Makes sense to avoid an extra shell pid when
2>&1 is the only shell metacharacter present.

Works for me.


--
Web mail, POP3, and SMTP
http://www.beewyz.com/freeaccounts.php

From: C.DeRykus on 30 Jul 2010 19:20
On Jul 30, 7:39 am, John Kelly <j...(a)isp2dial.com> wrote:
> The Camel book, 16.3.1. Anonymous Pipes says:
>
> > Perl uses your default system shell (/bin/sh on Unix) whenever a pipe
> > command contains special characters that the shell cares about. If
> > you're only starting one command, and you don't need--or don't want--to
> > use the shell, you can use the multi-argument form of a piped open ...
> > ... But then you don't get I/O redirection, wildcard expansion, or
> > multistage pipes, since Perl relies on your shell to do those.
>
> and 29.2.104. open says
>
> > Any pipe command containing shell metacharacters such as wildcards or
> > I/O redirections is passed to your system's canonical shell (/bin/sh on
> > Unix), so those shell-specific constructs can be processed first. If no
> > metacharacters are found, Perl launches the new process itself without
> > calling the shell.
>
> I have a script that traps the standard output of any command passed in
> as args to the script.   My piped open uses 2>&1 to grab stderr as well
> as stdout.  I thought > was a shell metacharacter so I expected to see
> /bin/sh between my script and the trapped command when doing ps ax.  But
> in many cases, Perl runs the trapped command directly, without needing
> /bin/sh.
>
> You can see that by running the script like this:
>
> ./myscript sleep 10
>
> and then doing a ps ax before the sleep ends.  Is the book wrong, or I
> am I missing something?  Here is the script:
>
> #!/usr/bin/perl
>
> #   Define author
> #       John Kelly, July 28, 2010
>
> #   Define copyright
> #       Copyright John Kelly, 2010. All rights reserved.
>
> #   Define license
> #       Licensed under the Apache License, Version 2.0 (the "License");
> #       you may not use this work except in compliance with the License.
> #       You may obtain a copy of the License at:
> #      http://www.apache.org/licenses/LICENSE-2.0
>
> #   Define symbols and (words)
> #       OT ...........  Output Trap
> #       bas0 .........  basename of $0
> #       binx .........  binary executable
> #       tt ...........  temporary time
>
> use strict;
> use FileHandle;
> use File::Basename;
> use POSIX qw (strftime);
>
> STDOUT->autoflush (1);
> STDERR->autoflush (1);
>
> my $bas0 = basename ($0);
> my $binx;
>
> unless ($binx = shift @ARGV) {
>     print "Usage: ", $bas0, " binary.executable [args]\n";
>     exit 1;
>
> }
>
> my $basx = basename ($binx);
>
my $kid; # edit #1

> if (!defined ($kid = open OT, "$binx @ARGV 2>&1 |")) { # edit # 2
>     printf "%s -> %s: failure starting %s: $!\n", &tt, $bas0, $binx;
>     exit 1;}
>
print "parent shell=",getppid()," perl process=$$", # edit # 3
" perl kid=$kid\n";
> while (<OT>) {
>     /^\s*$/ && next;
>     printf "%s -> %s: ", &tt, $basx;
>     print $_;}
>
> if (!(close OT) && $!) {
>     printf "%s -> %s: failure closing OT: $!\n", &tt, $bas0;} else {
>
>     if ($? & 127) {
>         printf "%s -> %s: %s signal %d, %s coredump\n", &tt, $bas0, $basx,
>           ($? & 127), ($? & 128) ? 'with' : 'without';
>     } else {
>         printf "%s -> %s: %s exit value %d\n", &tt, $bas0, $basx, $? >> 8;
>     }
>
> }
>
> sub tt {
>     strftime "%a %b %e %H:%M:%S %Z %Y", localtime;}
>


On FreeBSD (with small edits above), I don't see that
happening.

$ myscript.pl sleep 60
parent shell=71889 perl process=75147 perl kid=75148

$ ps -ax
PID TT STAT TIME COMMAND
71889 2 SNs 0:00.01 -bash (bash)
75147 2 SN+ 0:00.02 /usr/bin/perl ./shell.pl sleep 60
75148 2 SN+ 0:00.00 sleep 60
.....

I believe execl in the perl kid launches a shell
which then gets overlaid by sleep(). From doio.c:

PerlProc_execl(PL_sh_path, "sh", "-c", cmd, (char *)NULL);

--
Charles DeRykus
From: John Kelly on 30 Jul 2010 20:21
On Fri, 30 Jul 2010 16:20:45 -0700 (PDT), "C.DeRykus"
<derykus(a)gmail.com> wrote:

>On Jul 30, 7:39�am, John Kelly <j...(a)isp2dial.com> wrote:

>> I have a script that traps the standard output of any command passed in
>> as args to the script. � My piped open uses 2>&1 to grab stderr as well
>> as stdout. �I thought > was a shell metacharacter so I expected to see
>> /bin/sh between my script and the trapped command when doing ps ax. �But
>> in many cases, Perl runs the trapped command directly, without needing
>> /bin/sh.

>On FreeBSD (with small edits above), I don't see that
>happening.
>
>$ myscript.pl sleep 60
>parent shell=71889 perl process=75147 perl kid=75148

That's what I'm saying; the kid pid is only 1 greater than the perl pid,
which means there was never a shell pid launched.


>$ ps -ax
> PID TT STAT TIME COMMAND
>71889 2 SNs 0:00.01 -bash (bash)
>75147 2 SN+ 0:00.02 /usr/bin/perl ./shell.pl sleep 60
>75148 2 SN+ 0:00.00 sleep 60
>....
>
>I believe execl in the perl kid launches a shell
>which then gets overlaid by sleep().

I don't think so.

You could overlay the shell by prefixing the command with the "exec"
shell builtin, but I didn't do that.

Seems like perl is recognizing 2>&1 as a limited special case, copying
fd1 to fd2 , then exec'ing the binary directly, without using a shell.


--
Web mail, POP3, and SMTP
http://www.beewyz.com/freeaccounts.php

 |  Next  |  Last
Pages: 1 2 3 4 5 6
Prev: Thank you Rakudo-Star (and question about SciTE for Perl 6)
Next: Can this be done (by a noob :))