Prev: Need to use streams/buffers, but NOT related to stdin, stdoutor to files
Next: mt
From: FangQ on 18 Sep 2009 16:49
hi

I am trying to do a simple task: I have a buffer, I want to write this
buffer as the stdin of a command, say "tac", and capture the output
from the stdout and save to another buffer.

after searching examples and playing a little bit, I wrote the a
function (see below). However, it gave me a "Broken pipe" error when I
ran the code.

Can someone point out to me what was wrong? your help is greatly
appreciated!

Qianqian


int pipemydata(void *inbuf, long inlen, void *outbuf, long *outlen){
int fd[2];
int childid;
*outlen=0;
if(pipe(fd)==-1){
*outlen=-1;
return *outlen;
}
if((childid=fork())){ /*write data into the pipe*/ /*parent
thread*/
close(fd[0]);
FILE *f=fdopen(fd[1],"wb");
fwrite(inbuf,inlen,1,f);
fclose(f);
}else{ /*child thread to read output from tac*/
dup2(fd[0],fileno(stdin));
close(fd[1]);
*outlen=0;
FILE *f=popen("tac","rb");
while(!feof(f)){
*outlen+=fread(outbuf+(*outlen),1,1024,f);
}
pclose(f);
}
return *outlen;
}
From: Alan Curry on 18 Sep 2009 22:13
In article <6cec3c7f-94f4-4f59-9844-30ae2c9ff055(a)v2g2000vbb.googlegroups.com>,
FangQ <fangqq(a)gmail.com> wrote:
>hi
>
>I am trying to do a simple task: I have a buffer, I want to write this
>buffer as the stdin of a command, say "tac", and capture the output
>from the stdout and save to another buffer.
>
>after searching examples and playing a little bit, I wrote the a
>function (see below). However, it gave me a "Broken pipe" error when I
>ran the code.
>
>Can someone point out to me what was wrong? your help is greatly
>appreciated!

Well there are a few things wrong, but the one you need to fix first is than
popen() doesn't do "rb" mode. In unix there are no "text files" and "binary
files", there are just files. popen() is a unix function, not an ANSI C
function. I'd say the same for fdopen() but for some reason it does accept
the extra "b" (which has no effect).

You should add more error checking. If you had done this:

FILE *f=popen("tac","rb");
if(!f)
perror("popen");

you would have got a "popen: Invalid argument" message to give you a clue.

Change that "rb" to "r" and you'll make a little bit of progress.

[...]
> if((childid=fork())){ /*write data into the pipe*/ /*parent
>thread*/
> close(fd[0]);
> FILE *f=fdopen(fd[1],"wb");
> fwrite(inbuf,inlen,1,f);
> fclose(f);
> }else{ /*child thread to read output from tac*/
> dup2(fd[0],fileno(stdin));
> close(fd[1]);
> *outlen=0;
> FILE *f=popen("tac","rb");
> while(!feof(f)){
> *outlen+=fread(outbuf+(*outlen),1,1024,f);
> }
> pclose(f);
> }
> return *outlen;
>}

The next major thing is why are you using popen() at all? You already created
a pipe and a child process, you don't need yet another pipe and yet another
process (which is what popen gives you).

And then there's the problem that your child doesn't exit when it's done, so
you have both processes reaching the end of the function and returning to the
caller, which is probably not what you wanted.

--
Alan Curry
From: Marcel Bruinsma on 18 Sep 2009 23:53
Am Freitag 18 September 2009 22:49, FangQ a écrit :

> Can someone point out to me what was wrong?

Look here for a description of popen,
http://www.opengroup.org/onlinepubs/9699919799/functions/popen.html
and fork,
http://www.opengroup.org/onlinepubs/9699919799/functions/fork.html

Do not randomly mix all the different types; but most of all,
select /appropriate/ types (see manual pages).
Close descriptors when you no longer need them.


#include <sys/types.h>
#include <unistd.h>
#include <stdio.h>
#include <err.h>

ssize_t
pipemydata(const void *ib, size_t il, char *ob)
{
int d[2]; pid_t p; size_t ol = 0;

if (pipe(d)) return (ssize_t)-1;
if ((p = fork()) < 0) {
close(d[1]); close(d[0]);
return (ssize_t)-1;
}
if (p) { /* parent */
FILE *f; int c;
dup2(STDIN_FILENO, d[1]);
dup2(d[0], STDIN_FILENO); close(d[0]);
f = popen("tac", "r");
dup2(d[1], STDIN_FILENO); close(d[1]);
if (!f) {
warn("popen tac");
return (ssize_t)-1;
}
while ((c = getc(f)) != EOF) {
*ob++ = c; ol++;
}
if (pclose(f)) {
warn("pclose tac");
return (ssize_t)-1;
}
return ol;
}
/* child */
close(d[0]);
write(d[1], ib, il);
_exit(0);
}

const char ibuf[] = "\
I am trying to do a simple task: \n\
I have a buffer, I want to write \n\
this buffer as the stdin of a \n\
command, say “tac”, and capture \n\
the output from the stdout and \n\
save to another buffer.\n\
";
char obuf[9999];

int
main()
{
ssize_t l = sizeof(ibuf) - 1;

printf("*****\n");
write(STDOUT_FILENO, ibuf, l);
printf("*****\n");
l = pipemydata(ibuf, l, obuf);
if (l == (ssize_t)-1)
return 1;
write(STDOUT_FILENO, obuf, l);
printf("*****\n");
return 0;
}

--
printf -v email $(echo \ 155 141 162 143 145 154 142 162 165 151 \
156 163 155 141 100 171 141 150 157 157 056 143 157 155|tr \ \\\\)
# Live every life as if it were your last! #
From: Alan Curry on 19 Sep 2009 00:25
In article <h91eoh$f1c$1(a)aioe.org>, I wrote:
>
>The next major thing is why are you using popen() at all? You already created
>a pipe and a child process, you don't need yet another pipe and yet another
>process (which is what popen gives you).

Correction: You do need another pipe (since you're piping in and out of the
"tac" process) but you don't really need the extra process.

--
Alan Curry
From: Barry Margolin on 19 Sep 2009 01:22
In article <h91mfh$mm5$1(a)aioe.org>, pacman(a)kosh.dhis.org (Alan Curry)
wrote:

> In article <h91eoh$f1c$1(a)aioe.org>, I wrote:
> >
> >The next major thing is why are you using popen() at all? You already created
> >a pipe and a child process, you don't need yet another pipe and yet another
> >process (which is what popen gives you).
>
> Correction: You do need another pipe (since you're piping in and out of the
> "tac" process) but you don't really need the extra process.

You may need it to avoid deadlock. If the writer tries to write
everything to the send pipe before reading anything, the receive pipe's
buffer may fill up, and then the tac process would block, and stop
reading from the send pipe, so it will fill up and the writer will block.

I don't think this can happen with a command like "tac", since it can't
start writing until it reads everything you send it. But it can happen
with other programs that produce output as they go (e.g. awk, grep,
sed). And even if you know the program produces one line of output for
each line of input, you can't solve it by doing alternating writes and
reads, because it's likely that the program uses stdio, which by default
does full buffering when output is to a pipe.

Using separate processes (or threads) for writing and reading solves
this problem. You could also do it with select(), but that's a little
more complicated to code.

--
Barry Margolin, barmar(a)alum.mit.edu
Arlington, MA
*** PLEASE post questions in newsgroups, not directly to me ***
*** PLEASE don't copy me on replies, I'll read them in the group ***
 |  Next  |  Last
Pages: 1 2
Prev: Need to use streams/buffers, but NOT related to stdin, stdoutor to files
Next: mt