multiprocessing and accessing server's stdout
in [Python]
|
Prev: TypeError: _new_() takes exactly 3 arguments (2 given) - whatdoes it mean?
Next: TypeError: _new_() takes exactly 3 arguments (2 given) - what does it mean?
From: Adam Tauno Williams on 28 May 2010 16:44 On Fri, 2010-05-28 at 15:41 +0100, Martin P. Hellwig wrote: > On 05/28/10 13:17, Adam Tauno Williams wrote: > <cut> > > You should be able to point it any any file-like object. But, again, > > why? > > If you have the data in the process why send it to stdout and redirect > > it. Why not just send the data to the client directly? > Well you might want to multiplex it to more then one client, not saying > that this is the case here, just something I imagine possible. That still doesn't make sense. Why 'multiplex stdout'? Why not just multiplex the data into proper IPC channels in the first place? -- Adam Tauno Williams <awilliam(a)whitemice.org> LPIC-1, Novell CLA <http://www.whitemiceconsulting.com> OpenGroupware, Cyrus IMAPd, Postfix, OpenLDAP, Samba
From: Martin P. Hellwig on 28 May 2010 19:47 On 05/28/10 21:44, Adam Tauno Williams wrote: > On Fri, 2010-05-28 at 15:41 +0100, Martin P. Hellwig wrote: >> On 05/28/10 13:17, Adam Tauno Williams wrote: >> <cut> >>> You should be able to point it any any file-like object. But, again, >>> why? >>> If you have the data in the process why send it to stdout and redirect >>> it. Why not just send the data to the client directly? >> Well you might want to multiplex it to more then one client, not saying >> that this is the case here, just something I imagine possible. > > That still doesn't make sense. Why 'multiplex stdout'? Why not just > multiplex the data into proper IPC channels in the first place? I am going on a stretch here, I mostly agree with you, just trying to illustrate that there could be corner cases where this is sensible. The current situation could be that there is a client/server program (binary only perhaps) which is not multi-user safe. Python can be used as a wrapper around the server to make it multi-client, by emulating the exact behavior towards the client, the client program does not have to be changed. -- mph
From: Tim Arnold on 1 Jun 2010 11:35 On May 28, 7:47 pm, "Martin P. Hellwig" <martin.hell...(a)dcuktec.org> wrote: > On 05/28/10 21:44, Adam Tauno Williams wrote: > > > On Fri, 2010-05-28 at 15:41 +0100, Martin P. Hellwig wrote: > >> On 05/28/10 13:17, Adam Tauno Williams wrote: > >> <cut> > >>> You should be able to point it any any file-like object. But, again, > >>> why? > >>> If you have the data in the process why send it tostdoutand redirect > >>> it. Why not just send the data to the client directly? > >> Well you might want to multiplex it to more then one client, not saying > >> that this is the case here, just something I imagine possible. > > > That still doesn't make sense. Why 'multiplexstdout'? Why not just > > multiplex the data into proper IPC channels in the first place? > > I am going on a stretch here, I mostly agree with you, just trying to > illustrate that there could be corner cases where this is sensible. > The current situation could be that there is a client/server program > (binary only perhaps) which is not multi-user safe. > > Python can be used as a wrapper around the server to make it > multi-client, by emulating the exact behavior towards the client, the > client program does not have to be changed. > > -- > mph Hi, This is the setup I was asking about. I've got users using a python-written command line client. They're requesting services from a remote server that fires a LaTeX process. I want them to see the stdout from the LaTeX process. I was using multiprocessing to handle the requests, but the stdout shows up on the server's terminal window where I started the server.serve_forever process. I started using RPyC and now the stdout appears on the client terminal making the request. I was trying to minimize the number of packages I use, hoping I could get the same capability from multiprocessing that I get with RPyC. thanks for the comments. I'm still processing what's been written here. --Tim
From: Bryan on 2 Jun 2010 16:07 Tim Arnold wrote: > Hi, This is the setup I was asking about. > I've got users using a python-written command line client. They're > requesting services from a remote server that fires a LaTeX process. I > want them to see the stdout from the LaTeX process. So what you really need is to capture the output of a command, in this case LaTeX, so you can copy it back to the client. You can do that with the subprocess module in the Python standard library. If the command generated so much output so fast that you felt the need to avoid the extra copy, I suppose you could fork() then hook stdout directly to socket connected to the client with dup2(), then exec() the command. But no need for that just to capture LaTeX's output. -- --Bryan Olson
From: Bryan on 2 Jun 2010 19:00
I wrote: > So what you really need is to capture the output of a command, in this > case LaTeX, so you can copy it back to the client. You can do that > with the subprocess module in the Python standard library. > > If the command generated so much output so fast that you felt the need > to avoid the extra copy, I suppose you could fork() then hook stdout > directly to socket connected to the client with dup2(), then exec() > the command. But no need for that just to capture LaTeX's output. Upon further reading, I see that the subprocess module makes the direct-hookup method easy, at least on 'nix systems. Just tell subprocess.Popen to use the client-connected socket as the subprocess's stdout. The question here turns out to make more sense than I had though upon reading the first post. The server runs a command at the client's request, and we want to deliver the output of that command back to the client. A brilliantly efficient method is to direct the command's stdout to the client's connection. Below is a demo server that sends the host's words file to any client that connects. It assumes Unix. --Bryan Olson #!/usr/bin/python from thread import start_new_thread from subprocess import Popen def demo(sock): subp = Popen(['cat', '/usr/share/dict/words'], stdout=sock) subp.wait() sock.close() if __name__ == '__main__': listener_sock = socket.socket(socket.AF_INET, socket.SOCK_STREAM) listener_sock.bind(('', 54321)) listener_sock.listen(5) while True: sock, remote_address = listener_sock.accept() start_new_thread(demo, (sock,)) |