Python as a scripting language. Alternative to bash script?
in [Python]
|
From: Michael Torrie on 28 Jun 2010 13:39 On 06/28/2010 05:48 AM, Dave Pawson wrote: > Main queries are: Ease of calling out to bash to use something like > imageMagick or Java? Ease of grabbing return parameters? E.g. convert > can return both height and width of an image. Can this be returned to > the Python program? Can Python access the exit status of a program? Sure. I've created a module called runcmd that does 90% of what I want (easy access to stdout, stderr, error code). I've attached it to this e-mail. Feel free to use it; this post puts my code into the public domain. > I'd prefer the advantages of using Python, just wondering if I got so > far with the port then found it wouldn't do something? Python really isn't a shell scripting language. So there are things that Bash does much better, such as spawning processes and piping them together. I've tried over the years to create a pythonic library that would let me do that, but haven't found a good syntax that I like. It turns out, though, that much of what I use piping for in Bash is to run external processes to do things that I could use python modules for. For example, I typically pipe stuff to cut a lot to get certain fields. For example: ps ax | cut -c1-5 In python I could simply take the output of "ps ax" and use python's own, superior, cutting routines (using my module): (err, stdout, stderr) = runcmd.run( [ 'ps', 'ax' ] ) for x in stdout.split('\n'): print x.strip().split()[0] Sure it's a couple more lines in this case, but in other cases, python's abilities make it simpler than bash. A great document on how you can exploit python's abilities (particularly generators) to replace bash pipelines is here: http://www.dabeaz.com/generators/
From: John Nagle on 28 Jun 2010 13:57 On 6/28/2010 7:58 AM, Benjamin Kaplan wrote: > How does a program return anything other than an exit code? Ah, yes, the second biggest design mistake in UNIX. Programs have "argv" and "argc", plus environment variables, going in. So, going in, there are essentially subroutine parameters. But all that comes back is an exit code. They should have had something similar coming back, with arguments to "exit()" returning the results. Then the "many small intercommunicating programs" concept would have worked much better. C was like that once. In the 1970s, all you could return was an "int" or a "float". But that got fixed. John Nagle
From: Dave Pawson on 28 Jun 2010 14:02 On 28 June 2010 18:39, Michael Torrie <torriem(a)gmail.com> wrote: > On 06/28/2010 05:48 AM, Dave Pawson wrote: >> Main queries are: Ease of calling out to bash to use something like >> imageMagick or Java? Ease of grabbing return parameters? E.g. convert >> can return both height and width of an image. Can this be returned to >> the Python program? Can Python access the exit status of a program? > > Sure. Â I've created a module called runcmd that does 90% of what I want > (easy access to stdout, stderr, error code). Â I've attached it to this > e-mail. Â Feel free to use it; this post puts my code into the public domain. Thanks Michael. > >> I'd prefer the advantages of using Python, just wondering if I got so >> far with the port then found it wouldn't do something? > > Python really isn't a shell scripting language. Â So there are things > that Bash does much better, such as spawning processes and piping them > together. Â I've tried over the years to create a pythonic library that > would let me do that, but haven't found a good syntax that I like. I'm using xml quite a bit and tried xmlsh which does the job but has what I think of as irregular syntax? Bash is fine for smaller scripts, but I had to be really careful by the time I got to around 1200 loc. Hence my preference for Python. > > It turns out, though, that much of what I use piping for in Bash is to > run external processes to do things that I could use python modules for. I kept thinking that.. Then I had to use Java/Python for some plain old calculations, so it kept nagging me that a cleaner approach would be a single language. > Sure it's a couple more lines in this case, but in other cases, python's > abilities make it simpler than bash. Â A great document on how you can > exploit python's abilities (particularly generators) to replace bash > pipelines is here: Â http://www.dabeaz.com/generators/ Thanks for the reference. I'm nearly there with the mix of return codes and output values, using subprocess. p1 = subprocess.Popen(['identify', '-format' , '%[fx:w]' , f ], stdout=subprocess.PIPE) resp1= p1.communicate() #print dir(p1) p2 = subprocess.Popen(['identify', '-format' , '%[fx:h]' , f ], stdout=subprocess.PIPE) resp2= p2.communicate() if (p1.returncode or p2.returncode): print "Error ",p1.returncode sys.exit(2) else: width=int(resp1[0]) height=int(resp2[0]) print "Height %s, width %s " % (height, width) Not happy with the error handling yet, but this was the bit that I thought I'd struggle with. regards -- Dave Pawson XSLT XSL-FO FAQ. Docbook FAQ. http://www.dpawson.co.uk
From: Michael Torrie on 28 Jun 2010 14:25 On 06/28/2010 12:05 PM, Dave Pawson wrote: > On 28 June 2010 18:39, Michael Torrie <torriem(a)gmail.com> wrote: > >> Sure. I've created a module called runcmd that does 90% of what I >> want (easy access to stdout, stderr, error code). I've attached >> it to this e-mail. Feel free to use it; this post puts my code >> into the public domain. > > Request please Michael. > > Examples of usage in the docstring? Well the simplest form is as I showed in my e-mail: (err,stdout,stderr) = runcmd.run (['command','arg1','arg2', ... ]) An example where you need to pass stdin: input="some string\n" (err,stdout,stderr) = runcmd.run (['cmd','arg1',], stdin=input) err is the return error code (int) and stdout and stderr are the strings the result from the process's output, both strings. There are some other features (or bugs probably) that you can get from the source code itself. I think at one time I wanted to be able to have callbacks that would do something with stdout and stderr (in a real-time, unbuffered way), but I have never used them and don't think they work.
From: Paul Rubin on 28 Jun 2010 14:45
John Nagle <nagle(a)animats.com> writes: > Programs have "argv" and "argc", plus environment variables, > going in. So, going in, there are essentially subroutine parameters. > But all that comes back is an exit code. They should have had > something similar coming back, with arguments to "exit()" returning > the results. Then the "many small intercommunicating programs" > concept would have worked much better. That's interesting but I'm having a hard time seeing how it would work. I think environment variables didn't exist in early versions of Unix, and argc/argv were passed to the child process on its stack. I guess the reverse side could involve the "wait" system call taking a callback parameter with a buffer to receive the returned data. But that still only happens when the child actually exits, and presumably intercommunicating netween programs should be bidirectional. But Unix has always had pipes for that. |