Asp Forum - Executing a command from within python using the subprocess module

R (Chandra) Chandrasekhar

2/15/2010 12:36:00 PM

Dear Folks,

I want to execute a command from within python using the subprocess module.

Coming from a Perl background, I thought I could use variable
interpolation in strings, but found that this is neither supported nor
the Python way. Accordingly, I am a little at sea about how to
accomplish it.

I have stated what I am trying to do in the minimal example below:

---
import subprocess

width = 5
height = 30
colors = ['#abcdef]', '#456789']
filename = "/tmp/image.png"

# I want to get the equivalent of variable interpolation in Perl
# so that the command
#
# convert -size 5x30 gradient:#abcdef-#456789 /tmp/image.png
#
# is derived from the variables above
# and executed by subprocess.call() or subprocess.Popen()
# from within Python
#
# Note that the command "convert" is from the ImageMagick suite
# It exists and is executable by the shell;
# the remaining values are arguments.
# The command has been confirmed to execute correctly.
---

Thanks in advance.

Chandra

10 Answers

Alf P. Steinbach

2/15/2010 12:45:00 PM

* R (Chandra) Chandrasekhar:
>
> width = 5
> height = 30
> colors = ['#abcdef]', '#456789']
> filename = "/tmp/image.png"
>
> # I want to get the equivalent of variable interpolation in Perl
> # so that the command
> #
> # convert -size 5x30 gradient:#abcdef-#456789 /tmp/image.png
> #
> # is derived from the variables above

Assuming that the extra right square bracket in 'colors' is a typo, here's one way:

s = "convert -size {w}x{h} gradient:{g1}-{g2} {f}".format(
w = width, h = height, g1 = colors[0], g2 = colors[1], f = filename
)

Cheers & hth.,

- ALf

Peter Otten

2/15/2010 1:03:00 PM

R (Chandra) Chandrasekhar wrote:

> I want to execute a command from within python using the subprocess
> module.
>
> Coming from a Perl background, I thought I could use variable
> interpolation in strings, but found that this is neither supported nor
> the Python way. Accordingly, I am a little at sea about how to
> accomplish it.

import subprocess

def convert(width=5, height=30, colors=['#abcdef', '#456789'],
filename="tmp/image with space in its name.png"):
lookup = locals()
assert all("\n" not in str(s) for s in lookup.values())
subprocess.call("""\
convert
-size
{width}x{height}
gradient:{colors[0]}-{colors[1]}
{filename}""".format(**lookup).splitlines())

convert()

Peter

R (Chandra) Chandrasekhar

2/15/2010 3:48:00 PM

Peter Otten wrote:

> import subprocess
>
> def convert(width=5, height=30, colors=['#abcdef', '#456789'],
> filename="tmp/image with space in its name.png"):
> lookup = locals()
> assert all("\n" not in str(s) for s in lookup.values())
> subprocess.call("""\
> convert
> -size
> {width}x{height}
> gradient:{colors[0]}-{colors[1]}
> {filename}""".format(**lookup).splitlines())
>
> convert()
>
> Peter

Thank you. It works. Now I need to understand why and am able to follow
what you are doing part of the way:

1. Assign default values in the function definition.

2. Store the variables existing in the local namespace in the list lookup.

3. Assert that there are no newlines in the values in lookup converted
to strings. (Why? Is this because of splitlines() later on?)

4. Assemble a string (array?) for the subprocess.call argument using the
format string syntax (section 8.1.3 et seq. of the Python
documentation for 2.6.4). Your example works with default option of
shell=False for subprocess.call().

5. I am unable to decipher how you got to format(**lookup).splitlines())
especially the **prefix part, although I guess that splitlines() is
dishing out the arguments one by one from each line in the
subprocess.call argument.

Any correction of (1) to (4) and an explanation of (5) would be most
appreciated.

All in all, your code is a magnificent example of wrapping the function
within python. This is the first time I have seen something like this.

Thank you very much.

Chandra

John Posner

2/15/2010 3:59:00 PM

On 2/15/2010 7:35 AM, R (Chandra) Chandrasekhar wrote:
> Dear Folks,
>
> I want to execute a command from within python using the subprocess module.
>
> Coming from a Perl background, I thought I could use variable
> interpolation in strings, but found that this is neither supported

Yes, it is: see the use of string.Template below.

> ... nor
> the Python way.

That right -- it isn't the Python way. Python offers two basic
alternatives. Alf already presented the use of the "new" format() method
of string objects. I think "traditional" Python string formatting might
be what you want in this case:

colors = ['#abcdef', '#456789']
format_string = "convert -size 5x30 gradient:%s-%s /tmp/image.png"
cmd_string = format_string % tuple(colors)

.... or better, by making *colors* a tuple instead of a list ...

colors = ('#abcdef', '#456789')
format_string = "convert -size 5x30 gradient:%s-%s /tmp/image.png"
cmd_string = format_string % colors

As Peter demonstrated, you need to use split() on *cmd_string* when you
send the command to subprocess.call().

Now if you *really* miss using Perl, try this:

c1, c2 = ('#abcdef', '#456789')
template = string.Template(
"convert -size 5x30 gradient:$c1-$c2 /tmp/image.png")
cmd_string = template.substitute(locals())

Not worth the trouble, IMHO.

Refs:

http://www.python.org/doc/2.5.2/lib/typesseq-st...
http://www.python.org/doc/2.5.2/lib/n...

HTH,
John

R (Chandra) Chandrasekhar

2/15/2010 4:12:00 PM

Steve Holden

2/15/2010 4:32:00 PM

R (Chandra) Chandrasekhar wrote:
> Peter Otten wrote:
>> import subprocess
>>
>> def convert(width=5, height=30, colors=['#abcdef', '#456789'],
>> filename="tmp/image with space in its name.png"):
>> lookup = locals()
>> assert all("\n" not in str(s) for s in lookup.values())
>> subprocess.call("""\
>> convert
>> -size
>> {width}x{height}
>> gradient:{colors[0]}-{colors[1]}
>> {filename}""".format(**lookup).splitlines())
>>
>> convert()
>>
>> Peter
>
> One other question I forgot to ask is this why is there a terminal
> backslash in
>
>> subprocess.call("""\
>
> Removing the backslash makes the function fail.
>
> I wonder why, because """ is supposed to allow multi-line strings. I am
> sorry if this sounds obtuse but that backslash baffles me.

It does, but a leading newline would cause the splitlines() result to
start with an empty word - the backslash just causes the interpreter to
ignore the newline immediately following, rather than including it in
the string literal's value.

regards
Steve
--
Steve Holden +1 571 484 6266 +1 800 494 3119
PyCon is coming! Atlanta, Feb 2010 http://us....
Holden Web LLC http://www.hold...
UPCOMING EVENTS: http://holdenweb.event...

R (Chandra) Chandrasekhar

2/15/2010 4:37:00 PM

Alf P. Steinbach wrote:
> * R (Chandra) Chandrasekhar:
>>
>> width = 5
>> height = 30
>> colors = ['#abcdef]', '#456789']
>> filename = "/tmp/image.png"
>>
>> # I want to get the equivalent of variable interpolation in Perl
>> # so that the command
>> #
>> # convert -size 5x30 gradient:#abcdef-#456789 /tmp/image.png
>> #
>> # is derived from the variables above
>
> Assuming that the extra right square bracket in 'colors' is a typo,
> here's one way:
>
> s = "convert -size {w}x{h} gradient:{g1}-{g2} {f}".format(
> w = width, h = height, g1 = colors[0], g2 = colors[1], f = filename
> )
>
>
> Cheers & hth.,
>
> - ALf

Thanks, Alf. It works if I use it so:

subprocess.call(s, shell=True)

and I think that is because s is a single string assembled in almost the
variable interpolation fashion of Perl. It does not work with the
default shell=False argument, presumably because the arguments are not
split into separate strings (something that was horrible to do by hand
using the %s and %() syntax that I had tried out previously).

I think that you and Peter have, between you, shown me two ways of using
subprocess.call(): one with shell=True and the other with shell = False.

Thanks.

Chandra

Peter Otten

2/15/2010 4:44:00 PM

R (Chandra) Chandrasekhar wrote:

> Peter Otten wrote:
>
>> import subprocess
>>
>> def convert(width=5, height=30, colors=['#abcdef', '#456789'],
>> filename="tmp/image with space in its name.png"):
>> lookup = locals()
>> assert all("\n" not in str(s) for s in lookup.values())
>> subprocess.call("""\
>> convert
>> -size
>> {width}x{height}
>> gradient:{colors[0]}-{colors[1]}
>> {filename}""".format(**lookup).splitlines())
>>
>> convert()
>>
>> Peter
>
> Thank you. It works. Now I need to understand why and am able to follow
> what you are doing part of the way:
>
> 1. Assign default values in the function definition.
>
> 2. Store the variables existing in the local namespace in the list lookup.
>
> 3. Assert that there are no newlines in the values in lookup converted
> to strings. (Why? Is this because of splitlines() later on?)

Yes.

> 4. Assemble a string (array?) for the subprocess.call argument using the
> format string syntax (section 8.1.3 et seq. of the Python
> documentation for 2.6.4). Your example works with default option of
> shell=False for subprocess.call().

I wanted to pass the executable and its arguments as a list (a python "list"
is called array in other languages) to avoid bothering with shell escapes.

> 5. I am unable to decipher how you got to format(**lookup).splitlines())
> especially the **prefix part, although I guess that splitlines() is
> dishing out the arguments one by one from each line in the
> subprocess.call argument.

Given some_dict = {key1: value1, key2: value2,...}

f(**some_dict)

is a shortcut for

f(key1=value1, key2=value2, ...)

with the additional twist that you do not have to know the key/value pairs
in advance.

I introduced splitlines to avoid calling the format method on every argument
to "convert", i. e.

def convert2(width=5, height=30, colors=['#abcdef', '#456789'],
filename="tmp/image with space in its name.png"):
subprocess.call([
"convert",
"-size",
"{width}x{height}".format(width=width, height=height),
"gradient:{0}-{1}".format(*colors),
filename])

I supected that it would look cleaner than the above, but now that I tried
it I think convert2() is the better alternative.

Peter

John Doe

2/15/2010 4:57:00 PM

On Tue, 16 Feb 2010 00:11:36 +0800, R (Chandra) Chandrasekhar wrote:

> One other question I forgot to ask is this why is there a terminal
> backslash in
>
>> subprocess.call("""\
>
> Removing the backslash makes the function fail.
>
> I wonder why, because """ is supposed to allow multi-line strings. I am
> sorry if this sounds obtuse but that backslash baffles me.

The backslash causes the following newline to be skipped, so the "c" at
the beginning of "convert" is the first character in the string. Without
it, the newline would be the first character, and the .splitlines() would
result in an empty string as the first element in the list.

Example:

> """
= hello
= world
= """.splitlines()
['', 'hello', 'world']
> """\
= hello
= world
= """.splitlines()
['hello', 'world']

aahz

2/18/2010 9:41:00 PM

In article <5YudnaFysO8HouTWnZ2dnUVZ_tidnZ2d@westnet.com.au>,
R (Chandra) Chandrasekhar <chyavana@gmail.com> wrote:
>
>---
>import subprocess
>
>width = 5
>height = 30
>colors = ['#abcdef]', '#456789']
>filename = "/tmp/image.png"
>
># I want to get the equivalent of variable interpolation in Perl
># so that the command
>#
># convert -size 5x30 gradient:#abcdef-#456789 /tmp/image.png

Here's the equivalent of Peter's cute code in simpler form:

cmd = [
'convert',
'-size',
'%sx%s' % (width, height),
'gradient:%s-%s' % tuple(colors),
# above could also be: 'gradient:%s-%s' % (colors[0], colors[1]),
filename,
]
subprocess.Popen(cmd)
--
Aahz (aahz@pythoncraft.com) <*> http://www.python...

"At Resolver we've found it useful to short-circuit any doubt and just
refer to comments in code as 'lies'. :-)"

comp.lang.python

Executing a command from within python using the subprocess module

R (Chandra) Chandrasekhar

Alf P. Steinbach

Peter Otten

R (Chandra) Chandrasekhar

John Posner

R (Chandra) Chandrasekhar

Steve Holden

R (Chandra) Chandrasekhar

Peter Otten

John Doe

aahz

x Login to ForumsZone