Asp Forum - piping into a python script

donn

1/24/2008 3:17:00 PM

Hi,
(Gnu/Linux - Python 2.4/5)
Given these two examples:
1.
../fui.py *.py
2.
ls *.py | ./fui.py

How can I capture a list of the arguments?
I need to get all the strings (file or dir names) passed via the normal
command line and any that may come from a pipe.

There is a third case:
3.
ls *.jpg | ./fui.py *.png
Where I would be gathering strings from two places.

I am trying to write a command-line friendly tool that can be used in
traditional gnu/linux ways, otherwise I'd skip the pipe stuff totally.

I have tried:
1. pipedIn = sys.stdin.readlines()
Works fine for example 2, but example 1 goes into a 'wait for input' mode
and that's no good. Is there a way to tell when no input is coming from a
pipe at all?

2. import fileinput
for line in fileinput.input():
print (line)
But this opens each file and I don't want that.

I have seen a lot of search results that don't quite answer this angle of
the question, so I'm trying on the list.

\d

15 Answers

Marc 'BlackJack' Rintsch

1/24/2008 3:26:00 PM

On Thu, 24 Jan 2008 17:17:25 +0200, Donn Ingle wrote:

> Given these two examples:
> 1.
> ./fui.py *.py
> 2.
> ls *.py | ./fui.py
>
> How can I capture a list of the arguments?
> I need to get all the strings (file or dir names) passed via the normal
> command line and any that may come from a pipe.
>
> There is a third case:
> 3.
> ls *.jpg | ./fui.py *.png
> Where I would be gathering strings from two places.
>
> I am trying to write a command-line friendly tool that can be used in
> traditional gnu/linux ways, otherwise I'd skip the pipe stuff totally.
>
> I have tried:
> 1. pipedIn = sys.stdin.readlines()
> Works fine for example 2, but example 1 goes into a 'wait for input' mode
> and that's no good. Is there a way to tell when no input is coming from a
> pipe at all?

Usually Linux tools that can get the data from command line or files treat
a single - as file name special with the meaning of: read from stdin.

So the interface if `fui.py` would be:

1. ./fui.py *.a
2. ls *.a | ./fui.py -
3. ls *.a | ./fui.py *.b -

Ciao,
Marc 'BlackJack' Rintsch

Paddy

1/24/2008 3:26:00 PM

On Jan 24, 3:17 pm, Donn Ingle <donn.in...@gmail.com> wrote:
> Hi,
> (Gnu/Linux - Python 2.4/5)
> Given these two examples:
> 1.
> ./fui.py *.py
> 2.
> ls *.py | ./fui.py
>
> How can I capture a list of the arguments?
> I need to get all the strings (file or dir names) passed via the normal
> command line and any that may come from a pipe.
>
> There is a third case:
> 3.
> ls *.jpg | ./fui.py *.png
> Where I would be gathering strings from two places.
>
> I am trying to write a command-line friendly tool that can be used in
> traditional gnu/linux ways, otherwise I'd skip the pipe stuff totally.
>
> I have tried:
> 1. pipedIn = sys.stdin.readlines()
> Works fine for example 2, but example 1 goes into a 'wait for input' mode
> and that's no good. Is there a way to tell when no input is coming from a
> pipe at all?
>
> 2. import fileinput
> for line in fileinput.input():
> print (line)
> But this opens each file and I don't want that.
>
> I have seen a lot of search results that don't quite answer this angle of
> the question, so I'm trying on the list.
>
> \d

Try the fileinput module.
What you describe above is pretty close to the unix 'standard' but not
quite.
if we substitute the lp command instead of ./fui, the command normally
takes a list of files to act on as its arguments, and anything piped
in goes to its stdin where it is processed if it has an argument of -
fileinput works that way but you may have problems with your:
ls *.jpg | ./fui.py *.png
Which might better be expressed as:
./fui.py `ls *.jpg` *.png
which would work for ls and a python program using the fileinput
module.

- Paddy.

Paddy

1/24/2008 3:46:00 PM

On Jan 24, 3:25 pm, Marc 'BlackJack' Rintsch <bj_...@gmx.net> wrote:
> On Thu, 24 Jan 2008 17:17:25 +0200, Donn Ingle wrote:
> > Given these two examples:
> > 1.
> > ./fui.py *.py
> > 2.
> > ls *.py | ./fui.py
>
> > How can I capture a list of the arguments?
> > I need to get all the strings (file or dir names) passed via the normal
> > command line and any that may come from a pipe.
>
> > There is a third case:
> > 3.
> > ls *.jpg | ./fui.py *.png
> > Where I would be gathering strings from two places.
>
> > I am trying to write a command-line friendly tool that can be used in
> > traditional gnu/linux ways, otherwise I'd skip the pipe stuff totally.
>
> > I have tried:
> > 1. pipedIn = sys.stdin.readlines()
> > Works fine for example 2, but example 1 goes into a 'wait for input' mode
> > and that's no good. Is there a way to tell when no input is coming from a
> > pipe at all?
>
> Usually Linux tools that can get the data from command line or files treat
> a single - as file name special with the meaning of: read from stdin.
>
> So the interface if `fui.py` would be:
>
> 1. ./fui.py *.a
> 2. ls *.a | ./fui.py -
> 3. ls *.a | ./fui.py *.b -
>
> Ciao,
> Marc 'BlackJack' Rintsch

If X.a X.b Y.a Y.b are all files whose contents are to be processed
then
To process all files:
./fui.py *.a *.b
Or:
./fui.py `ls *.a *.b`
To process one file from a pipe unix usually does:
cat X.a | ./fui.py -
To get the filenames from stdin would usually need a command line
switch telling fui.py to read a file *list* from stdin. For verilog
simulators for example you have the -f switch that says insert further
command line arguments from the file name in the next argument, so you
could do:
ls *.a | ./fui.py -f - *.b
For equivalent functionality to my first example.

- Paddy.

donn

1/24/2008 4:03:00 PM

> Try the fileinput module.
I did give the fileinput module a go, but I can't find much info on it and
the help is ... well, it's python help ;)

> in goes to its stdin where it is processed if it has an argument of -
> fileinput works that way
Okay, I did think of the dash, but did not know how to handle it. Is it a
bash thing or will that dash get passed into the args? (I am using getopt
to parse the options and args)

> which would work for ls and a python program using the fileinput
> module.
Any examples of fileinput (that do not open each file) would be great!
(I'll go searching now anyway)

Thanks,
\d

donn

1/24/2008 4:13:00 PM

Paddy wrote:
> ls *.a | ./fui.py -f - *.b
To be sure I grok this: I am seeing the single dash as a placeholder for
where all the piped filenames will go, so *.b happens after *.a has been
expanded and they all get fed to -f, right?

I'm also guessing you mean that I should detect the single dash and then go
look for stdin at that point. How do I detect a lack of stdin?

Thanks,
\d

Paddy

1/24/2008 4:39:00 PM

On Jan 24, 4:02 pm, Donn Ingle <donn.in...@gmail.com> wrote:
> > Try the fileinput module.
>
> I did give the fileinput module a go, but I can't find much info on it and
> the help is ... well, it's python help ;)

Try http://effbot.org/librarybook/fil...

>
> > in goes to its stdin where it is processed if it has an argument of -
> > fileinput works that way
>
> Okay, I did think of the dash, but did not know how to handle it. Is it a
> bash thing or will that dash get passed into the args? (I am using getopt
> to parse the options and args)

- gets passed in and fileinput handles it.

>
> > which would work for ls and a python program using the fileinput
> > module.
>
> Any examples of fileinput (that do not open each file) would be great!
> (I'll go searching now anyway)
fileinput is set to process each file a line at a time unfortunately.

>
> Thanks,

Your welcome :-)

- Paddy.

donn

1/24/2008 4:59:00 PM

Paddy wrote:
> fileinput is set to process each file a line at a time unfortunately.
Wow. So there seems to be no solution to my OP. I'm amazed, I would have
thought a simple list of strings, one from stdin and one from the args,
would be easy to get.

I *really* don't want to open each file, that would be insane.

Perhaps I shall have to forgo the stdin stuff then, after all.

\d

donn

1/24/2008 5:03:00 PM

> wget -i -
> it doesn't do anything, just waits for your input. Your applications
> probably should behave the same.
Okay, that works for me.

> Paddy wrote:
> > ls *.a | ./fui.py -f - *.b
> It doesn't seem to me that -f parameter is necessary for your
> application.
Yes and no, I have another option that needs to take a variable number of
args.

> It should treat all the arguments as the filenames,
> shouldn't it? And when one of the filenames is -, just try to read
> stdin.
I have tested getopt and it strips the lone '-' out. I can get it from
sys.argv, but then I am really doing more parsing than I want to. It's a
tricky job this. I think I will look in sys.argv, if I find a single dash the
I will replace that element in the list with whatever comes from stdin. Then
I'll pass all of it to getopt.

Thanks for the help.
\d

--
When you allow legends to rule your life, your world is based on fiction
-- Segio Aragones (Groo the Wanderer Number 99)

Fonty Python and other dev news at:
http://otherwiseingle.blo...

Hexamorph

1/24/2008 5:53:00 PM

Donn Ingle wrote:
> Paddy wrote:
>> fileinput is set to process each file a line at a time unfortunately.
> Wow. So there seems to be no solution to my OP. I'm amazed, I would have
> thought a simple list of strings, one from stdin and one from the args,
> would be easy to get.
>
> I *really* don't want to open each file, that would be insane.
>
> Perhaps I shall have to forgo the stdin stuff then, after all.
>

Hi!

I'm not sure if I completely get what you want, but what's about this:

#!/usr/bin/python

import sys

filelist = []
with_stdin=0

if len(sys.argv) > 1:
for file in sys.argv[1:]:
if file == "-":
with_stdin=1
continue
filelist.append(file)
else:
with_stdin=1

if with_stdin:
for file in sys.stdin:
filelist.append(file)

for file in filelist:
print "Processing file: %s" % file

It's a bit clumsy, but seems to do what I guess you want.

HTH

donn

1/24/2008 7:44:00 PM

Thanks for the tips, I'll decode and try 'em all out.

> Ah yes, Groo. Ever wonder who would win if Groo and Forrest Gump fought
> each other?
Heh ;) I reckon they'd both die laughing. Be fun to watch -- if anyone else
survived!

\d

--
"A computer without Windows is like chocolate cake without mustard."
-- Anonymous Coward /.

Fonty Python and other dev news at:
http://otherwiseingle.blo...

comp.lang.python

piping into a python script

donn

Marc 'BlackJack' Rintsch

Paddy

Paddy

donn

donn

Paddy

donn

donn

Hexamorph

donn

x Login to ForumsZone