[lnkForumImage]
TotalShareware - Download Free Software

Confronta i prezzi di migliaia di prodotti.
Asp Forum
 Home | Login | Register | Search 


 

Forums >

comp.programming

compiling python

kenobi

1/9/2015 11:36:00 PM

can someone explain if python can be compiled (to exe) if not why cant be compiled

afaik python is not quite compiled but more like
interpreter+bytecode contained in exe form..
really there is something that disallows compilation of script languages? I do not
understand this thing, what is really the thing that makes scripts not compilable?
17 Answers

Bartc

1/9/2015 11:50:00 PM

0

On 09/01/2015 23:36, fir wrote:
> can someone explain if python can be compiled (to exe) if not why cant be compiled

Try comp.lang.python. That's quite active.

> afaik python is not quite compiled but more like
> interpreter+bytecode contained in exe form..
> really there is something that disallows compilation of script languages? I do not
> understand this thing, what is really the thing that makes scripts not compilable?

You can certainly create a self-contained executable, but it might not
be any faster than running the interpreter. Take this bit of code:

a=b+c

In a scripting language with dynamic types, what would you compile it
to? You don't know what the types are! They could be integers, floats,
strings, lists, tuples, ranges, functions, classes or any combination.
You have to sort this all out at runtime, which is not going to be
helped by compiling. Especially with Python where /everything/ is
dynamic and can change from one line to the next.

(There are new techniques for speeding up interpreted code, but they are
different from the traditional methods for compiling static languages.)

--
Bartc


kenobi

1/9/2015 11:59:00 PM

0

W dniu sobota, 10 stycznia 2015 00:50:35 UTC+1 uzytkownik Bart napisal:
> On 09/01/2015 23:36, fir wrote:
> > can someone explain if python can be compiled (to exe) if not why cant be compiled
>
> Try comp.lang.python. That's quite active.
>
> > afaik python is not quite compiled but more like
> > interpreter+bytecode contained in exe form..
> > really there is something that disallows compilation of script languages? I do not
> > understand this thing, what is really the thing that makes scripts not compilable?
>
> You can certainly create a self-contained executable, but it might not
> be any faster than running the interpreter. Take this bit of code:
>
> a=b+c
>
> In a scripting language with dynamic types, what would you compile it
> to? You don't know what the types are! They could be integers, floats,

i think in most cases (code floe paths) you know, even if you dont know, they are one of the few probably so you could combine different versions and call the appriopriate once with strings once with integers..
no? maybe they are to lazy to do that? 9wrote compiles as maybe it is a bit of work? idont know

this is the only reason?
still i dont see it clearly but it seem that teopretically with an amount of worc it could be compiled..


> strings, lists, tuples, ranges, functions, classes or any combination.
> You have to sort this all out at runtime, which is not going to be
> helped by compiling. Especially with Python where /everything/ is
> dynamic and can change from one line to the next.
>
> (There are new techniques for speeding up interpreted code, but they are
> different from the traditional methods for compiling static languages.)
>
> --
> Bartc

robertwessel2@yahoo.com

1/10/2015 12:13:00 AM

0

On Fri, 9 Jan 2015 15:36:09 -0800 (PST), fir <profesor.fir@gmail.com>
wrote:

>can someone explain if python can be compiled (to exe) if not why cant be compiled
>
>afaik python is not quite compiled but more like
>interpreter+bytecode contained in exe form..
>really there is something that disallows compilation of script languages? I do not
>understand this thing, what is really the thing that makes scripts not compilable?


There are a number of Python compilers and JITs. A partial list:

https://en.wikipedia.org/wiki/List_of_Python_software#Python_imple...

Which doesn't mean that some of the more dynamic things you can do in
Python are actually going to get faster if you compile them.

I know of someone using PyPy for a reasonably large project, and they
seem happy, but I've no personal experience.

kenobi

1/10/2015 12:28:00 AM

0

W dniu sobota, 10 stycznia 2015 01:08:54 UTC+1 uzytkownik robert...@yahoo.com napisal:
> On Fri, 9 Jan 2015 15:36:09 -0800 (PST), fir <profesor.fir@gmail.com>
> wrote:
>
> >can someone explain if python can be compiled (to exe) if not why cant be compiled
> >
> >afaik python is not quite compiled but more like
> >interpreter+bytecode contained in exe form..
> >really there is something that disallows compilation of script languages? I do not
> >understand this thing, what is really the thing that makes scripts not compilable?
>
>
> There are a number of Python compilers and JITs. A partial list:
>
> https://en.wikipedia.org/wiki/List_of_Python_software#Python_imple...
>
> Which doesn't mean that some of the more dynamic things you can do in
> Python are actually going to get faster if you compile them.
>
> I know of someone using PyPy for a reasonably large project, and they
> seem happy, but I've no personal experience.

dont know if this is real static compilation, or just in-proces interpreter or some half-way,many depends on it

Bartc

1/10/2015 11:46:00 AM

0

On 09/01/2015 23:59, fir wrote:
> W dniu sobota, 10 stycznia 2015 00:50:35 UTC+1 użytkownik Bart napisaÅ?:

>> a=b+c
>>
>> In a scripting language with dynamic types, what would you compile it
>> to? You don't know what the types are! They could be integers, floats,
>
> i think in most cases (code floe paths) you know, even if you dont know, they are one of the few probably so you could combine different versions and call the appriopriate once with strings once with integers..
> no? maybe they are to lazy to do that? 9wrote compiles as maybe it is a bit of work? idont know

If your aim is to get a faster Python, then there have been plenty of
attempts, with varying success. Some might use type inference. (But
PyPy, the most impressive I've seen, uses rather different methods.)

>
> this is the only reason?
> still i dont see it clearly but it seem that teopretically with an
amount of worc it could be compiled..

Perhaps you'd like to have go; source codes for Python are readily
available.

(I've tried compiling for my own interpreter projects: translating
naively to C source then compiling that. But it was slower than just
interpreting! I think partly because it was no longer practical to use
function inlining, and the code was more sprawling compared with a
compact interpreter core.)

--
Bartc

kenobi

1/10/2015 3:10:00 PM

0

W dniu sobota, 10 stycznia 2015 12:45:39 UTC+1 uzytkownik Bart napisal:
> On 09/01/2015 23:59, fir wrote:
> > W dniu sobota, 10 stycznia 2015 00:50:35 UTC+1 uzytkownik Bart napisal:
>
> >> a=b+c
> >>
> >> In a scripting language with dynamic types, what would you compile it
> >> to? You don't know what the types are! They could be integers, floats,
> >
> > i think in most cases (code floe paths) you know, even if you dont know, they are one of the few probably so you could combine different versions and call the appriopriate once with strings once with integers..
> > no? maybe they are to lazy to do that? 9wrote compiles as maybe it is a bit of work? idont know
>
> If your aim is to get a faster Python, then there have been plenty of
> attempts, with varying success. Some might use type inference. (But
> PyPy, the most impressive I've seen, uses rather different methods.)
>
> >
> > this is the only reason?
> > still i dont see it clearly but it seem that teopretically with an
> amount of worc it could be compiled..
>
> Perhaps you'd like to have go; source codes for Python are readily
> available.
>
> (I've tried compiling for my own interpreter projects: translating
> naively to C source then compiling that. But it was slower than just
> interpreting! I think partly because it was no longer practical to use
> function inlining, and the code was more sprawling compared with a
> compact interpreter core.)
>
later aim is to get more knowledge, read sources, mybe wrote script compiler myself etc;

now i would like just to understand in robust way what is a trouble of compiling scripting language, is there some or maybe is there none

maybe a bit find to inspect, as i felt tired a bit

Kaz Kylheku

1/10/2015 7:19:00 PM

0

On 2015-01-09, BartC <bc@freeuk.com> wrote:
> On 09/01/2015 23:36, fir wrote:
>> can someone explain if python can be compiled (to exe) if not why cant be compiled
>
> Try comp.lang.python. That's quite active.
>
>> afaik python is not quite compiled but more like
>> interpreter+bytecode contained in exe form..
>> really there is something that disallows compilation of script languages? I do not
>> understand this thing, what is really the thing that makes scripts not compilable?
>
> You can certainly create a self-contained executable, but it might not
> be any faster than running the interpreter. Take this bit of code:
>
> a=b+c
>
> In a scripting language with dynamic types, what would you compile it
> to? You don't know what the types are! They could be integers, floats,
> strings, lists, tuples, ranges, functions, classes or any combination.
> You have to sort this all out at runtime, which is not going to be
> helped by compiling.

Yes, it is still helped by compiling.

Firstly, the compiler knows certain facts, such as that a, b, and c are (let us
say) lexical variables and can generate fast references to them.

A pure interpreter, by contrast, has to look these things up: it looks at a
and wonders, is that a local? A global? It has to do some environmental lookup.

It also might take advantage of knowing that + is a language built-in that
cannot be redefined (let's say).

The compiled code just does something like:

load R0, offset_b(R29) ;; R29 is the current closure
load R1, offset_c(R23)
call builtin_plus ;; result is in R0
store offset_a(R29), R0 ;; store in a

Though builtin_plus does the work, this compiled code saves cycles
compared to the actions which have to be carried out by pure
interpretation, which basically does:

(eval '(set a (+ a b)) current-env)

eval has to look at "set", invoke the right operator for that,
then set has to analyze the target place a, looking up its location
(probably by calling a function). Then it has to recursively call

(eval '(+ a b) current-env)

to get the value and so on.


basically, you're neglecting to look at this from the point of view that
a, b and c can be regarded as variables having the generic type "value",
and that the expression is basically just like C:

a = builtin_plus(b, c);

you wouldn't say that compiling a function call and assignment isn't
beneficial, right? Even if a, b and c are pointers, and the function
actually has to be called.

On top of that, advanced dynamic languages support optimization. If you
promise to the compiler, via declarations, that a, b, and c have certain
types, then it can generate better code.

Bartc

1/10/2015 10:03:00 PM

0

On 10/01/2015 19:19, Kaz Kylheku wrote:
> On 2015-01-09, BartC <bc@freeuk.com> wrote:

>> a=b+c
>>
>> In a scripting language with dynamic types, what would you compile it
>> to? You don't know what the types are! They could be integers, floats,
>> strings, lists, tuples, ranges, functions, classes or any combination.
>> You have to sort this all out at runtime, which is not going to be
>> helped by compiling.
>
> Yes, it is still helped by compiling.
>
> Firstly, the compiler knows certain facts, such as that a, b, and c are (let us
> say) lexical variables and can generate fast references to them.
>
> A pure interpreter, by contrast, has to look these things up: it looks at a
> and wonders, is that a local? A global? It has to do some environmental lookup.

If we're talking about languages such as Python, then there will already
be a compilation stage, usually transparent, that converts source code
to bytecode. I assumed the OP was talking about compilation to native code.

With bytecode, it will already know much of that information.

> basically, you're neglecting to look at this from the point of view that
> a, b and c can be regarded as variables having the generic type "value",
> and that the expression is basically just like C:
>
> a = builtin_plus(b, c);
>
> you wouldn't say that compiling a function call and assignment isn't
> beneficial, right? Even if a, b and c are pointers, and the function
> actually has to be called.

It's not necessarily beneficial. Most of the work is going to be done
inside that builtin_plus() function (say a C function that is a
permanent part of the implementation, while the a=builtin_plus(b,c) part
is the result of a 'compiling' a bit of script code into C).

The only advantage might be in not having a dispatch loop that finds out
what the next operation might be.

However, imagine you had 100 lines of a=b+c, then you will have 100
successive calls to a=builtin_plus(b,c), these are actual calls with
parameters needing pushing and so on, which are not going to be
practical to inline.

Now compare with a pure bytecode interpreter implemented with a switch:

while (1) {
switch (*pcptr) {
....
case kadd: do_add(); break;
....
}
}

This is an actual example, where do_add() corresponds to
a=builtin_plus(b,c) (this uses a stack model so there will be extra
bytecode instructions to push the operands and pop the result).

The compiler will likely inline those handler functions like do_add()
since there is only one call to each. It will spend those 100 lines in
this loop, with no function calls and no parameter passing.

> On top of that, advanced dynamic languages support optimization. If you
> promise to the compiler, via declarations, that a, b, and c have certain
> types, then it can generate better code.

That wouldn't count as pure Python then (but as a different version,
perhaps CPython). When I tried adding type info to my own language
efforts, the modest gains weren't worth the extra complexity or losing
the elegance of the language's variant types.

--
Bartc

kenobi

1/11/2015 6:52:00 PM

0

W dniu sobota, 10 stycznia 2015 23:02:51 UTC+1 uzytkownik Bart napisal:
> On 10/01/2015 19:19, Kaz Kylheku wrote:
> > On 2015-01-09, BartC <bc@freeuk.com> wrote:
>
> >> a=b+c
> >>
> >> In a scripting language with dynamic types, what would you compile it
> >> to? You don't know what the types are! They could be integers, floats,
> >> strings, lists, tuples, ranges, functions, classes or any combination.
> >> You have to sort this all out at runtime, which is not going to be
> >> helped by compiling.
> >
> > Yes, it is still helped by compiling.
> >
> > Firstly, the compiler knows certain facts, such as that a, b, and c are (let us
> > say) lexical variables and can generate fast references to them.
> >
> > A pure interpreter, by contrast, has to look these things up: it looks at a
> > and wonders, is that a local? A global? It has to do some environmental lookup.
>
> If we're talking about languages such as Python, then there will already
> be a compilation stage, usually transparent, that converts source code
> to bytecode. I assumed the OP was talking about compilation to native code.
>
> With bytecode, it will already know much of that information.
>
> > basically, you're neglecting to look at this from the point of view that
> > a, b and c can be regarded as variables having the generic type "value",
> > and that the expression is basically just like C:
> >
> > a = builtin_plus(b, c);
> >
> > you wouldn't say that compiling a function call and assignment isn't
> > beneficial, right? Even if a, b and c are pointers, and the function
> > actually has to be called.
>
> It's not necessarily beneficial. Most of the work is going to be done
> inside that builtin_plus() function (say a C function that is a
> permanent part of the implementation, while the a=builtin_plus(b,c) part
> is the result of a 'compiling' a bit of script code into C).
>
> The only advantage might be in not having a dispatch loop that finds out
> what the next operation might be.
>
> However, imagine you had 100 lines of a=b+c, then you will have 100
> successive calls to a=builtin_plus(b,c), these are actual calls with
> parameters needing pushing and so on, which are not going to be
> practical to inline.
>
> Now compare with a pure bytecode interpreter implemented with a switch:
>
> while (1) {
> switch (*pcptr) {
> ....
> case kadd: do_add(); break;
> ....
> }
> }
>
> This is an actual example, where do_add() corresponds to
> a=builtin_plus(b,c) (this uses a stack model so there will be extra
> bytecode instructions to push the operands and pop the result).
>
> The compiler will likely inline those handler functions like do_add()
> since there is only one call to each. It will spend those 100 lines in
> this loop, with no function calls and no parameter passing.
>
> > On top of that, advanced dynamic languages support optimization. If you
> > promise to the compiler, via declarations, that a, b, and c have certain
> > types, then it can generate better code.
>
> That wouldn't count as pure Python then (but as a different version,
> perhaps CPython). When I tried adding type info to my own language
> efforts, the modest gains weren't worth the extra complexity or losing
> the elegance of the language's variant types.
>
> --
> Bartc

so you think that efficiency loss (and python is considered slow) is taken not from lack of compilation but form python shaped structure
of executable content? 9that i dont know, is doing operations through runtime layers?)

Kaz Kylheku

1/11/2015 7:24:00 PM

0

On 2015-01-10, BartC <bc@freeuk.com> wrote:
> On 10/01/2015 19:19, Kaz Kylheku wrote:
>> On top of that, advanced dynamic languages support optimization. If you
>> promise to the compiler, via declarations, that a, b, and c have certain
>> types, then it can generate better code.
>
> That wouldn't count as pure Python then (but as a different version,
> perhaps CPython). When I tried adding type info to my own language
> efforts, the modest gains weren't worth the extra complexity or losing
> the elegance of the language's variant types.

I see, and whatever you've been able to achieve is representative of
what is possible. Nay, the upper bound.