Asp Forum - Ruby code generation: string vs. AST based (2.0 or beyond

Its Me

11/10/2004 6:42:00 AM

Code generation in ruby is great.

The only thing I find is that, since it is essentially all string
manipulation prior to eval, none of it is executable (or testable!) until it
is eval'd. So I find myself writing straight code, testing it, then mentally
abstracting out the string patterns in that code, then using those in my
code generation.

Reading those string patterns more than a day later makes my brain hurt.

What are the chances, somewhere in or beyond 2.0, of code generation being
based on something more like the following:
- write and test executable code
- get access to an AST of that code itself
- implement code generation by cloning/modifying the AST clone

This would require a published AST, some manipulations on it, and an eval
that takes ASTs.

Thanks.

7 Answers

Hal E. Fulton

11/10/2004 7:22:00 AM

itsme213 wrote:
> Code generation in ruby is great.
>
> The only thing I find is that, since it is essentially all string
> manipulation prior to eval, none of it is executable (or testable!) until it
> is eval'd. So I find myself writing straight code, testing it, then mentally
> abstracting out the string patterns in that code, then using those in my
> code generation.
>
> Reading those string patterns more than a day later makes my brain hurt.
>
> What are the chances, somewhere in or beyond 2.0, of code generation being
> based on something more like the following:
> - write and test executable code
> - get access to an AST of that code itself
> - implement code generation by cloning/modifying the AST clone
>
> This would require a published AST, some manipulations on it, and an eval
> that takes ASTs.

I'm very much in favor of that. I think it is planned, but I don't think
the plan is mature yet.

Hal

Eric Hodel

11/10/2004 6:43:00 PM

On Nov 9, 2004, at 10:43 PM, itsme213 wrote:

> Code generation in ruby is great.
>
> The only thing I find is that, since it is essentially all string
> manipulation prior to eval, none of it is executable (or testable!)
> until it
> is eval'd. So I find myself writing straight code, testing it, then
> mentally
> abstracting out the string patterns in that code, then using those in
> my
> code generation.
>
> Reading those string patterns more than a day later makes my brain
> hurt.
>
> What are the chances, somewhere in or beyond 2.0, of code generation
> being
> based on something more like the following:
> - write and test executable code
> - get access to an AST of that code itself
> - implement code generation by cloning/modifying the AST clone

Not released just yet, but using ParseTree (a soon to be released
RubyForge project), Ryan Davis and myself are working on a something
similar. We have loads of cool tools to help us munge the AST to do
horrible things.

(If you've been hanging out on #ruby-lang, you probably know what that
project is.)

> This would require a published AST, some manipulations on it, and an
> eval
> that takes ASTs.

Well, you could probably emit Ruby code instead from the modified AST
instead, then run the munged Ruby code through the interpreter in a
second stage...

Tobias Peters

11/10/2004 9:21:00 PM

itsme213 wrote:
> Code generation in ruby is great.
>
> The only thing I find is that, since it is essentially all string
> manipulation prior to eval, none of it is executable (or testable!) until it
> is eval'd. So I find myself writing straight code, testing it, then mentally
> abstracting out the string patterns in that code, then using those in my
> code generation.
>
> Reading those string patterns more than a day later makes my brain hurt.
>
> What are the chances, somewhere in or beyond 2.0, of code generation being
> based on something more like the following:
> - write and test executable code
> - get access to an AST of that code itself
> - implement code generation by cloning/modifying the AST clone
>
> This would require a published AST, some manipulations on it, and an eval
> that takes ASTs.

It would be an exciting feature. But I sriously doubt that AST
manipulations will be less brain-hurting.

Tobias

Its Me

11/10/2004 10:15:00 PM

"Tobias Peters" <tpeters@invalid.uni-oldenburg.de> wrote i

> > This would require a published AST, some manipulations on it, and an
eval
> > that takes ASTs.
>
> It would be an exciting feature. But I sriously doubt that AST
> manipulations will be less brain-hurting.
>
> Tobias

It's not trivial, but it is a whole bunch closer to my mental model than
string manipulation.

It also gives Ruby a nice push in the direction of MDA-like approaches.
Let's see:

- I can build domain-specific languages in Ruby.

- I'm asking to get a standard AST representation of programs in those
languages.

- If I write my language-definition language itself in Ruby (e.g. racc), I
also have standard access to the language definition of the domain-specific
language itself.

So I can write generic tools (e.g. transforms, editors, ...) that leverage
the language definitions.

We could have a very promising combination of MDA and DSLs.

Eric Hodel

11/11/2004 12:51:00 AM

Lothar Scholz

11/11/2004 1:25:00 AM

Hello Eric,

EH> On Nov 9, 2004, at 10:43 PM, itsme213 wrote:

>> Code generation in ruby is great.
>>
>> The only thing I find is that, since it is essentially all string
>> manipulation prior to eval, none of it is executable (or testable!)
>> until it
>> is eval'd. So I find myself writing straight code, testing it, then
>> mentally
>> abstracting out the string patterns in that code, then using those in
>> my
>> code generation.
>>
>> Reading those string patterns more than a day later makes my brain
>> hurt.
>>
>> What are the chances, somewhere in or beyond 2.0, of code generation
>> being
>> based on something more like the following:
>> - write and test executable code
>> - get access to an AST of that code itself
>> - implement code generation by cloning/modifying the AST clone

EH> Ok, ParseTree is now released, see ruby-talk:119807. If you need
EH> help/ideas pop into irc://irc.freenode.net/#ruby2c.

But he would still need a "BuildTree".

--
Best regards, emailto: scholz at scriptolutions dot com
Lothar Scholz http://www.ru...
CTO Scriptolutions Ruby, PHP, Python IDE 's

Eric Hodel

11/11/2004 1:44:00 AM

On Nov 10, 2004, at 5:24 PM, Lothar Scholz wrote:

> Hello Eric,
>
> EH> On Nov 9, 2004, at 10:43 PM, itsme213 wrote:
>
>>> What are the chances, somewhere in or beyond 2.0, of code generation
>>> being
>>> based on something more like the following:
>>> - write and test executable code
>>> - get access to an AST of that code itself
>>> - implement code generation by cloning/modifying the AST clone
>
> EH> Ok, ParseTree is now released, see ruby-talk:119807. If you need
> EH> help/ideas pop into irc://irc.freenode.net/#ruby2c.
>
> But he would still need a "BuildTree".

You could emit Ruby code out the other end, instead of reloading the
AST. Then run the munged, emitted ruby code back through Ruby. It
wouldn't be much different than using gcc.

Emitting Ruby code then running it through the interpreter is easier
than putting the munged AST back into a live Ruby.

comp.lang.ruby

Ruby code generation: string vs. AST based (2.0 or beyond

Its Me

Hal E. Fulton

Eric Hodel

Tobias Peters

Its Me

Eric Hodel

Lothar Scholz

Eric Hodel

x Login to ForumsZone