Asp Forum - Syntax extensions - question

samuel_x_winters_x

3/25/2005 9:37:00 PM

Context: using Ruby for domain-specific languages. Ruby's properties of
dynamic + reflective + blocks are GREAT for the DSL semantics, so this
is about allowing DSL syntax.

I have written a few "macro" like methods in the Ruby way, usually
against some meta-object such as Class or Module. Blocks are hugely
helpful, in large part because they are not evaluated right away. But
everything else is evaluated, so I need to do things like write :abc or
"abc" in places where it would be more natural to just write abc. Plus
it is hard to do things like manipulate variable bindings from such
macros.

At the same time there are various convenient syntax 'extensions' like
%w{...}. And there is really nice work on Syntax and Ripper to make
ruby ASTs accessible.

I was wondering: what would be the interest, value, and feasibility of
something like this:

Suppose there was a Grammer type (it can parse, returns something like
an AST). And some distinguished syntax to introduce a grammar object
(I'll use "!!" here, some variant of %w might be better) so that an
expression like this:

X!!{ ..... }

would result at syntax analysis time in:
1. call X.grammer # expect a Grammar object, g
2. pass everything between {...} to g.parse
3. expect (a fragment of) a Ruby AST back, t
4. call t.process (or whatever is Ruby's way to combine and analyze its
AST fragments and eventually produce classes, methods, etc. Probably a
richer protocol than a single method)
5. keep going from just after "}"

Note that step 3 need not return a "standard" AST, it could just return
something that responded appropriately to "process" in #4, so this need
not be standard "macro" like expansion. However, I would expect to very
often use a standard AST.

Most likely the "X" will have to be a constant. And most likely this
will require some standardization of Ruby's (duck-typed) ways of
handling ASTs.

Result: Can define, use, mix, and combine (fragments of)
domain-specific language with domain-specific syntax.

Thoughts?

4 Answers

ptkwt

3/26/2005 1:39:00 AM

In article <1111786628.337914.310980@l41g2000cwc.googlegroups.com>,
<samuel_x_winters_x@hotmail.com> wrote:
>Context: using Ruby for domain-specific languages. Ruby's properties of
>dynamic + reflective + blocks are GREAT for the DSL semantics, so this
>is about allowing DSL syntax.
>
>I have written a few "macro" like methods in the Ruby way, usually
>against some meta-object such as Class or Module. Blocks are hugely
>helpful, in large part because they are not evaluated right away. But
>everything else is evaluated, so I need to do things like write :abc or
>"abc" in places where it would be more natural to just write abc. Plus
>it is hard to do things like manipulate variable bindings from such
>macros.
>
>At the same time there are various convenient syntax 'extensions' like
>%w{...}. And there is really nice work on Syntax and Ripper to make
>ruby ASTs accessible.
>
>I was wondering: what would be the interest, value, and feasibility of
>something like this:
>
>Suppose there was a Grammer type (it can parse, returns something like
>an AST). And some distinguished syntax to introduce a grammar object
>(I'll use "!!" here, some variant of %w might be better) so that an
>expression like this:
>
> X!!{ ..... }
>
>would result at syntax analysis time in:
>1. call X.grammer # expect a Grammar object, g
>2. pass everything between {...} to g.parse
>3. expect (a fragment of) a Ruby AST back, t
>4. call t.process (or whatever is Ruby's way to combine and analyze its
>AST fragments and eventually produce classes, methods, etc. Probably a
>richer protocol than a single method)
>5. keep going from just after "}"
>
>Note that step 3 need not return a "standard" AST, it could just return
>something that responded appropriately to "process" in #4, so this need
>not be standard "macro" like expansion. However, I would expect to very
>often use a standard AST.
>
>Most likely the "X" will have to be a constant. And most likely this
>will require some standardization of Ruby's (duck-typed) ways of
>handling ASTs.
>
>Result: Can define, use, mix, and combine (fragments of)
>domain-specific language with domain-specific syntax.
>
>Thoughts?
>

Interesting ideas. It would be very nice to be able to access the AST as
an object and have this builtin to Ruby. I would certainly have uses for
this sort of thing.

....however, I suspect that this might smack of macros (as in Lisp) in
Ruby and Matz tends to frown upon that sort of thing. I think there has
been some talk of allowing greater access to the AST in Ruby 2.0 (Rite),
though.

BTW: have you taken a look at ParseTree?
http://blog.zenspider.com/archives/parsetree/...

Phil

Its Me

3/26/2005 7:23:00 AM

"Phil Tomson" <ptkwt@aracnet.com> wrote in message
> <samuel_x_winters_x@hotmail.com> wrote:

> >Suppose there was a Grammer type (it can parse, returns something like
> >an AST). And some distinguished syntax to introduce a grammar object
> >(I'll use "!!" here, some variant of %w might be better) so that an
> >expression like this:
> >
> > X!!{ ..... }
> >
> >would result at syntax analysis time in:
> >1. call X.grammer # expect a Grammar object, g
> >2. pass everything between {...} to g.parse
> >3. expect (a fragment of) a Ruby AST back, t
> >4. call t.process (or whatever is Ruby's way to combine and analyze its
> >AST fragments and eventually produce classes, methods, etc. Probably a
> >richer protocol than a single method)
> >5. keep going from just after "}"

This sounds most interesting (though I had to think about it quite a bit). I
can define a module that provided a set of classes + a syntax for using them
(or even multiple alternate concrete syntaxes for same abstract language).

module InteriorDecoration
class Item ... end
class Room ... end

def self.grammar
# custom syntax to manipulate Items in Rooms
end
end

InteriorDecoration!! {
kitchen items: fridge, microwave, oven, table
# Ruby AST syntax: kitchen = Room.new
# fridge = Item.new ...
move the fridge to the NE corner of the kitchen
# Ruby AST syntax: fridge.move :to=>kitchen.corner(:NE); kitchen
}

Or:
UML_Sequence_Diagram!! {
testing a
a: A, b: B, c:C
initially b.x = 5
a->b.foo(10) {
b->c.bar(50)
d = b->c.baz
return d
}
finally a.y = d, d.val = 500
}
Ruby AST: a Unit Test with several assertions.

The grammars as objects could themselves extend or compose other grammers.
The DSL grammar could even be defined to fall-through to straight Ruby
syntax where needed i.e. normal ruby code is like Ruby!!{ ....}).

> >Note that step 3 need not return a "standard" AST, it could just return
> >something that responded appropriately to "process" in #4, so this need
> >not be standard "macro" like expansion. However, I would expect to very
> >often use a standard AST.

Hmm. While that could be nice, it may be tricky as something like Rite may
be quite coupled to the concrete structure of the AST.

> Interesting ideas. It would be very nice to be able to access the AST as
> an object and have this builtin to Ruby. I would certainly have uses for
> this sort of thing.

Likewise.

Some months ago I had proposed in RCR http://rcrchive.net/rc... that
individual methods freely define their invocation syntax (mix-fix, in-fix,
etc.). In hindsight I don't think that would work with Ruby's dynamic
typing, although perhaps OK with static typing. But this approach is doable
(with X as a parse-time constant) and would be a really nice thing for DSLs.

> ...however, I suspect that this might smack of macros (as in Lisp) in
> Ruby and Matz tends to frown upon that sort of thing.

Hmmm. I see some connection, but think there is a clearer "language" unit
and grammar structure here.

gabriele renzi

3/26/2005 9:25:00 AM

samuel_x_winters_x@hotmail.com ha scritto:

<snipall>
What's wrong with:

Grammar.new %{
foo: bau
bar: miao-102
}

the %{} literal gives you a string, and you can parse it with the full
power of regexp/racc/cocorb/whatever in Grammar#initialize.

samuel_x_winters_x

3/26/2005 5:41:00 PM

Yes, that is a reasonable way to get the string into the grammar. Could
you explain if / how it would deal with #3, #4 (and hence #5)?

3. expect (a fragment of) a Ruby AST back, t
4. call t.process (or whatever is Ruby's way to combine and analyze its
AST fragments and eventually produce classes, methods, etc. Probably a
richer protocol than a single method)
5. keep going from just after "}"

I want the DSL fragment to contribute to building the overall AST and
subsequently processing it. e.g. even use a DSL fragment inside a
normal Ruby method. Any suggestions in that direction?

comp.lang.ruby

Syntax extensions - question

samuel_x_winters_x

ptkwt

Its Me

gabriele renzi

samuel_x_winters_x

x Login to ForumsZone