[lnkForumImage]
TotalShareware - Download Free Software
Confronta i prezzi di migliaia di prodotti.
Asp Forum
 Home | Login | Register | Search 


 

Forums >

comp.lang.ruby

[ANN] RubyMacros 0.1.0 Released

Caleb Clausen

10/24/2008 8:24:00 PM

RubyMacros is a lisp-like macro pre-processor for Ruby. More than just a
purely textual substitution scheme, RubyMacros can manipulate and morph
Ruby parse trees (in the form of RedParse Nodes) at parse time in just about
any way you see fit.

Macros are programmed in ruby itself. And since parse trees are represented
in RedParse format, they're easier to use (programatically) and more object-
oriented than other available ruby parsetree formats. (RedParse Node format
is actually designed to be straightforward to use and to represent the
structure of ruby source code very closely.)

== Benefits:
* Powerful and easy metaprogramming
* Create better DSLs
* Manipulate syntax trees to suit yourself
* Access local variables and other caller context unavailable to methods
* Macros as inline methods: should be slightly faster than equivalent methods

== Drawbacks:
Although in theory already as powerful as lisp macros, the current
implementation has a number of problems which added together make it merely
a proof of concept or toy at this point:
* pre-processing is very, very slow (because of RedParse)
* macro calls must be inside some sort of method;
* straight out macro calls at the top level won't work
* macros can't have blocks or receivers
* some ruby syntax is unsupported in files using macros
* files using macros must be loaded via Macro.require;
* Kernel#require will not recognize macros
* RedParse Node tree format will be changing slightly
* macros cannot be scoped
* no variable (or other) hygiene

== Requirements:
RubyMacros requires RedParse.

== Install:
gem install rubymacros

== Examples:
macro simple(a,b)
:(^a+^b)
end
def simple_user
p simple(1,2) #prints 3
end

#loop as a macro, should be a bit faster than the #loop method
macro loop(body)
:(while true
^body
end
)
end

#for more examples, see the examples/ directory

== New Syntax:
I have invented 3 new syntactical constructions in order to allow reasonably
easy to use macros. Macros themselves look just like methods except that
'macro' instead of 'def' is used to start the macro definition off. A form
literal is an expression surrounded by ':(' and ')'. The form escape operator
is '^'. '^' is a unary operator of fairly high precedence.

== Forms and Form Escapes:
Forms are an essential adjunct to macros. Forms represent quoted source
code, which has been parsed but not evaled yet. When a form literal is
executed, it returns a RedParse::Node representing the parse tree for the
enclosed source code. Within a form literal, a ^, used as a unary operator,
will escape the expression it controls, so that instead of being part of the
form's data, it is executed at the same time as the form literal, and the
result of an escaped expression (which should be a Node) is interpolated
into the form at that point. The whole effect is much like that of string
interpolations (#{}) inside string literals.

== How Macros Work
Typically, macros return a single form literal, which contains form escape
expressions within it which make use of the macro's parameters. However,
macro bodies may contain anything at all; more complicated macros will
likely not contain any forms. (Likewise, form literals may be used outside
macros, but the utility of doing so may be minimal.)

At parse time (well, really at method definition time, but in effect it's
much the same thing) method bodies are scanned for callsites which have the
names of known macros. When such a call is found, it is expanded as follows.
The parsetrees for the arguments to the callsite are passed as arguments to
the macro. The macro is expected to return a parsetree, which replaces the
macro callsite in the parsetree which contained it.

== License:
LGPL
13 Answers

Pit Capitain

10/25/2008 8:48:00 AM

0

2008/10/24 Caleb Clausen <vikkous@gmail.com>:
> RubyMacros is a lisp-like macro pre-processor for Ruby.

Sounds interesting. When will you release some code to look at?

Regards,
Pit

ThoML

10/25/2008 9:42:00 AM

0

> When will you release some code to look at?

I was close to asking the very same question. But then I saw "gem
install", so it's available as gem.

Anyway, how does this approach compare to:

http://blog.drewolson.org/2008/06/ruby-and-macros-exper...
http://weblog.raganwald.com/2008/06/macros-hygiene-and-call-by-name-in...

May I humbly ask, how do I run the examples? I tried calling running
the example/*_wrap.rb scripts with ruby18 and ruby19 but only got
Syntax errors:

ruby 1.8.7 (2008-08-11 patchlevel 72) [i386-cygwin]
ruby 1.9.0 (2008-08-26 revision 18849) [i386-cygwin]

Reg Braithwaite

10/26/2008 1:07:00 AM

0

Caleb Clausen wrote:

> Here's the shorter of his two of his attempts to write an andand macro
> that operates like the && operator:
>
> called_by_name(:our_and) { |x,y|
> if temp = x
> y
> else
> temp
> end
> }
>
> Not so bad in itself, but all called_by_name's must be wrapped in a
> 'with', which must get tiresome.

The use of "with" is a deliberate design choice. Rather than making
macros global and "automagical," you state what you are using and where
you are using it. This is somewhat akin to writing 'require
such-and-such' in each ruby source file.

Of course, some people like magic, and if you look at Rails, the
initializers and environment.rb file allow you to sprinkle magic
throughout your project implicitly. My feeling when I designed rewrite
was that that if I started with explicit "with," it would easy to build
implicit into a project or framework later.

> The longer one is too horrible to contemplate.

> Speak for yourself ;-)

> The equivalent in RubyMacros is something like this:
>
> macro andand(a,b)
> :( if temp = ^a
> ^b
> else
> temp
> end
> )
> #or maybe just :( ^a && ^b )
> end

:( ^a && ^b ) is a little too metacircular for my taste, but I put it
to you that rewrite allows you to define your own syntactic replacement
using && if you want to. Now to get more specific. called_by_name is
actually not a way of doing macros, it's a way of writing functions with
call by name semantics.

Rewrite actually provides a facility for code rewriting, which is one
level *above* simple unhygienic macros. A traditional unhygienic macro
is a way of saying "when you see something that looks like a method
call, replace it with the following code, performing substitutions here
and here and here." Rewrite supports this as well as a number of other
arbitrary rewriting rules.

For example, you can say "when you see foo.select { ...blah... }.map {
...blah-blah }, replace it with a single call to .each that performs the
selction and mapping with out iterating over the collection twice.

Now, called_by_name is actually a macro written using rewrite. So it's a
meta-macro. I would say that gievn your example, the macro is better
because it does not "compile" into a function call, whereas anything
built with called_by_name will be rewritten as a function call. If what
you want is the fastest, tightest code, use a ruby macro or use rewrite
to directly rewrite the function call as an if statement.

If you want to compare rewrite and ruby macros more directly, there's a
little thing I wrote called Unhygienic. It does code rewriting "by
example." Now, I use the term "andand" to refer to
http://andand.ruby..., so here is how to write part of that gem
using Rewriting by example:

Unhygienic.from(:receiver, :message, [:parameters]) {
receiver.andand.message(parameters)
}.to {
lambda { |andand_temp|
andand_temp.message(parameters) if andand_temp
}.call(receiver)
}

By the way, I use lambdas a lot in my rewrites to try to alleviate the
pain of Ruby's scoping rules. If you like to live a little more
dangerously (and the example above does), this rule can be made shorter:

Unhygienic.from(:receiver, :message, [:parameters]) {
receiver.andand.message(parameters)
}.to {
temp.message(parameters) if (temp = receiver)
}

Both examples are longer than the macro or called_by_name definitions.
The "from" says "here is a snippet of code where receiver, message, and
parameters are placehilders for an expression, and expression, and a
list of expressions. The "to" says "when you find that, replace it with
this, plugging in the placeholders."

The idea here is that you can use any arbitrary ruby expression, not
just something that looks like a function call. In this case, you are
making something that looks like a method call expand into something
else entirely.

My motivation with Rewrite was very specific: I was trying to show that
we have alternatives to wide scale opening of core classes to implement
DSLs and syntactic abstractions. This goal necessarily encompassed
providing an alternative to existing idioms like #andand or
Symbol#to_proc or #try. The goal of Ruby macros seems to be a little
different, and thus the two gems work in different ways.

Overall, I wish Ruby macros every success and hope that people get
excited about syntactic abstractions.
--
Posted via http://www.ruby-....

ara.t.howard

10/26/2008 9:25:00 PM

0


On Oct 24, 2008, at 2:23 PM, Caleb Clausen wrote:

> == Benefits:
> * Powerful and easy metaprogramming
> * Create better DSLs
> * Manipulate syntax trees to suit yourself
> * Access local variables and other caller context unavailable to
> methods
> * Macros as inline methods: should be slightly faster than
> equivalent methods


only manipulating the syntax tree seems like a real advantage from
here, generating a dsl is already as nearly painless as it could be in
ruby, and methods do in fact have access to local variables and other
caller context

cfp:~ > cat a.rb
def context &block
eval 'a += 40', block
block.call
end

a = 2
context{ p a }
p a



cfp:~ > ruby a.rb
42
42


can you show us something that cannot be done using ruby currently,
which macros make possible?


cheers.


a @ http://codeforp...
--
we can deny everything, except that we have the possibility of being
better. simply reflect on that.
h.h. the 14th dalai lama

Caleb Clausen

11/2/2008 10:40:00 PM

0

On 10/26/08, ara.t.howard <ara.t.howard@gmail.com> wrote:
>
> On Oct 24, 2008, at 2:23 PM, Caleb Clausen wrote:
>
>> == Benefits:
>> * Powerful and easy metaprogramming
>> * Create better DSLs
>> * Manipulate syntax trees to suit yourself
>> * Access local variables and other caller context unavailable to
>> methods
>> * Macros as inline methods: should be slightly faster than
>> equivalent methods
>
>
> only manipulating the syntax tree seems like a real advantage from
> here, generating a dsl is already as nearly painless as it could be in
> ruby, and methods do in fact have access to local variables and other
> caller context

Sorry for the late reply... I thought this thread had died.

You seem to have left out inline methods, for whatever they're worth.

Yes, DSLs in ruby are currently very easy to create and use; I
anticipate that macros will make DSLs slightly harder to write, but
easier to use. Ruby DSLs tend to be very natural looking to users, but
there are often small compromises to usability that won't make much
sense to domain users, such as the need to begin some words with a
colon and the need to use 'do' at certain places in the language for
no apparent reason. If you use macros to define your DSL, it should be
possible to rid oneself of those features. The situation is already
pretty good, but macros will make it slightly better. I say "will",
because, to be honest, at the moment most of the sugary convenience
features needed for nicer DSLs are not present.

As an example of a DSL (perhaps the wrong word in this case...) that
could be written in macros, there is iterate for common lisp:
http://common-lisp.net/project/iterate/doc/...
Iterate is a looping mini-language with special syntax for many common
looping tasks. It looks kind of like what list comprehensions do for
you in python, but more powerful. Now I imagine that something like
this could be written entirely with methods.... but it would be too
slow. (I admit, tho, that I don't understand iterate -- or lisp in
general -- very well. If someone out there wants to correct my
misapprehensions, please feel free.)

> cfp:~ > cat a.rb
> def context &block
> eval 'a += 40', block
> block.call
> end

This is a slick way of getting to your caller's lvars. I would have
passed in a Binding myself, but this way is probably a little cleaner.

But, the caller must pass a block (or binding) in order to make this
work. Sometimes, that's not a problem. Sometimes it is. For instance
(I've run into this) if you want to create an api that works exactly
like Regexp#match, you'll find that it can't be done. #match sets its
caller's $~ (a local variable); methods can't do that. In the past,
I've passed in an optional Binding to handle this case, but
practically speaking, it was a little too clumsy. Using a block
instead is a better idea, but you're still changing the interface used
by your custom #match. The whole point is to re-use your user's
existing knowledge about #match.... if he has to remember, "oh yeah,
and if you use $~ or other matching variables, you have to pass an
extra block to #match", then that's not an effective re-use of
existing knowledge; it might as well be a new interface.

> can you show us something that cannot be done using ruby currently,
> which macros make possible?

A recently requested new feature on ruby-core was __DIR__, which acts
like __FILE__, but returns the directory containing the current source
file. As a macro, that is:

macro __DIR__
:( File.dirname __FILE__ )
end

Now maybe (now that I've seen your block-as-binding trick) you can
actually write this as a method, something like,

def __DIR__(&ctx)
File.dirname(eval("__FILE__",ctx))
end

I have no ruby ATM, and can't check if that works or not, sorry. But
if it does, it will have to be called like __DIR__{}, instead of
__DIR__. I'd find that a little jarring.

Another recent request was a 'with' keyword, which operates like
instance_eval, but only changes the default receiver for code in the
block passed in, and not self as seen by instance variables. I have an
implementation of this as well (in the example directory of
RubyMacros), but for various reasons I'm unsatisfied with it right
now, so I'd rather not post it.

I'm not claiming that either of these macros is actually a good idea;
I'm just trying to illustrate the possible.

It's likely that quite a few of the features for ruby that get
requested could actually be implemented by macros. It's probably
appropriate that most of these requests are rejected; we don't really
need a lot of global changes to the language. However, if users can
write their own macros to scratch some of these itches, that's a
better solution. They get the feature they want in just the program
that needs it, and the rest of us get a stable, predictable language
without a lot of weird new features in all other ruby programs.

Trans

11/3/2008 2:39:00 AM

0

On Nov 2, 5:40=A0pm, "Caleb Clausen" <vikk...@gmail.com> wrote:

> It's likely that quite a few of the features for ruby that get
> requested could actually be implemented by macros. It's probably
> appropriate that most of these requests are rejected; we don't really
> need a lot of global changes to the language. However, if users can
> write their own macros to scratch some of these itches, that's a
> better solution. They get the feature they want in just the program
> that needs it, and the rest of us get a stable, predictable language
> without a lot of weird new features in all other ruby programs.

Or we could just write our programs in "Macro-Ruby" and to put a death
nail in so called predictable language. ;)

_._ _._
|||| ||||
||||_ ___ _||||
| || .-'___`-. || |
\ / .' .'_ _'. '. \ /
/~~| | (| b d |) | |~~ /' | | | ' | | | `, /__.-: ,| | `-' | |, :-.__\ ,
|'-------( \-''""/.| /\___/\ |.\""''-/ )------'|
| \_.-'\ / '-._____.-' \ /'-._/ |
|.---------\ /'._| _ .---. =3D=3D=3D |_.'\ /--------.|
' \ / | |\_\ _ \=3Dv=3D/ _ | | \ / '
`. | | \_\_\ ~~~ (_) | | .'
`'"'|`'--.__.^.__.--'`|'"'`
\ /
`,..---'"'---..,'
:--..___..--: TO DSL...
\ /
|`. .'| AND BEYOND!
| :___: |
| | | |
| | | |
|.-.| |.-.|
|`-'| |`-'|
| | | |
/ | | |_____| |_____|
':---:-'-:---:'
/ | | jgs /.---.| |.---. `.____; :____.'


T.

ara.t.howard

11/3/2008 7:03:00 AM

0


On Nov 2, 2008, at 3:40 PM, Caleb Clausen wrote:
>>
>
> Sorry for the late reply... I thought this thread had died.
>
> You seem to have left out inline methods, for whatever they're worth.

yeah - if they are really a lot faster it's worth considering but, for
now, i'll assume they're not...

>
>
> Yes, DSLs in ruby are currently very easy to create and use; I
> anticipate that macros will make DSLs slightly harder to write, but
> easier to use. Ruby DSLs tend to be very natural looking to users, but
> there are often small compromises to usability that won't make much
> sense to domain users, such as the need to begin some words with a
> colon and the need to use 'do' at certain places in the language for
> no apparent reason. If you use macros to define your DSL, it should be
> possible to rid oneself of those features. The situation is already
> pretty good, but macros will make it slightly better. I say "will",
> because, to be honest, at the moment most of the sugary convenience
> features needed for nicer DSLs are not present.
>
>

that's an interesting point. not sure about it though - if people are
writing dsls that are not ruby support debugging becomes quite
difficult. still, i get it.


>
>> cfp:~ > cat a.rb
>> def context &block
>> eval 'a += 40', block
>> block.call
>> end
>
> This is a slick way of getting to your caller's lvars. I would have
> passed in a Binding myself, but this way is probably a little cleaner.
>
> But, the caller must pass a block (or binding) in order to make this
> work. Sometimes, that's not a problem. Sometimes it is. For instance
> (I've run into this) if you want to create an api that works exactly
> like Regexp#match, you'll find that it can't be done. #match sets its
> caller's $~ (a local variable); methods can't do that. In the past,
> I've passed in an optional Binding to handle this case, but
> practically speaking, it was a little too clumsy. Using a block
> instead is a better idea, but you're still changing the interface used
> by your custom #match. The whole point is to re-use your user's
> existing knowledge about #match.... if he has to remember, "oh yeah,
> and if you use $~ or other matching variables, you have to pass an
> extra block to #match", then that's not an effective re-use of
> existing knowledge; it might as well be a new interface.
>

i wouldn't strictly agree with your analysis, but i do agree that it's
very hard to do so. the key is having a context or marker which
allows the method to be safely used on all objects, this is what tagz
does for html/xml generation to avoid this - basically methods called
on any 'self' have easy access to the caller. for instance

class Object
def LikeRegexp
LikeRegexpObject.new(self)
end
ed

LikeRegexp.match .....


is one workaround. i do clearly see the value of being inside an
object though. however, is the implication that all macros are global?


>> can you show us something that cannot be done using ruby currently,
>> which macros make possible?
>
> A recently requested new feature on ruby-core was __DIR__, which acts
> like __FILE__, but returns the directory containing the current source
> file. As a macro, that is:
>
> macro __DIR__
> :( File.dirname __FILE__ )
> end
>
> Now maybe (now that I've seen your block-as-binding trick) you can
> actually write this as a method, something like,
>
> def __DIR__(&ctx)
> File.dirname(eval("__FILE__",ctx))
> end
>
> I have no ruby ATM, and can't check if that works or not, sorry. But
> if it does, it will have to be called like __DIR__{}, instead of
> __DIR__. I'd find that a little jarring.
>

def __DIR__
filename = caller[0][/^(.*):/, 1]
File.expand_path(File.dirname(filename))
end


stolen wholesale from Ramaze (lot's of good stuff in there ;-) )


> Another recent request was a 'with' keyword, which operates like
> instance_eval, but only changes the default receiver for code in the
> block passed in, and not self as seen by instance variables. I have an
> implementation of this as well (in the example directory of
> RubyMacros), but for various reasons I'm unsatisfied with it right
> now, so I'd rather not post it.
>


def with &block
scope = Scope.new

instance_variables.each do |ivar|
scope.instance_variable_set ivar, instance_variable_get(ivar)
end

scope.instance_eval &block
end

hacky? yes. but it works well enough for ActionView...



> I'm not claiming that either of these macros is actually a good idea;
> I'm just trying to illustrate the possible.
>
> It's likely that quite a few of the features for ruby that get
> requested could actually be implemented by macros. It's probably
> appropriate that most of these requests are rejected; we don't really
> need a lot of global changes to the language. However, if users can
> write their own macros to scratch some of these itches, that's a
> better solution. They get the feature they want in just the program
> that needs it, and the rest of us get a stable, predictable language
> without a lot of weird new features in all other ruby programs.
>


well now that's something everyone can agree on! seriously, the
project looks super interesting - just trying to think of a real use
case.

hrrrrm. could we possibly use it to skin the

self.ivar = value

problem?


and, to repeat from above, are the global? i'm hoping macros can be
scoped to an object like instance methods....


cheers.


a @ http://codeforp...
--
we can deny everything, except that we have the possibility of being
better. simply reflect on that.
h.h. the 14th dalai lama

Pit Capitain

11/3/2008 9:22:00 AM

0

2008/11/3 ara.t.howard <ara.t.howard@gmail.com>:
> hrrrrm. could we possibly use it to skin the
>
> self.ivar = value
>
> problem?

Of course:

irb(main):001:0> require "nolocal-spike"
=> true

irb(main):002:0> class C
irb(main):003:1> attr_accessor :a, :b, :c
irb(main):004:1> def initialize &blk
irb(main):005:2> instance_eval( &nolocal( &blk ) )
irb(main):006:2> end
irb(main):007:1> end
=> nil

irb(main):008:0> C.new do
irb(main):009:1* a = b = 5
irb(main):010:1> c = a + 2
irb(main):011:1> end
=> #<C:0x2fce8e0 @c=7, @b=5, @a=5>

Two years ago I was experimenting with modifying the AST using plain
Ruby. (On Windows, I couldn't use parsetree for example back then.)
The code above is one of the results. The same should be easy with the
actual libraries.

Regards,
Pit

ThoML

11/3/2008 10:29:00 AM

0


> just trying to think of a real use case.

Macros help lisp to adapt to about every new idea/trend without making
changes to the language as such.

One simple use case that comes to mind is conditional compilation, eg
inserting some code only if a certain flag is set.

> i'm hoping macros can be scoped to an object like instance methods....

I personally think it would be a good idea to somehow restrict macro
expansion to specified classes/namespaces. Otherwise, just of what
would happen if you define a macro foo and use some library that
defines a method of the same name.

Brian Candler

11/3/2008 1:23:00 PM

0

Caleb Clausen wrote:
> A recently requested new feature on ruby-core was __DIR__, which acts
> like __FILE__, but returns the directory containing the current source
> file. As a macro, that is:
>
> macro __DIR__
> :( File.dirname __FILE__ )
> end
>
> Now maybe (now that I've seen your block-as-binding trick) you can
> actually write this as a method, something like,
>
> def __DIR__(&ctx)
> File.dirname(eval("__FILE__",ctx))
> end
>
> I have no ruby ATM, and can't check if that works or not, sorry.

It seems to, as long you provide a dummy block: __DIR__{}.

I think there's a simpler solution though:

def __DIR__
# filename:nnn[:in `method']
File.dirname(caller.first.split(':').first)
end

Actually, it's a pain that 'caller' doesn't provide a proper composite
object with filename, line number, method name. Windows people may have
C:... at the front of their filename. A regexp split should fix that.
--
Posted via http://www.ruby-....