Asp Forum - Seven new VMs, all in a row

Peter Suk

4/7/2005 3:45:00 PM

Hello everyone,

I thought I'd talk about my new project here, since there is a good
chance that someone might be interested in it. I'm planning to put
Ruby on top of Smalltalk VMs. Ruby and Smalltalk are very similar
under the covers, so Smalltalk VMs are a very good match for the
language. This will give Ruby a much faster execution environment
(perhaps 30X), VMs which are capable of incremental garbage collection,
generational garbage collection that is so fast your progra still works
even with an infinite loop allocating new objects (I do this as a lark
sometimes), a wonderful debugger which will let programmers modify
methods on the fly & continue execution, a "workspace" window where you
can execute arbitrary code, a visual "inspect", a powerful "Refactoring
Browser," an industrial strength OODB (Gemstone) with objects and
methods you can define in Ruby, and a readily accessible meta-level
which will allow Rubyists to readily modify their own language. (For
example, you could then use Method wrappers to very quickly implement
an Aspect-Oriented Ruby.)

My strategy for doing this involves writing a Ruby parser (or, rather,
translating the existing one in JRuby to Ruby) then writing a Smalltalk
Parser object to request parsing of Ruby code into an AST from a Ruby
program outside of Smalltalk. We then reify the AST inside the image
and use it to compile Ruby methods into bytecodes. (Multiple syntaxes
can coexist in one Smalltalk image.) Once this is done, we can then
compile the external Ruby parser and bring it into Smalltalk.
Afterwards, we can use the Refactoring Browser Smalltalk parser plus a
little runtime type inferencing to incrementally transform the image
into pure Ruby.

We can do all of this without changing the structure of Ruby files &
Modules or requiring Rubyists to do Smalltalk style image oriented
development. And for those of you sharp enough to wonder: yes, we can
handle Modules, Mixins, and fully qualified Method names without
changing the Smalltalk VMs. (At least those that have Namespaces.)

If anyone is interested, please drop me a line.

--Peter

--
There's neither heaven nor hell, save what we grant ourselves.
There's neither fairness nor justice, save what we grant each other.

82 Answers

R. Mark Volkmann

4/7/2005 3:48:00 PM

I think it would be really cool if you could do this on top of the free Squeak
VM.

Quoting Peter Suk <peter.kwangjun.suk@mac.com>:

> Hello everyone,
>
> I thought I'd talk about my new project here, since there is a good
> chance that someone might be interested in it. I'm planning to put
> Ruby on top of Smalltalk VMs. Ruby and Smalltalk are very similar
> under the covers, so Smalltalk VMs are a very good match for the
> language. This will give Ruby a much faster execution environment
> (perhaps 30X), VMs which are capable of incremental garbage collection,
> generational garbage collection that is so fast your progra still works
> even with an infinite loop allocating new objects (I do this as a lark
> sometimes), a wonderful debugger which will let programmers modify
> methods on the fly & continue execution, a "workspace" window where you
> can execute arbitrary code, a visual "inspect", a powerful "Refactoring
> Browser," an industrial strength OODB (Gemstone) with objects and
> methods you can define in Ruby, and a readily accessible meta-level
> which will allow Rubyists to readily modify their own language. (For
> example, you could then use Method wrappers to very quickly implement
> an Aspect-Oriented Ruby.)
>
> My strategy for doing this involves writing a Ruby parser (or, rather,
> translating the existing one in JRuby to Ruby) then writing a Smalltalk
> Parser object to request parsing of Ruby code into an AST from a Ruby
> program outside of Smalltalk. We then reify the AST inside the image
> and use it to compile Ruby methods into bytecodes. (Multiple syntaxes
> can coexist in one Smalltalk image.) Once this is done, we can then
> compile the external Ruby parser and bring it into Smalltalk.
> Afterwards, we can use the Refactoring Browser Smalltalk parser plus a
> little runtime type inferencing to incrementally transform the image
> into pure Ruby.
>
> We can do all of this without changing the structure of Ruby files &
> Modules or requiring Rubyists to do Smalltalk style image oriented
> development. And for those of you sharp enough to wonder: yes, we can
> handle Modules, Mixins, and fully qualified Method names without
> changing the Smalltalk VMs. (At least those that have Namespaces.)
>
> If anyone is interested, please drop me a line.
>
> --Peter
>
> --
> There's neither heaven nor hell, save what we grant ourselves.
> There's neither fairness nor justice, save what we grant each other.
>
>
>

--
R. Mark Volkmann
Partner, Object Computing, Inc.

Peter Suk

4/7/2005 3:58:00 PM

On Apr 7, 2005, at 10:47 AM, R. Mark Volkmann wrote:

> I think it would be really cool if you could do this on top of the
> free Squeak
> VM.

Yes, I was thinking along these lines. However, no real Namespaces
(yet) in Squeak. We could finagle this by referencing classes with
Module-name prefixes. (This has already been proposed.) When real
Namespaces appear in Squeak, we could then properly place the classes
in modules/namespaces and rename them programatically.

The VMs I am targeting first are commercial ones, but they have free
versions/licenses. (VisualWorks and Gemstone) In addition, they are
very attractive for implementing servers. (VisualWorks is damn fast,
and Gemstone is a mature, rock-solid OODB.)

--
There's neither heaven nor hell, save what we grant ourselves.
There's neither fairness nor justice, save what we grant each other.

Lothar Scholz

4/7/2005 4:32:00 PM

Hello Peter,

PS> Hello everyone,

PS> I thought I'd talk about my new project here, since there is a good
PS> chance that someone might be interested in it. I'm planning to put

Is this just brainstorming or have you already done something ?

--
Best regards, emailto: scholz at scriptolutions dot com
Lothar Scholz http://www.ru...
CTO Scriptolutions Ruby, PHP, Python IDE 's

Peter Suk

4/7/2005 5:57:00 PM

On Apr 7, 2005, at 11:32 AM, Lothar Scholz wrote:

> Hello Peter,
>
> PS> Hello everyone,
>
> PS> I thought I'd talk about my new project here, since there is a good
> PS> chance that someone might be interested in it. I'm planning to put
>
> Is this just brainstorming or have you already done something ?

Lothar,

I have my hands on a working copy of Racc, and the JRuby source. I've
begun the port of the RubyLexer into Smalltalk, but I've decided to
abandon that and use the bootstrap from outside the image technique
instead. I've also started the port of DefaultRubyParser.y from the
JRuby source. I have a simple Ruby script that uses Regexps to
transform the DefaultRubyParser.ry into Ruby. (Regexps are not
powerful enough to tackle the job generically -- that would take a
Context Free Grammar -- but I am including enough specific context in
the regexps that it will work in the specific cases needed.)

I have also worked out and demonstrated the manipulation of method
dictionaries so that we can compile fully qualified Ruby method names
for the VisualWorks VM. I have sketched out a design for handling Ruby
Mixins and assignment-implicit accessors.

I have also recruited a helper for translating the AST node classes.
(There is something to be said for little languages. It's a lot easier
to build syntax driven tools for them!)

Next, I will be taking a very simplified Ruby subset (the calculator
example from the "Dragon Book" which comes with Racc, and the ability
to define methods in Ruby classes) and working out the
Parser-Outside-The-Image scheme with that. Hopefully I'll have that by
this weekend.

--Peter

--
There's neither heaven nor hell, save what we grant ourselves.
There's neither fairness nor justice, save what we grant each other.

jc

4/7/2005 7:15:00 PM

Hi Peter,

- Would this support programming both within or without an image,
choice up to the user?

- From what I understand Smalltalk has a small VM written in C, with
the rest written in Smalltalk itself. Would your version be the Ruby
equivalent of this (VM with everything else in Ruby), or would there
still be some Smalltalk behind the Ruby?

-jc

pat eyler

4/7/2005 7:36:00 PM

On Apr 7, 2005 12:19 PM, jc <james.cromwell@gmail.com> wrote:
> Hi Peter,
>
> - Would this support programming both within or without an image,
> choice up to the user?
>
> - From what I understand Smalltalk has a small VM written in C, with
> the rest written in Smalltalk itself. Would your version be the Ruby
> equivalent of this (VM with everything else in Ruby), or would there
> still be some Smalltalk behind the Ruby?

See the MetaRuby project:

http://blog.zenspider.com/archives/metaruby/... (among other places)

(Although this is about Ruby in Ruby, not Ruby in smalltalk)

>
> -jc
>
>

--
thanks,
-pate
-------------------------
We are often unable to tell people what they need to know, because
they want to know something else, and would therefore only
misunderstand what we said
- the Raven (George MacDonald, Lilith)

Florian Groß

4/7/2005 7:39:00 PM

Peter Suk wrote:

> I thought I'd talk about my new project here, since there is a good
> chance that someone might be interested in it. I'm planning to put
> Ruby on top of Smalltalk VMs.

This does indeed sound promising. Having a self-hosted Ruby would have
certain benefits as well.

> a wonderful debugger which will let programmers modify methods on the
> fly & continue execution

That is already possible with ruby-breakpoint which spawns an IRB
session at a specific place of your code. See
http://ruby-breakpoint.ruby... if you are interested in this sort
of things.

It sure can't hurt to have more options, though.

Peter Suk

4/7/2005 8:19:00 PM

On Apr 7, 2005, at 2:19 PM, jc wrote:

> Hi Peter,
>
> - Would this support programming both within or without an image,
> choice up to the user?

Basically, you could treat the image as just a very powerful
debugger/IDE. The image would spit out new versions of your module
files, which you could then edit/diff to your liking. I have devised a
very powerful algorithm to respect comments, based on diffs of token
sequences.

> - From what I understand Smalltalk has a small VM written in C, with
> the rest written in Smalltalk itself. Would your version be the Ruby
> equivalent of this (VM with everything else in Ruby), or would there
> still be some Smalltalk behind the Ruby?

Initially, there would be some Smalltalk behind the Ruby, but we'd fix
it so you'd never see it. Eventually, we can develop a completely Ruby
meta-layer.

--
There's neither heaven nor hell, save what we grant ourselves.
There's neither fairness nor justice, save what we grant each other.

Peter Suk

4/7/2005 8:21:00 PM

On Apr 7, 2005, at 2:36 PM, pat eyler wrote:

> On Apr 7, 2005 12:19 PM, jc <james.cromwell@gmail.com> wrote:
>> Hi Peter,
>>
>> - Would this support programming both within or without an image,
>> choice up to the user?
>>
>> - From what I understand Smalltalk has a small VM written in C, with
>> the rest written in Smalltalk itself. Would your version be the Ruby
>> equivalent of this (VM with everything else in Ruby), or would there
>> still be some Smalltalk behind the Ruby?
>
> See the MetaRuby project:
>
> http://blog.zenspider.com/archives/metaruby/... (among other
> places)
>
> (Although this is about Ruby in Ruby, not Ruby in smalltalk)

I am aware of MetaRuby. This project is also about Ruby in Ruby.
(Smalltalk is in Smalltalk. But the VM doesn't care which
language/meta-language it is running!)

However, this meta-Ruby will run two orders of magnitude faster!

--
There's neither heaven nor hell, save what we grant ourselves.
There's neither fairness nor justice, save what we grant each other.

Peter Suk

4/7/2005 8:34:00 PM

On Apr 7, 2005, at 2:39 PM, Florian Groß wrote:

> Peter Suk wrote:
>
>> I thought I'd talk about my new project here, since there is a good
>> chance that someone might be interested in it. I'm planning to put
>> Ruby on top of Smalltalk VMs.
>
> This does indeed sound promising. Having a self-hosted Ruby would have
> certain benefits as well.
>
>> a wonderful debugger which will let programmers modify methods on the
>> fly & continue execution
>
> That is already possible with ruby-breakpoint which spawns an IRB
> session at a specific place of your code. See
> http://ruby-breakpoint.ruby... if you are interested in this
> sort
> of things.
>
> It sure can't hurt to have more options, though.

In the Smalltalk debuggers, you can spawn off as many debugger sessions
as you want by highlighting arbitrary code in the debugger as well as
any browser window, as well as visually "inspect" any object (with each
one appearing in a window), with the context of every window sensitive
to the class being displayed. The effect is basically an
"irb-anywhere" which you can arbitrarily interleave with any number of
visual inspects. That, and a self-hosted language, faster VM, more
garbage collection options, etc...

--
There's neither heaven nor hell, save what we grant ourselves.
There's neither fairness nor justice, save what we grant each other.

comp.lang.ruby

Seven new VMs, all in a row

Peter Suk

R. Mark Volkmann

Peter Suk

Lothar Scholz

Peter Suk

jc

pat eyler

Florian Groß

Peter Suk

Peter Suk

Peter Suk

x Login to ForumsZone