[lnkForumImage]
TotalShareware - Download Free Software

Confronta i prezzi di migliaia di prodotti.
Asp Forum
 Home | Login | Register | Search 


 

Forums >

comp.lang.ruby

Extension Language for a Text Editor

Nikolai Weibull

10/8/2003 8:07:00 PM

OK. So I'm going to write a text editor for my masters' thesis. The
general idea of it is fixed but the extension language has not been
settled on, yet. I have thought of, in the footsteps of editors such as
Emacs and its clones, using some version of LISP or Scheme, possibly
GNU's Guile[1], or Jade's librep[2]. LISP-like languages have proved
very successful in describing the way we edit text and is thus a sure
bet. It would, however, perhaps be of interest to use another language
to try the waters. In my previous editor (slackedit/sled[3]) I used
Tcl[4] as an extension language, and it proved very easy to use, but I
never got far enough to actually give it a good run.

Anyway, what I'm getting at is:

Do you figure Ruby to be a good extension language for a text editor?
* What would be easier to do using Ruby?
* LISP?
* What language allows the most of the editing commands to be written in
the given language?
* Is Ruby good enough for the task at the moment, performance wise?
* Resource wise? Does anyone have any statistics on this?

I'm not talking about the ease of implementing a web browser in the
target language, rather the ease of structuring an extensible framework
in it.

Also, is there any way of redefining the // operator for constructing
regular expression objects? I'm planning on implementing a new regex
syntax for the editor (to make searches/substitutions easier to
describe).

thanks in advance,
nikolai

[1] http://www.gnu.org/softw...
[2] http://librep.source...
[3] http://www.pcppopper.org/code...
[4] http://www.scri...

--
::: name: Nikolai Weibull :: aliases: pcp / lone-star / aka :::
::: born: Chicago, IL USA :: loc atm: Gothenburg, Sweden :::
::: page: www.pcppopper.org :: fun atm: gf,lps,ruby,lisp,war3 :::
main(){printf(&linux["\021%six\012\0"],(linux)["have"]+"fun"-97);}

44 Answers

Ryan Pavlik

10/8/2003 8:28:00 PM

0

On Thu, 9 Oct 2003 05:06:32 +0900
Nikolai Weibull <ruby-talk@pcppopper.org> wrote:

> OK. So I'm going to write a text editor for my masters' thesis. The
> general idea of it is fixed but the extension language has not been
> settled on, yet.
<snip>

The thing with lisp, though, is that everyone has used lisp already,
and while I love lisp, there wouldn't be much of a reason to switch
from XEmacs and its twenty million extensions.

> Anyway, what I'm getting at is:
>
> Do you figure Ruby to be a good extension language for a text
> editor?

YES. I've been wanting "erubs" (Editor for Ruby Scripts ;-) for
awhile; something like emacs, but s/lisp/ruby. Same design otherwise.

> * What would be easier to do using Ruby?

Well, the main thing is that ruby has a lot of very convenient pattern
and text matching functionality, and a boatload of extensions. And
it's a very easy language, so anyone can pick it up and start
extending the editor, unlike emacs, where fewer brave the waters.

For instance, I recently wrote a bit of code to align things:

a = b ___\ a = b
foo = bar / foo = bar

This was a few simple lines of ruby. Not counting comments, it's 44
lines of lisp that don't quite work perfectly.

> * LISP?

As much as I love lisp, I can't really think of things that would be
easier to in any of the above scheme flavors that wouldn't be easier
in ruby.

> * What language allows the most of the editing commands to be written in
> the given language?

Ruby certainly allows this. It's extremely trivial to extend and use
Ruby in C. They go hand-in-hand. Plus there are existing GUI
packages you could interface to if you wanted to provide UI handling
in ruby without a lot of work.

> * Is Ruby good enough for the task at the moment, performance wise?
> * Resource wise? Does anyone have any statistics on this?

This is a decent benchmark:

http://www.bagley.org/~doug/shootout/c...

Is ruby at the top of that list? No. (Note that most of the
languages above it are compiled, including the bigloo scheme
implementation). But XEmacs elisp falls near the bottom, and I use
that for editing tasks daily. Tcl and Rep both fall below.

So, I wouldn't worry about performance. And, memory-wise, I can't
really offer benchmarks, but I have a system managing tens of
thousands of objects that doesn't present a problem.

> I'm not talking about the ease of implementing a web browser in the
> target language, rather the ease of structuring an extensible framework
> in it.

Then Ruby is ideal. Extending and embedding it is extremely easy.
It's a really great scripting framework, even though many people
including myself use it for entire applications.

> Also, is there any way of redefining the // operator for constructing
> regular expression objects? I'm planning on implementing a new regex
> syntax for the editor (to make searches/substitutions easier to
> describe).

No, but I would encourage you leave the existing regular expression
format intact, as, while somewhat write-only, is familiar to most
people who will be using your editor. Redefining // would break a lot
of extension code, and alienate a large part of your audience.

However, that's not to say you can't make your own matching format.
I'm not sure what you have in mind, but there are a number of ways to
bend the syntax to integrate such things.

> thanks in advance,

If you write a ruby editor... thank _you_. ;-)

hth,

--
Ryan Pavlik <rpav@mephle.com>

"You'd be surprised what a platoon of heartless ninja lawyers
can do in favor of a position." - 8BT

Simon Strandgaard

10/8/2003 9:03:00 PM

0

On Thu, 09 Oct 2003 06:06:32 +0900, Nikolai Weibull wrote:

> OK. So I'm going to write a text editor for my masters' thesis.

What a coincidence.. I am also writing a programmers editor (AEditor)
in Ruby.
http://aeditor.ruby...


> In my previous editor (slackedit/sled[3]) I used
> Tcl[4] as an extension language, and it proved very easy to use, but I
> never got far enough to actually give it a good run.

where is the homepage ? ;-)
http://sourceforge.net/projects...
Your Trove-info says 'gnome', but no unix install.


> Do you figure Ruby to be a good extension language for a text editor?

My original plan were to use Ruby as an extension language and write
the core in C++... At some point started experimenting using Ruby for
everything, and now I have actually written AEditor completely in Ruby!!


> * What would be easier to do using Ruby?

undo/redo/macros.. The code which is responsible for this only takes up
119 lines of Ruby code. This is more complex in C++ because you cannot
#clone arbitrary objects.

unittesting: AEditor has about 440 testcases and 1400 assertions.
unittesting is annoying in C++. In Ruby its easy.

It would be easier for users of your editor to make changes
to the internals of your editor (because Ruby is intuitive).
If you have a testharness then they will also could test if
their changes has unforeseen sideeffects.


> * LISP?

not enough intuitive.


> * What language allows the most of the editing commands to be written in
> the given language?

If your editor core is written in a different language than your extension
language. Then you will no matter what language chosen, have to make a
bridge between. There are shortcuts, such as SWIG.
SWIG can generate wrappers for *many* script languages.
My vote would be Ruby :-)


> * Is Ruby good enough for the task at the moment, performance wise?

You can try out AEditor, to feel if Ruby is fast enough.
On a 100MHz machine AEditor feels quite Okay.
On a 33MHz 486 it can be slow.
I can live with this hardware requirements, thats why I choose to write
everything in Ruby :-)


> * Resource wise? Does anyone have any statistics on this?

My datastructure has a huge amount of memory overhead (because it uses an
Array of characters). I am in the process of rethinking the datastructure
and maybe write a datastructure Ruby-extension module in C++..?


> Also, is there any way of redefining the // operator for constructing
> regular expression objects? I'm planning on implementing a new regex
> syntax for the editor (to make searches/substitutions easier to
> describe).

Overloading Regexp... I don't know.
I am currently writing my own regex engine.



What features do you plan for your editor ?
Is it a programmers editor, or a propertional font editor ?

--
Simon Strandgaard

Nikolai Weibull

10/8/2003 9:09:00 PM

0

* Ryan Pavlik <rpav@mephle.com> [Oct, 08 2003 22:30]:
> The thing with lisp, though, is that everyone has used lisp already,
> and while I love lisp, there wouldn't be much of a reason to switch
> from XEmacs and its twenty million extensions.
Yes, this is one of the downsides in a way. You won't be able to get
people to switch very easily when everything is the same ;-)
> YES. I've been wanting "erubs" (Editor for Ruby Scripts ;-) for
> awhile; something like emacs, but s/lisp/ruby. Same design otherwise.
hehe, that won't be the name, but yeah, that's in a sense what I want.
Except s/Emacs/Vim/ ;-).
> Well, the main thing is that ruby has a lot of very convenient pattern
> and text matching functionality, and a boatload of extensions. And
> it's a very easy language, so anyone can pick it up and start
> extending the editor, unlike emacs, where fewer brave the waters.
Well, no dependencies on Ruby extensions would be necesarry, but I
guess, since they exist, they could be used.
> For instance, I recently wrote a bit of code to align things:
>
> a = b ___\ a = b
> foo = bar / foo = bar
>
> This was a few simple lines of ruby. Not counting comments, it's 44
> lines of lisp that don't quite work perfectly.
Could you provide the code? I'd love to see the comparison.
> As much as I love lisp, I can't really think of things that would be
> easier to in any of the above scheme flavors that wouldn't be easier
> in ruby.
OK. The way I see it, Ruby is OO-programming personified. LISP is,
well, functional? programming personified. Anyway, I have gotten the
feeling that it's generally easier to think in terms of functional, not
OO, when editing text. I mean, what do tho OO constructs really add?
>
> > * What language allows the most of the editing commands to be written in
> > the given language?
>
> Ruby certainly allows this. It's extremely trivial to extend and use
> Ruby in C. They go hand-in-hand. Plus there are existing GUI
> packages you could interface to if you wanted to provide UI handling
> in ruby without a lot of work.
Ah, I think you misunderstood the question. Or I'm misunderstanding the
answer ;-). I want the C core to be as small as possible, leaving the
most possible flexibility using the extension language. And I don't
generally see the need for GUI extensions and such.
> > * Is Ruby good enough for the task at the moment, performance wise?
> > * Resource wise? Does anyone have any statistics on this?
>
> This is a decent benchmark:
>
> http://www.bagley.org/~doug/shootout/c...
>
> Is ruby at the top of that list? No. (Note that most of the
> languages above it are compiled, including the bigloo scheme
> implementation). But XEmacs elisp falls near the bottom, and I use
> that for editing tasks daily. Tcl and Rep both fall below.
Thanks, great summary!
> So, I wouldn't worry about performance. And, memory-wise, I can't
> really offer benchmarks, but I have a system managing tens of
> thousands of objects that doesn't present a problem.
OK. Good.
> > I'm not talking about the ease of implementing a web browser in the
> > target language, rather the ease of structuring an extensible framework
> > in it.
> Then Ruby is ideal. Extending and embedding it is extremely easy.
Yeah, this, I know. My experiences from the Ruby-GNOME2 project have
been great. Very easy to add extensions. I haven't used the embedding
facilities yet, but I'm sure they're good.
> > Also, is there any way of redefining the // operator for constructing
> > regular expression objects? I'm planning on implementing a new regex
> > syntax for the editor (to make searches/substitutions easier to
> > describe).
>
> No, but I would encourage you leave the existing regular expression
> format intact, as, while somewhat write-only, is familiar to most
> people who will be using your editor. Redefining // would break a lot
> of extension code, and alienate a large part of your audience.
>
> However, that's not to say you can't make your own matching format.
> I'm not sure what you have in mind, but there are a number of ways to
> bend the syntax to integrate such things.
Yeah, but that's just the thing. This is one of the real selling
points, if you will ;-). Oh well, I guess one can always do some
MyRegex.new(string)
but then you wind up with the string interpolation problems (\n and
friends). :-(
>
> > thanks in advance,
>
> If you write a ruby editor... thank _you_. ;-)
hehe, we'll see ;-)
>
> hth,
>
thanks for your long and quick response. It's good to know that at
least someone is interested :-),
nikolai

--
::: name: Nikolai Weibull :: aliases: pcp / lone-star / aka :::
::: born: Chicago, IL USA :: loc atm: Gothenburg, Sweden :::
::: page: www.pcppopper.org :: fun atm: gf,lps,ruby,lisp,war3 :::
main(){printf(&linux["\021%six\012\0"],(linux)["have"]+"fun"-97);}

gabriele renzi

10/8/2003 9:39:00 PM

0

il Thu, 9 Oct 2003 05:06:32 +0900, Nikolai Weibull
<ruby-talk@pcppopper.org> ha scritto::


my 2c:

>Anyway, what I'm getting at is:
>
>Do you figure Ruby to be a good extension language for a text editor?
>* What would be easier to do using Ruby?

the learning curve for casual user would be really less hard, and it
won't collide with already existing lisp based editors ;)


>* What language allows the most of the editing commands to be written in
> the given language?
Well, given that tab expansion, string replacing/finding,
byte/word/line count, justification and other stuff can be easily done
with one line of ruby...
Anyway, I think the apis you make available from the core of the
editor would be something really important in this analisys.

>* Is Ruby good enough for the task at the moment, performance wise?
Sure :)

>* Resource wise? Does anyone have any statistics on this?
dunno

>Also, is there any way of redefining the // operator for constructing
>regular expression objects? I'm planning on implementing a new regex
>syntax for the editor (to make searches/substitutions easier to
>describe).

well, you have the soruce code, dig in it and change what you need, if
you really need it :)
Anyway, why would you do such a thing ? the regex language is quite
standardized now, why would you change it?

Dale Martenson

10/8/2003 9:59:00 PM

0

You may want to look at the VIM's use of Ruby for writing extensions.
Documentation available inside of VIM via ":help ruby".

For an up-to-date copy of VIM see: http://w...

--Dale

Ryan Pavlik

10/8/2003 10:04:00 PM

0

On Thu, 9 Oct 2003 06:09:29 +0900
Nikolai Weibull <ruby-talk@pcppopper.org> wrote:

> * Ryan Pavlik <rpav@mephle.com> [Oct, 08 2003 22:30]:

> > YES. I've been wanting "erubs" (Editor for Ruby Scripts ;-) for
> > awhile; something like emacs, but s/lisp/ruby. Same design
> > otherwise.

> hehe, that won't be the name, but yeah, that's in a sense what I want.
> Except s/Emacs/Vim/ ;-).

Well, whichever, just be aware that emacs is designed as an extensible
editor, and vim is not, even though such things have crept in with
varying degrees of usefulness.

> > Well, the main thing is that ruby has a lot of very convenient pattern
> > and text matching functionality, and a boatload of extensions. And
> > it's a very easy language, so anyone can pick it up and start
> > extending the editor, unlike emacs, where fewer brave the waters.

> Well, no dependencies on Ruby extensions would be necesarry, but I
> guess, since they exist, they could be used.

Right. They're there, people can write extensions that interface to
the web, or whatever.

> > This was a few simple lines of ruby. Not counting comments, it's 44
> > lines of lisp that don't quite work perfectly.

> Could you provide the code? I'd love to see the comparison.

See http://ogmo.mephle.org/tabular-ali... for the Lisp
version. The ruby one I deleted, as it was pretty simple to
reproduce, I'm sure someoone can whip up an example.

> > As much as I love lisp, I can't really think of things that would be
> > easier to in any of the above scheme flavors that wouldn't be easier
> > in ruby.

> OK. The way I see it, Ruby is OO-programming personified. LISP is,
> well, functional? programming personified. Anyway, I have gotten the
> feeling that it's generally easier to think in terms of functional, not
> OO, when editing text. I mean, what do tho OO constructs really
> add?

I don't really find that. I don't think functional programming is any
easier for editor-related tasks. I'm not even sure how you would come
up with such an assumption. ;)

> > Ruby certainly allows this. It's extremely trivial to extend and use
> > Ruby in C. They go hand-in-hand. Plus there are existing GUI
> > packages you could interface to if you wanted to provide UI handling
> > in ruby without a lot of work.

> Ah, I think you misunderstood the question. Or I'm misunderstanding the
> answer ;-). I want the C core to be as small as possible, leaving the
> most possible flexibility using the extension language. And I don't
> generally see the need for GUI extensions and such.

Right, tiny C core like emacs, everything higher-level in the language
of choice. Ruby is highly suited for this task.

People could even write high-performance ruby extensions in C...

<snip>
> > However, that's not to say you can't make your own matching format.
> > I'm not sure what you have in mind, but there are a number of ways to
> > bend the syntax to integrate such things.

> Yeah, but that's just the thing. This is one of the real selling
> points, if you will ;-). Oh well, I guess one can always do some

Well, to be blunt, whatever you come up with won't be as popular or
useful as the existing regular expressions, just because they'll be a
nonstandard replacement of something already very common. PCRE
regexps are extremely flexible and well-known.

That isn't to say people won't use them, especially if they're
simpler, but it probably won't be the main selling point of your
editor to _other people_.

> MyRegex.new(string)

> but then you wind up with the string interpolation problems (\n and
> friends). :-(

I'm not sure how that's a problem. The same applies to // regexps.
They're just basically strings, except stored in a different type of
object with a few flags.

hth,

--
Ryan Pavlik <rpav@mephle.com>

"You'd be surprised what a platoon of heartless ninja lawyers
can do in favor of a position." - 8BT

Simon Strandgaard

10/8/2003 10:17:00 PM

0

On Wed, 08 Oct 2003 22:39:13 +0000, gabriele renzi wrote:

> Anyway, why would you do such a thing ? the regex language is quite
> standardized now, why would you change it?

I am in the process of writing my own regex engine (for AEditor).
The reason why I cannot use Ruby's are, that I must run regex's on
a wide range of iterators.

I want to use same syntax as Ruby's regex, and perhaps extend it
with some editor stuff, perhaps like:
* how to traverse the datastructure, jump over folds, jump into folds.
* regex on columnar selections.
* mechanism for getting the syntax-state: perhaps a regex which
only operates inside comments, apply regex to variable-names, etc.

Status for my regex project:
DONE studies of how Rubys GNU-regex engine behave.
DONE nfa 2 dfa.
DONE regex-datastructure 2 nfa.
TODO translate regex-string into regex-datastructure

There is probably some item which I have overseen.

You can check it out here:
http://rubyforge.org/cgi-bin/cvsweb.cgi/projects/experimental/nfa_to_dfa/?cvsro...

--
Simon Strandgaard

Seth Kurtzberg

10/8/2003 10:19:00 PM

0

Well, for one thing, doing it in lisp would be reinventing the wheel, as
emacs has been around for ages.

One main advantage that I see to ruby is that one could modify existing
code and add new code much more easily if the modifier doesn't have a
lisp background. lisp code is horribly unreadable.

If you do want to use a functional language Haskell is a much better
choice, and would in fact be interesting in this context.

Nikolai Weibull wrote:

>OK. So I'm going to write a text editor for my masters' thesis. The
>general idea of it is fixed but the extension language has not been
>settled on, yet. I have thought of, in the footsteps of editors such as
>Emacs and its clones, using some version of LISP or Scheme, possibly
>GNU's Guile[1], or Jade's librep[2]. LISP-like languages have proved
>very successful in describing the way we edit text and is thus a sure
>bet. It would, however, perhaps be of interest to use another language
>to try the waters. In my previous editor (slackedit/sled[3]) I used
>Tcl[4] as an extension language, and it proved very easy to use, but I
>never got far enough to actually give it a good run.
>
>Anyway, what I'm getting at is:
>
>Do you figure Ruby to be a good extension language for a text editor?
>* What would be easier to do using Ruby?
>* LISP?
>* What language allows the most of the editing commands to be written in
> the given language?
>* Is Ruby good enough for the task at the moment, performance wise?
>* Resource wise? Does anyone have any statistics on this?
>
>I'm not talking about the ease of implementing a web browser in the
>target language, rather the ease of structuring an extensible framework
>in it.
>
>Also, is there any way of redefining the // operator for constructing
>regular expression objects? I'm planning on implementing a new regex
>syntax for the editor (to make searches/substitutions easier to
>describe).
>
>thanks in advance,
> nikolai
>
>[1] http://www.gnu.org/softw...
>[2] http://librep.source...
>[3] http://www.pcppopper.org/code...
>[4] http://www.scri...
>
>--
>::: name: Nikolai Weibull :: aliases: pcp / lone-star / aka :::
>::: born: Chicago, IL USA :: loc atm: Gothenburg, Sweden :::
>::: page: www.pcppopper.org :: fun atm: gf,lps,ruby,lisp,war3 :::
>main(){printf(&linux["\021%six\012\0"],(linux)["have"]+"fun"-97);}
>
>
>.
>
>
>


--
Seth Kurtzberg
CTO
Scottsdale Research and Network Operations Center
ISEC.us
seth@isec.us



Ryan Pavlik

10/8/2003 10:23:00 PM

0

On Thu, 9 Oct 2003 07:10:42 +0900
Simon Strandgaard <qj5nd7l02@sneakemail.com> wrote:

<snip>
> I want to use same syntax as Ruby's regex, and perhaps extend it
> with some editor stuff, perhaps like:

There seem to be simpler/more elegant solutions to these problems than
reimplementing pcre:

> * how to traverse the datastructure, jump over folds, jump into
> folds.
> * regex on columnar selections.
> * mechanism for getting the syntax-state: perhaps a regex which
> only operates inside comments, apply regex to variable-names, etc.

All three of these can easily be accomplished by providing constrained
input to the matcher. For folds etc, first you grab the pieces you
want (folds, no folds), then apply the re. For colums, same deal.
Extract the column, then apply. I'm sure you can guess where this is
going for the last one... extract and apply the regexp only to
comments or variable names.

You could have a simple "Region" class which selects the appropriate
areas, make an array of applicable regions, and then apply the regexp
to each. This gives you three exceedingly simple tasks:

1) Region objects defined by <start> and <end>.
2) Regexp application to a region.
3) Matching code to produce Regions. (Obviously, you'll need
this in any case, to select only comments, etc.)

I'd say this is way simpler, more effective, and more extensible than
redefining all-new regexp syntax. Especially since the existing
syntax is hairy enough. ;-)

--
Ryan Pavlik <rpav@mephle.com>

"You'd be surprised what a platoon of heartless ninja lawyers
can do in favor of a position." - 8BT

Gavin Sinclair

10/8/2003 11:00:00 PM

0

On Thursday, October 9, 2003, 7:59:07 AM, Dale wrote:

> You may want to look at the VIM's use of Ruby for writing extensions.
> Documentation available inside of VIM via ":help ruby".

> For an up-to-date copy of VIM see: http://w...

There's a little discussion of this at

http://www.rubygarden.org/ruby?VimRub...

This capability of Vim doesn't seem to be heavily used. I certainly
don't use it. Partially because the integration is a bit lacking (but
you could do Ryan's aligner very easily - but then there's an
excellent and well-supported vim plugin for that anyway). Partially
because I couldn't be bothered.

Gavin