[lnkForumImage]
TotalShareware - Download Free Software

Confronta i prezzi di migliaia di prodotti.
Asp Forum
 Home | Login | Register | Search 


 

Forums >

comp.lang.ruby

When little languages grow...

Hugh Sasse

1/26/2005 5:08:00 PM

52 Answers

Zach Dennis

1/26/2005 5:24:00 PM

0

To make sure myself and the rest of the list is correctly hearing you
question. You are interested in the ruby way to write a code generator?

And you are looking for input other parsing or implemented solutions
others may have experience with, with languges and tools such as; lisp,
forth, yacc, bison, racc, etc.. ?

Zach




Hugh Sasse Staff Elec Eng wrote:
> I seem to have run into my parsing problem again. Whatever I'm
> doing I usually end up having to parse non-simplistic input, and I'm
> still not happy about the apparently available solutions to this.
> So I'm wondering what other people do.
>
> The application is immaterial at the moment, but the problem is that
> I need to do more than can be done with a simple case statement, and
> if I were to use case statements managing the problem would get too
> big.
>
> The conventional wisdom is to use some form of parser
> generator (Yacc, Bison, Racc, Rockit,...) but I don't have
> confidence in my ability to get these working well.[1].
> I have had great difficulty in the past, certainly.
>
> Other possibilities I have considered and tried are to lash together
> some form of Lisp [cf Greenspun's 10th rule of programming] or Forth,
> but I don't consider myself fluent in either of those languages, and
> they are not as easy a user interface for other people as Ruby would
> be. I can get something working, but find it hard to maintain or
> improve. [2]
>
> So the next possibility is to use something like
>
> input = nil
> File.open("input.txt"){|f|
> input = f.read
> }
> Thread.new(input){|source|
> $SAFE=5
> instance_eval source
> }.value
>
> or something, and actually make the commands in the language methods
> of some Ruby object.
>
> It is often observed that it is difficult to add security to a
> system, compared to building it in from the start. Can I do this
> and still have a good level of security? Should I make the parser
> object (whose method's I'm using) a subclass of Nil, to limit it as
> much as possible? I need to give people enough rope to hold their
> input together, but not enough to hang themselves (or me). I don't
> want people to be able to execute arbitrary code, or fiddle with
> objects they should not need to touch.
>
> Is there another way to handle input flexibly that I have completely
> missed? I've googled for things to do with little languages and
> parsing, but have found nothing enlightening.
>
> Thank you,
> Hugh
>
> [1] I find that thinking in the manner of a shift/reduce parser is
> particularly unnatural to me. This might just be a weakness on my
> part or may have something to do with people's difficulties in
> handling modal interfaces: it is hard to switch contexts rapidly.
> Maybe there is something I can read which will turn the problem
> around, so it becomes easy to handle?
>
> [2] Immensely powerful and fast systems have been written in Forth,
> and Lisp is very powerful in the right hands. I just don't have the
> experience with these to be effective, yet.
>
>
>



ts

1/26/2005 5:29:00 PM

0

>>>>> "H" == Hugh Sasse Staff Elec Eng <hgs@dmu.ac.uk> writes:

H> Thread.new(input){|source|
H> $SAFE=5
H> instance_eval source
H> }.value

Sorry to say this but this is the most common error that I see when
someone try to eval some code with $SAFE >= 4

The code will be eval'ed with $SAFE >= 4 but the result (#value) will be
used with $SAFE = 0 and you can have problems.

The result of #eval must be cleaned with $SAFE >= 4, before it's
returned.


Guy Decoux


Hugh Sasse

1/26/2005 5:43:00 PM

0

Hugh Sasse

1/26/2005 5:48:00 PM

0

Robert Klemme

1/26/2005 5:49:00 PM

0


Hugh,

the one thing I didn't see in your posting is a statement about the
language. What capabilities should it have? If it's just assigning
constants to vars (like often needed for configurations) then Regexp is
probably fine. From what you write I'm guessing that your envisioned
language is more complex - but how complex? Maybe it's a special case for
which someone somewhere has a solution already.

Regards

robert


"Hugh Sasse Staff Elec Eng" <hgs@dmu.ac.uk> schrieb im Newsbeitrag
news:Pine.GSO.4.60.0501261622400.24999@brains.eng.cse.dmu.ac.uk...
>I seem to have run into my parsing problem again. Whatever I'm
> doing I usually end up having to parse non-simplistic input, and I'm
> still not happy about the apparently available solutions to this.
> So I'm wondering what other people do.
>
> The application is immaterial at the moment, but the problem is that
> I need to do more than can be done with a simple case statement, and
> if I were to use case statements managing the problem would get too
> big.
>
> The conventional wisdom is to use some form of parser
> generator (Yacc, Bison, Racc, Rockit,...) but I don't have
> confidence in my ability to get these working well.[1].
> I have had great difficulty in the past, certainly.
>
> Other possibilities I have considered and tried are to lash together
> some form of Lisp [cf Greenspun's 10th rule of programming] or Forth,
> but I don't consider myself fluent in either of those languages, and
> they are not as easy a user interface for other people as Ruby would
> be. I can get something working, but find it hard to maintain or
> improve. [2]
>
> So the next possibility is to use something like
>
> input = nil
> File.open("input.txt"){|f|
> input = f.read
> }
> Thread.new(input){|source|
> $SAFE=5
> instance_eval source
> }.value
>
> or something, and actually make the commands in the language methods
> of some Ruby object.
>
> It is often observed that it is difficult to add security to a
> system, compared to building it in from the start. Can I do this
> and still have a good level of security? Should I make the parser
> object (whose method's I'm using) a subclass of Nil, to limit it as
> much as possible? I need to give people enough rope to hold their
> input together, but not enough to hang themselves (or me). I don't
> want people to be able to execute arbitrary code, or fiddle with
> objects they should not need to touch.
>
> Is there another way to handle input flexibly that I have completely
> missed? I've googled for things to do with little languages and parsing,
> but have found nothing enlightening.
>
> Thank you,
> Hugh
>
> [1] I find that thinking in the manner of a shift/reduce parser is
> particularly unnatural to me. This might just be a weakness on my
> part or may have something to do with people's difficulties in
> handling modal interfaces: it is hard to switch contexts rapidly.
> Maybe there is something I can read which will turn the problem
> around, so it becomes easy to handle?
>
> [2] Immensely powerful and fast systems have been written in Forth,
> and Lisp is very powerful in the right hands. I just don't have the
> experience with these to be effective, yet.
>
>

why the lucky stiff

1/26/2005 5:56:00 PM

0

Hugh Sasse Staff Elec Eng (hgs@dmu.ac.uk) wrote:
> The application is immaterial at the moment, but the problem is that
> I need to do more than can be done with a simple case statement, and
> if I were to use case statements managing the problem would get too
> big.
>
> ...
>
> So the next possibility is to use something like
>
> input = nil
> File.open("input.txt"){|f|
> input = f.read
> }
> Thread.new(input){|source|
> $SAFE=5
> instance_eval source
> }.value
>
> or something, and actually make the commands in the language methods
> of some Ruby object.

I hear ya. I wish there was a way to open a jailed namespace. It'd be
like chroot'ing into a module. The sandbox module would be addressable
from ::Object, but would only include a limited set of modules when
offered under chroot.

_why


Mark Probert

1/26/2005 6:07:00 PM

0

Hi ..

On Wednesday 26 January 2005 09:08, Hugh Sasse Staff Elec Eng wrote:
> I seem to have run into my parsing problem again. Whatever I'm
> doing I usually end up having to parse non-simplistic input, and I'm
> still not happy about the apparently available solutions to this.
> So I'm wondering what other people do.
>
My personal solution to this is to use Coco/R, an LL(1) scanner/generator.
You can find more information at:

http://www.scifac.ru....

The primary advantage of this approach, IMHO, is that all of the grammar /
scanning rules are in a single file (rather than the lex/yacc approach).
This makes the grammar quite easy to read and extend, once you are familiar
with the process. Ryan Davies has a pure ruby version, and I have a ruby
extension version. Both seem to work well for little languages.

> [1] I find that thinking in the manner of a shift/reduce parser is
> particularly unnatural to me. ... Maybe there is something I can read which
> will turn the problem around, so it becomes easy to handle?

Pat Terry has a book "Compilers and Compiler Generators" that covers LL(1)
(and other) topics very well. You can find it at:

http://www.scifac.ru.ac.za/...

The primary disadvantage of Coco/R is the LL(1) part. This means that your
grammar needs to be fairly well formed and not arbitrarily complex. As an
example, Ruby can not, as far as I have tried, be converted into an LL(1)
grammar, though C can.

A simple example of the ruby grammar (this is for the famous four function
calculator) for my extension library. Note that this will generate a Ruby
extension. When you compile and link, you can use it in Ruby like this:

# ---( test.rb )-------------
require 'Calc'

f = File.readlines("calc.inp")
t = Calc.new
t.run(f)

if t.success
puts "parsed ok!"
t.capture.each { |ans| puts " ans==#{ans}" }
else
puts "Errors ::"
t.errs.each { |err| puts " --> #{err}" }
end


# ---( calc.inp )-----------
var a,b,c,d;

write 1+(2*3)+4;
write 100/10;

a := 37-12-(4*5);
write a;
b := a*16;
write b*2



# ---( calc.atg )-----------
$C /* Generate Main Module */
COMPILER Calc

#define upcase(c) ((c >= 'a' && c <= 'z')? c-32:c)
int VARS[10000];

int get_spix()
{
char name[20];
LEX_S(name, sizeof(name) - 1);
if (strlen(name) >= 2)
return 26*(upcase(name[1])-'A')+(upcase(name[0])-'A');
else
return (upcase(name[0])-'A');
}

int get_number()
{
char name[20];
LEX_S(name, sizeof(name) - 1);
return atoi(name);
}

void new_var(int spix)
{
VARS[spix] = 0;
}

int get_var(int spix)
{
return VARS[spix];
}

void write_val(int val)
{
char tmp[20];

sprintf(tmp, "%d", val);
t_capture_output(tmp);
}

void set_var(int spix, int val)
{
VARS[spix] = val;
}

IGNORE CASE

CHARACTERS
letter = "ABCDEFGHIJKLMNOPQRSTUVWXYZabcdefghijklmnopqrstuvwxyz".
digit = "0123456789".
eol = CHR(13) .
lf = CHR(10) .

COMMENTS
FROM '--' TO eol

IGNORE eol + lf

TOKENS
ident = letter {letter | digit} .
number = digit {digit} .

PRODUCTIONS
Calc =
[Declarations] StatSeq .

Declarations
= (. int spix; .)
'VAR'
Ident <&spix> (. new_var(spix); .)
{ ',' Ident <&spix> (. new_var(spix); .)
} ';'.

StatSeq =
Stat {';' Stat}.

Stat
= (. int spix, val; .)
| "WRITE" Expr <&val> (. write_val(val); .)
| Ident <&spix> ":=" Expr <&val> (. set_var(spix, val); .) .

Expr <int *exprVal>
= (. int termVal; .)
Term <exprVal>
{ '+' Term <&termVal> (. *exprVal += termVal; .)
| '-' Term <&termVal> (. *exprVal -= termVal; .)
} .

Term <int *termVal>
= (. int factVal; .)
Fact <termVal>
{ '*' Fact <&factVal> (. *termVal *= factVal; .)
| '/' Fact <&factVal> (. *termVal /= factVal; .)
} .

Fact <int *factVal>
= (. int spix; .)
Ident <&spix> (. *factVal = get_var(spix); .)
| number (. *factVal = get_number(); .)
| '(' Expr <factVal> ')' .

Ident <int *spix>
= ident (. *spix = get_spix(); .) .

END Calc.


I hope that this helps.

Regards,

--
-mark. (probertm at acm dot org)


Petite Abeille

1/26/2005 6:13:00 PM

0


On Jan 26, 2005, at 18:08, Hugh Sasse Staff Elec Eng wrote:

> I've googled for things to do with little languages and parsing, but
> have found nothing enlightening.

Perhaps you should take a look at Lua:

http://www.lua.org/...

It started life as a "data entry language" of sort:

http://www.lua.org/hi...

Then read the book:

http://www.lu...

Cheers

--
PA
http://alt.text...



Hugh Sasse

1/26/2005 6:18:00 PM

0

Hugh Sasse

1/26/2005 6:21:00 PM

0