Asp Forum - String#de_inspect (and Kernel#suspicious

Erik Veenstra

1/25/2006 10:22:00 AM

If you do an inspect on a collection of Ruby objects, like a
hash, you end up with a string. It's possible to store this
string in a file, read it again somewhere in the future,
evaluate it and end up with the same collection of Ruby objects
in core.

So I've written this String#de_inspect, which uses
Kernel#suspicious (slow!) to avoid any malicious code from
being evaluated.

A kind of human-readable marshaling. That is-human-readable is
important, for me, in this situation.

(You can only dump objects which inspect to Ruby code, e.g.
Strings, Numerics, Symbols, Arrays, Hashes, nil, true and
false.)

I've attached the code and an example, though the example isn't
important.

Thoughts? Comments?

gegroet,
Erik V. - http://www.erikve...

----------------------------------------------------------------

module Kernel
def suspicious(*parms, &block) # Just forget about the parms...
Thread.new(*parms) do |*parms|
$SAFE = 5

block.call(*parms)
end.value
end
end

class String
def de_inspect
suspicious do
eval(self, Module.new.module_eval{binding})
end
end
end

def journal(file)
File.open(file) do |f|
while (line = f.gets)
yield(line.de_inspect)
end
end
end

journal("journal") do |x|
p x
end

----------------------------------------------------------------

7 Answers

Robert Klemme

1/25/2006 10:30:00 AM

Erik Veenstra wrote:
> If you do an inspect on a collection of Ruby objects, like a
> hash, you end up with a string. It's possible to store this
> string in a file, read it again somewhere in the future,
> evaluate it and end up with the same collection of Ruby objects
> in core.
>
> So I've written this String#de_inspect, which uses
> Kernel#suspicious (slow!) to avoid any malicious code from
> being evaluated.
>
> A kind of human-readable marshaling. That is-human-readable is
> important, for me, in this situation.
>
> (You can only dump objects which inspect to Ruby code, e.g.
> Strings, Numerics, Symbols, Arrays, Hashes, nil, true and
> false.)
>
> I've attached the code and an example, though the example isn't
> important.
>
> Thoughts? Comments?

A question: what is the advantage of this over YAML?

Kind regards

robert

Erik Veenstra

1/25/2006 11:47:00 AM

> A question: what is the advantage of this over YAML?

1) It's faster (see below). Probably because it uses the highly
optimized parser/lexer/whatever of the Ruby interpreter
itself. (You can turn off the suspicious mode if the data
can be trusted, which makes it faster then YAML. If the
suspicious is enabled, it's as fast as YAML.)

2) Memory (suspicious mode turned off) (see below).

3) It's small, whereas YAML is relatively huge. (Is being small
an advantage? Not necessarily, but I mention it anyway...)

4) You can store not only raw data, but code as well. (I know,
this is really DANGEROUS, like macros in Word. That's why I
introduced Kernel#suspicious.)

5) I my real situation, I raise an exception if the line, read
from the journal, doesn't end with \r, \n or both. This is
an indication for a corrupted journal. Half a line in the
journal could be valid Ruby code and, as such, appear to be
valid data. That's why I check for the "commit". (Maybe YAML
does this too. I don't know.)

In my case, where the data is only accessible via a dedicated
daemon on a server, I can turn off the suspicious mode. That's
the big win.

gegroet,
Erik V. - http://www.erikve...

----------------------------------------------------------------

$ wc test.rbo test.yaml # SAME DATA!
3077 26739 681698 test.rbo
29816 62709 697071 test.yaml

$ ruby test.rb 10 # 10 times
CPU ELAPSED COUNT CPU/INSTANCE LABEL
3.770000 4.016498 1 3.770000 :yaml
3.630000 3.862287 1 3.630000 :rbo
1.140000 1.140604 1 1.140000 :rbo_fast

$ ruby testmem.rb rbo # Disable GC, load testset once.
VmSize: 21988 kB

$ ruby testmem.rb rbo_fast # Disable GC, load testset once.
VmSize: 10904 kB

$ ruby testmem.rb yaml # Disable GC, load testset once.
VmSize: 18004 kB

----------------------------------------------------------------

Mauricio Fernández

1/25/2006 11:59:00 AM

On Wed, Jan 25, 2006 at 07:23:12PM +0900, Erik Veenstra wrote:
> So I've written this String#de_inspect, which uses
> Kernel#suspicious (slow!) to avoid any malicious code from
> being evaluated.
[...]

### code by Mr. Evil

File.open("journal", "w") do |f|
f.puts <<-EOF.gsub("\n", ";")
def (o=Object.new).inspect
puts "gotcha! I'm running in $SAFE=\#{$SAFE}"
puts "Fear my rm -rf"
'"Just an innocent little string"'
end
o
EOF
end

# back to your code
module Kernel
def suspicious(*parms, &block) # Just forget about the parms...
Thread.new(*parms) do |*parms|
$SAFE = 5

block.call(*parms)
end.value
end
end

class String
def de_inspect
suspicious do
eval(self, Module.new.module_eval{binding})
end
end
end

def journal(file)
File.open(file) do |f|
while (line = f.gets)
yield(line.de_inspect)
end
end
end

journal("journal") do |x|
p x
end
# >> gotcha! I'm running in $SAFE=0
# >> Fear my rm -rf
# >> "Just an innocent little string"

--
Mauricio Fernandez

Robert Klemme

1/25/2006 12:01:00 PM

Erik Veenstra wrote:
>> A question: what is the advantage of this over YAML?
>
> 1) It's faster (see below). Probably because it uses the highly
> optimized parser/lexer/whatever of the Ruby interpreter
> itself. (You can turn off the suspicious mode if the data
> can be trusted, which makes it faster then YAML. If the
> suspicious is enabled, it's as fast as YAML.)
>
> 2) Memory (suspicious mode turned off) (see below).
>
> 3) It's small, whereas YAML is relatively huge. (Is being small
> an advantage? Not necessarily, but I mention it anyway...)
>
> 4) You can store not only raw data, but code as well. (I know,
> this is really DANGEROUS, like macros in Word. That's why I
> introduced Kernel#suspicious.)
>
> 5) I my real situation, I raise an exception if the line, read
> from the journal, doesn't end with \r, \n or both. This is
> an indication for a corrupted journal. Half a line in the
> journal could be valid Ruby code and, as such, appear to be
> valid data. That's why I check for the "commit". (Maybe YAML
> does this too. I don't know.)

That's quite an impressive list. I'm glad I asked.

Kind regards

robert

Erik Veenstra

1/25/2006 12:26:00 PM

Okay, the block was defined in SAFE mode 0... :)

In the first version, I didn't introduce Kernel#suspicious (see
below). That one worked fine.

Then I naively abstracted the thread thing, which didn't do
what I expected. Oops...

Back to version 1...

Thanks.

gegroet,
Erik V. - http://www.erikve...

----------------------------------------------------------------

File.open("journal", "w") do |f|
f.puts "[:SAFE => $SAFE]"
end

# back to your code

class String
def de_inspect
Thread.new do
$SAFE = 5

eval(self, Module.new.module_eval{binding})
end.value
end
end

def journal(file)
File.open(file) do |f|
while (line = f.gets)
yield(line.de_inspect)
end
end
end

journal("journal") do |x|
p x
end

----------------------------------------------------------------

Joel VanderWerf

1/25/2006 7:36:00 PM

Erik Veenstra wrote:
> If you do an inspect on a collection of Ruby objects, like a
> hash, you end up with a string. It's possible to store this
> string in a file, read it again somewhere in the future,
> evaluate it and end up with the same collection of Ruby objects
> in core.
>
> So I've written this String#de_inspect, which uses
> Kernel#suspicious (slow!) to avoid any malicious code from
> being evaluated.
>
> A kind of human-readable marshaling. That is-human-readable is
> important, for me, in this situation.
>
> (You can only dump objects which inspect to Ruby code, e.g.
> Strings, Numerics, Symbols, Arrays, Hashes, nil, true and
> false.)

For the object->string direction, it may be useful to use amarshal,
rather than inspect:

http://cvs.m17n.org/~akr...

One advantage is with cyclic references. Using #inspect will not
preserve enough information to reconstruct the reference.

--
vjoel : Joel VanderWerf : path berkeley edu : 510 665 3407

Erik Veenstra

1/25/2006 9:05:00 PM

> For the object-> string direction, it may be useful to use
> amarshal, rather than inspect:
>
> http://cvs.m17n.org/~akr...
>
> One advantage is with cyclic references. Using #inspect will
> not preserve enough information to reconstruct the reference.

% ruby -ramarshal -e 'AMarshal.dump([1,2,3], STDOUT)'
v = []
v[0] = Array.allocate()
v[0] << 1
v[0] << 2
v[0] << 3
v[0]

This idea is really nice, indeed.

gegroet,
Erik V. - http://www.erikve...

comp.lang.ruby

String#de_inspect (and Kernel#suspicious

Erik Veenstra

Robert Klemme

Erik Veenstra

Mauricio Fernández

Robert Klemme

Erik Veenstra

Joel VanderWerf

Erik Veenstra

x Login to ForumsZone