[lnkForumImage]
TotalShareware - Download Free Software

Confronta i prezzi di migliaia di prodotti.
Asp Forum
 Home | Login | Register | Search 


 

Forums >

comp.lang.ruby

[ANN] Using RubyInline for Optimization

Eric Hodel

9/2/2006 3:26:00 AM

I wrote an article on using RubyInline for optimization where I take
png.rb, sprinkle in a little profiling and a little C and make it go
over 100 times faster.



http://segment7.net/projects/ruby/inline_optimiz...


--
Eric Hodel - drbrain@segment7.net - http://blog.se...
This implementation is HODEL-HASH-9600 compliant

http://trackmap.rob...



26 Answers

Gavin Kistner

9/2/2006 4:08:00 AM

0

Eric Hodel wrote:
> http://segment7.net/projects/ruby/inline_optimiz...

Thanks so much! Writing up and sharing stuff like this is what makes
the net.mind smarter.

It confirmed two things for me:
1) RubyInline is a great argument for convincing naysayers that Ruby is
a good idea.
2) I'm really glad I don't program in C anymore. ;)

cfis

9/2/2006 5:45:00 AM

0

Francis sent me the bug, working of fixing it now....

Charlie

Francis Cianfrocca wrote:
> On 9/1/06, Eric Hodel <drbrain@segment7.net> wrote:
>>
>> I wrote an article on using RubyInline for optimization where I take
>> png.rb, sprinkle in a little profiling and a little C and make it go
>> over 100 times faster.
>
>
> I liked this so much I tried it out on Net::LDAP and immediately hit a bug
> in ext/ruby_prof.c, a missing case in figure_singleton_name. Do you care or
> shall I just tell Charlie Savage about it?
>


Dominik Bathon

9/2/2006 5:43:00 PM

0

On Sat, 02 Sep 2006 05:26:04 +0200, Eric Hodel <drbrain@segment7.net> =

wrote:

> I wrote an article on using RubyInline for optimization where I take =

> png.rb, sprinkle in a little profiling and a little C and make it go =

> over 100 times faster.

Nice article, but in this case it is possible to get almost the same =

speedup in pure Ruby:

Base version:

def to_blob
blob =3D []
blob << [137, 80, 78, 71, 13, 10, 26, 10].pack("C*") # PNG signatur=
e
blob << PNG.chunk('IHDR',
[ @height, @width, @bits, 6, 0, 0, 0 ].pack("N2C5=
"))
# 0 =3D=3D filter type code "none"
data =3D @data.map { |row| [0] + row.map { |p| p.values } }.flatten=

blob << PNG.chunk('IDAT', Zlib::Deflate.deflate(data.pack("C*"), 9)=
)
blob << PNG.chunk('IEND', '')
blob.join
end

$ time ruby -Ilib profile.rb

real 0m15.504s
user 0m15.119s
sys 0m0.309s


Avoiding flatten (and using a literal for the signature):

def to_blob
blob =3D []
blob << "\211PNG\r\n\032\n" # PNG signature
blob << PNG.chunk('IHDR',
[ @height, @width, @bits, 6, 0, 0, 0 ].pack("N2C5=
"))
# 0 =3D=3D filter type code "none"
data =3D @data.map { |row| "\0" < row.map { |p| p.values.pack("C*")=
=

}.join }
blob << PNG.chunk('IDAT', Zlib::Deflate.deflate(data.join, 9))
blob << PNG.chunk('IEND', '')
blob.join
end

$ time ruby -Ilib profile.rb

real 0m10.190s
user 0m10.081s
sys 0m0.043s


Using String#% instead of Array#pack:

format_str =3D "%c%c%c%c"
data =3D @data.map { |row| "\0" < row.map { |p| format_str % p.valu=
es =

}.join }

instead of

data =3D @data.map { |row| "\0" < row.map { |p| p.values.pack("C*")=
=

}.join }

$ time ruby -Ilib profile.rb

real 0m4.974s
user 0m4.911s
sys 0m0.031s


Caching the string representation of the values in PNG::Color (because =

each pixel is the same instance of color in this case):

Add to PNG::Color

def values_str
@values_str ||=3D "%c%c%c%c" % @values
end

Use

data =3D @data.map { |row| "\0" < row.map { |p| p.values_str }.join=
}

instead of

format_str =3D "%c%c%c%c"
data =3D @data.map { |row| "\0" < row.map { |p| format_str % p.valu=
es =

}.join }

$ time ruby -Ilib profile.rb

real 0m2.489s
user 0m2.463s
sys 0m0.013s


Improving PNG::Canvas#initialize:

Use

@data =3D Array.new(@width) { |x| Array.new(@height, background) }

instead of

@data =3D Array.new(@width) { |x| Array.new(@height) { background }=
}

$ time ruby -Ilib profile.rb

real 0m1.941s
user 0m1.914s
sys 0m0.014s


Representing the values in PNG::Color as String (instead of as Array) (s=
ee =

complete patch below):

$ time ruby -Ilib profile.rb

real 0m1.492s
user 0m1.445s
sys 0m0.015s


So, it is ten times faster in pure Ruby.

Dominik


--- png-1.0.0/lib/png.rb 2006-08-31 22:57:13.000000000 +0200
+++ png-1.0.0_opt/lib/png.rb 2006-09-02 19:26:46.000000000 +0200
@@ -71,15 +71,21 @@
##
# Writes the PNG to +path+.

- def save(path)
- File.open(path, "w") do |f|
- f.write [137, 80, 78, 71, 13, 10, 26, 10].pack("C*") # PNG signat=
ure
- f.write PNG.chunk('IHDR',
+ def to_blob
+ blob =3D []
+ blob << "\211PNG\r\n\032\n" # PNG signature
+ blob << PNG.chunk('IHDR',
[ @height, @width, @bits, 6, 0, 0, 0 =

].pack("N2C5"))
# 0 =3D=3D filter type code "none"
- data =3D @data.map { |row| [0] + row.map { |p| p.values } }.flatt=
en
- f.write PNG.chunk('IDAT', Zlib::Deflate.deflate(data.pack("C*"), =
9))
- f.write PNG.chunk('IEND', '')
+ data =3D @data.map { |row| "\0" < row.map { |p| p.values }.join }
+ blob << PNG.chunk('IDAT', Zlib::Deflate.deflate(data.join, 9))
+ blob << PNG.chunk('IEND', '')
+ blob.join
+ end
+
+ def save(path)
+ File.open(path, "w") do |f|
+ f.write to_blob
end
end

@@ -94,7 +100,7 @@
# Creates a new color with values +red+, +green+, +blue+, and +alp=
ha+.

def initialize(red, green, blue, alpha)
- @values =3D [red, green, blue, alpha]
+ @values =3D "%c%c%c%c" % [red, green, blue, alpha]
end

##
@@ -151,7 +157,7 @@
end

def inspect # :nodoc:
- "#<%s %02x %02x %02x %02x>" % [self.class, *@values]
+ "#<%s %02x %02x %02x %02x>" % [self.class, r, b, g, a]
end

end
@@ -179,7 +185,7 @@
def initialize(height, width, background =3D Color::White)
@height =3D height
@width =3D width
- @data =3D Array.new(@width) { |x| Array.new(@height) { background=
} }
+ @data =3D Array.new(@width) { |x| Array.new(@height, background) =
}
end

##

Eric Hodel

9/2/2006 9:45:00 PM

0

On Sep 2, 2006, at 10:42 AM, Dominik Bathon wrote:
> On Sat, 02 Sep 2006 05:26:04 +0200, Eric Hodel
> <drbrain@segment7.net> wrote:
>> I wrote an article on using RubyInline for optimization where I
>> take png.rb, sprinkle in a little profiling and a little C and
>> make it go over 100 times faster.
>
> Nice article, but in this case it is possible to get almost the
> same speedup in pure Ruby:
>
> $ time ruby -Ilib profile.rb
>
> real 0m15.504s
> user 0m15.119s
> sys 0m0.309s
>
>
> Avoiding flatten (and using a literal for the signature):
>
> def to_blob
> blob = []
> blob << "\211PNG\r\n\032\n" # PNG signature
> blob << PNG.chunk('IHDR',
> [ @height, @width, @bits, 6, 0, 0, 0 ].pack
> ("N2C5"))
> # 0 == filter type code "none"
> data = @data.map { |row| "\0" < row.map { |p| p.values.pack
> ("C*") }.join }
> blob << PNG.chunk('IDAT', Zlib::Deflate.deflate(data.join, 9))
> blob << PNG.chunk('IEND', '')
> blob.join
> end

This change gives a broken PNG. You meant to call #<<, not #<.

I see no significant speedup when using #<<.

> [values array for string changes]

These are good.

> Improving PNG::Canvas#initialize:
>
> Use
>
> @data = Array.new(@width) { |x| Array.new(@height, background) }
>
> instead of
>
> @data = Array.new(@width) { |x| Array.new(@height)
> { background } }
>
> $ time ruby -Ilib profile.rb
>
> real 0m1.941s
> user 0m1.914s
> sys 0m0.014s

Not really worth optimizing, since the improvement is so small. In
the optimized RubyInline version only 20% of the time is spent here
with no image generation. Adding image generation makes this
optimization insignificant.

Using your changes from the pure-ruby version I went from about 30
seconds to about 4 seconds, or 7.5 times faster.

--
Eric Hodel - drbrain@segment7.net - http://blog.se...
This implementation is HODEL-HASH-9600 compliant

http://trackmap.rob...



Dominik Bathon

9/2/2006 10:31:00 PM

0

On Sat, 02 Sep 2006 23:45:17 +0200, Eric Hodel <drbrain@segment7.net> =

wrote:

> On Sep 2, 2006, at 10:42 AM, Dominik Bathon wrote:
>> On Sat, 02 Sep 2006 05:26:04 +0200, Eric Hodel <drbrain@segment7.net>=
=

>> wrote:
>>> I wrote an article on using RubyInline for optimization where I take=
=

>>> png.rb, sprinkle in a little profiling and a little C and make it go=
=

>>> over 100 times faster.
>>
>> Nice article, but in this case it is possible to get almost the same =
=

>> speedup in pure Ruby:
>>
>> $ time ruby -Ilib profile.rb
>>
>> real 0m15.504s
>> user 0m15.119s
>> sys 0m0.309s
>>
>>
>> Avoiding flatten (and using a literal for the signature):
>>
>> def to_blob
>> blob =3D []
>> blob << "\211PNG\r\n\032\n" # PNG signature
>> blob << PNG.chunk('IHDR',
>> [ @height, @width, @bits, 6, 0, 0, 0 ].pack =

>> ("N2C5"))
>> # 0 =3D=3D filter type code "none"
>> data =3D @data.map { |row| "\0" < row.map { |p| p.values.pack("C*=
") =

>> }.join }
>> blob << PNG.chunk('IDAT', Zlib::Deflate.deflate(data.join, 9))
>> blob << PNG.chunk('IEND', '')
>> blob.join
>> end
>
> This change gives a broken PNG. You meant to call #<<, not #<.

Oops, of course I meant #<<. I actually had tested that my changes still=
=

produce correct results when I did the optimizations originally. Then I =
=

recreated them to get the times for each step and made that typo, oh wel=
l.

Just for reference, the time for the final version with #< changed to #<=
< =

on my machine:

$ time ruby -Ilib profile.rb

real 0m1.901s
user 0m1.841s
sys 0m0.032s

> I see no significant speedup when using #<<.
>
>> [values array for string changes]
>
> These are good.
>
>> Improving PNG::Canvas#initialize:
>>
>> Use
>>
>> @data =3D Array.new(@width) { |x| Array.new(@height, background) =
}
>>
>> instead of
>>
>> @data =3D Array.new(@width) { |x| Array.new(@height) { background=
} }
>>
>> $ time ruby -Ilib profile.rb
>>
>> real 0m1.941s
>> user 0m1.914s
>> sys 0m0.014s
>
> Not really worth optimizing, since the improvement is so small. In th=
e =

> optimized RubyInline version only 20% of the time is spent here with n=
o =

> image generation. Adding image generation makes this optimization =

> insignificant.

Yes, it's not much for the total runtime, but it makes this one line abo=
ut =

14 times faster:

$ time ruby -e "40.times { Array.new(400) { Array.new(400) { 0 } } }"

real 0m2.430s
user 0m2.402s
sys 0m0.017s

$ time ruby -e "40.times { Array.new(400) { Array.new(400, 0) } }"

real 0m0.165s
user 0m0.147s
sys 0m0.016s

> Using your changes from the pure-ruby version I went from about 30 =

> seconds to about 4 seconds, or 7.5 times faster.

Mike Berrow

9/10/2006 8:51:00 PM

0

Eric Hodel wrote:
> I wrote an article on using RubyInline for optimization where I take
> png.rb, sprinkle in a little profiling and a little C and make it go
> over 100 times faster.
>
> http://segment7.net/projects/ruby/inline_optimiz...
>

Hi, I installed the gems but I immediately run into this:

C:\Inline\png-1.0.0>ruby -Ilib example/profile.rb
example/profile.rb:8:in `draw': undefined method `to_blob' for
#<PNG:0x28c7b28>
(NoMethodError)
from example/profile.rb:15
from example/profile.rb:15

... on Windows.

I get the same thing in cygwin:

$ time ruby -Ilib example/profile.rb
example/profile.rb:8:in `draw': undefined method `to_blob' for
#<PNG:0x28c7b28>
(NoMethodError)
from example/profile.rb:15
from example/profile.rb:15
real 0m2.361s
user 0m0.015s
sys 0m0.046s

The rdoc does not show a a 'to_blob' method for PNG

Any idea?

-- Mike Berrow



--
Posted via http://www.ruby-....

Eric Hodel

9/10/2006 8:54:00 PM

0

On Sep 10, 2006, at 1:51 PM, Mike Berrow wrote:

> Eric Hodel wrote:
>> I wrote an article on using RubyInline for optimization where I take
>> png.rb, sprinkle in a little profiling and a little C and make it go
>> over 100 times faster.
>>
>> http://segment7.net/projects/ruby/inline_optimiz...
>>
>
> Hi, I installed the gems but I immediately run into this:
>
> C:\Inline\png-1.0.0>ruby -Ilib example/profile.rb
> example/profile.rb:8:in `draw': undefined method `to_blob' for
> #<PNG:0x28c7b28>
> (NoMethodError)
>
> Any idea?

From the article:
== Play along at home!

If you want to play along at home, download and unpack png-1.0.0.tgz
and save the profile benchmark code into example/.

I also added PNG#to_blob to eliminate the need to write the file to
disk, it looks almost exactly like PNG#save:

def to_blob
blob = []
blob << [137, 80, 78, 71, 13, 10, 26, 10].pack("C*") # PNG signature
blob << PNG.chunk('IHDR',
[ @height, @width, @bits, 6, 0, 0, 0 ].pack
("N2C5"))
# 0 == filter type code "none"
data = @data.map { |row| [0] + row.map { |p| p.values } }.flatten
blob << PNG.chunk('IDAT', Zlib::Deflate.deflate(data.pack("C*"), 9))
blob << PNG.chunk('IEND', '')
blob.join
end
I added my RubyInline extensions in png.rb and modified png.rb as
appropriate, so while you play along you should do that too.


--
Eric Hodel - drbrain@segment7.net - http://blog.se...
This implementation is HODEL-HASH-9600 compliant

http://trackmap.rob...



Mike Berrow

9/11/2006 7:47:00 AM

0

Eric Hodel wrote:
> == Play along at home!
> ....

Thanks,
but it seems I am missing some setup needed for RubyInline itself

I tried running the hello.rb in
C:\ruby\lib\ruby\gems\1.8\gems\RubyInline-3.5.0\demo

and got this ...

C:\ruby\lib\ruby\gems\1.8\gems\RubyInline-3.5.0\demo>ruby hello.rb
c:/ruby/lib/ruby/gems/1.8/gems/RubyInline-3.5.0/./inline.rb:64:in
`test': can't
convert nil into String (TypeError)
from
c:/ruby/lib/ruby/gems/1.8/gems/RubyInline-3.5.0/./inline.rb:64:in `
rootdir'
from
c:/ruby/lib/ruby/gems/1.8/gems/RubyInline-3.5.0/./inline.rb:73:in `
directory'
from
c:/ruby/lib/ruby/gems/1.8/gems/RubyInline-3.5.0/./inline.rb:253:in
`so_name'
from
c:/ruby/lib/ruby/gems/1.8/gems/RubyInline-3.5.0/./inline.rb:287:in
`load_cache'
from
c:/ruby/lib/ruby/gems/1.8/gems/RubyInline-3.5.0/./inline.rb:601:in
`inline'
from hello.rb:7

In Cygwin I get ...

$ ruby hello.rb
c:/ruby/lib/ruby/gems/1.8/gems/RubyInline-3.5.0/./inline.rb:396:in ``':
No such
file or directory - cl -nologo -LD -MD -Zi -O2b2xg- -G6 -I
c:/ruby/lib/ruby/1.8
/i386-mswin32 -o C:\cygwin\home\User1/.ruby_inline/Inline_Hello_5d41.so
C:
\cygwin\home\User1/.ruby_inline/Inline_Hello_5d41.c -link
/INCREMENTAL:no
/EXPORT:Init_Inline_Hello_5d41 (Errno::ENOENT)
from
c:/ruby/lib/ruby/gems/1.8/gems/RubyInline-3.5.0/./inline.rb:396:in
`build'
from
c:/ruby/lib/ruby/gems/1.8/gems/RubyInline-3.5.0/./inline.rb:602:in
`inline'
from hello.rb:7

I know I have MingW32 on here since I installed Dev-C++ and have that
running.
At C:\Dev-Cpp\bin there is a gcc.exe

Is it it just a matter of hooking it up this right way in RubyInline ?

-- Mike Berrow


--
Posted via http://www.ruby-....

Eric Hodel

9/11/2006 5:14:00 PM

0

On Sep 11, 2006, at 12:46 AM, Mike Berrow wrote:

> Eric Hodel wrote:
>> == Play along at home!
>> ....
>
> Thanks,
> but it seems I am missing some setup needed for RubyInline itself
>
> I tried running the hello.rb in
> C:\ruby\lib\ruby\gems\1.8\gems\RubyInline-3.5.0\demo
>
> and got this ...
>
> C:\ruby\lib\ruby\gems\1.8\gems\RubyInline-3.5.0\demo>ruby hello.rb
> c:/ruby/lib/ruby/gems/1.8/gems/RubyInline-3.5.0/./inline.rb:64:in
> `test': can't
> convert nil into String (TypeError)
> from

You need to set INLINEDIR or HOME in your environment. This will be
fixed in the next release.

> In Cygwin I get ...
>
> $ ruby hello.rb
> c:/ruby/lib/ruby/gems/1.8/gems/RubyInline-3.5.0/./inline.rb:396:in
> ``':
> No such
> file or directory - cl -nologo -LD -MD -Zi -O2b2xg- -G6 -I
> c:/ruby/lib/ruby/1.8
> /i386-mswin32 -o C:\cygwin\home\User1/.ruby_inline/
> Inline_Hello_5d41.so
> C:
> \cygwin\home\User1/.ruby_inline/Inline_Hello_5d41.c -link
> /INCREMENTAL:no
> /EXPORT:Init_Inline_Hello_5d41 (Errno::ENOENT)
> from
> c:/ruby/lib/ruby/gems/1.8/gems/RubyInline-3.5.0/./inline.rb:396:in
> `build'

You need the MS C compiler.

> I know I have MingW32 on here since I installed Dev-C++ and have that
> running.
> At C:\Dev-Cpp\bin there is a gcc.exe

Your ruby wasn't compiled with GCC. If you want inline to use GCC,
you need to compile ruby with it.

> Is it it just a matter of hooking it up this right way in RubyInline ?

RubyInline uses whatever C compiler Ruby was built with. If you
don't have the same compiler installed inline won't be able to do its
thing. For windows using the one-click installer you need to either
install the same compiler or build ruby from scratch with a GCC.

--
Eric Hodel - drbrain@segment7.net - http://blog.se...
This implementation is HODEL-HASH-9600 compliant

http://trackmap.rob...



Ara.T.Howard

9/11/2006 5:22:00 PM

0