Asp Forum - [ANN] FastRI 0.1.0: faster, smarter RI docs for Ruby, DRb-enabled

Mauricio Fernández

11/8/2006 12:19:00 PM

FastRI 0.1.0: faster, smarter RI docs for Ruby, DRb-enabled

FastRI can be installed with RubyGems:
gem install fastri
(if you get an old version/a 404 error, please allow some time after the
release until the package propagates to the RubyForge mirrors). Please read
below for an important note regarding the RubyGems packages.

Additional information, tarballs... at
http://eige.../hiki...

Overview
========
FastRI is an alternative to the ri command-line tool. It is *much* faster, and
also allows you to offer RI lookup services over DRb. FastRI is smarter than
ri, and can find classes anywhere in the hierarchy without specifying the
"full path". It also knows about gems, and can tell you e.g. which extensions
to a core class were added by a specific gem.

Usage
=====
There are two parts to FastRI:
* the server: fastri-server
* the client: fri

FastRI uses a Rinda Ring to allow servers to be discovered automatically
without needing to indicate the DRb URIs manually. It can work across
machines if you make sure the ring server is bound to the correct interface,
and the ACL permissions are correct.

Examples
========
$ fastri-server (creates the index on the first run, blocks)

Later,
$ fri Array#fetch

Speed
=====
fri is much faster than ri; here are some times measured with a cold cache:

$ time fri -f plain Array#fetch
------------------------------------------------------------ Array#fetch
array.fetch(index) -> obj
[...]
real 0m0.287s (real 0m0.127s with a hot cache)
user 0m0.048s
sys 0m0.008s

Compare to:
$ time ri -T -f plain Array#fetch
------------------------------------------------------------ Array#fetch
[...]
real 0m10.136s (real ~ 1.5s with a hot cache)
user 0m1.140s
sys 0m0.464s

Smart searches
==============
This illustrates FastRI's ability to locate classes deep in the class
hierarchy:

$ fri Base
------------------------------------------------------ Multiple choices:

ActionMailer::Base, ActionView::Base, ActionWebService::API::Base,
ActionWebService::Base, ActionWebService::Client::Base,
ActiveRecord::Base, MapReduce::ActiveRecord::Base,
RSS::Maker::Base, Scruffy::Components::Base,
Scruffy::Formatters::Base, Scruffy::Layers::Base,
Scruffy::Renderers::Base, Scruffy::Themes::Base

$ fri Themes::Base
------------------------------------------- Class: Scruffy::Themes::Base
Scruffy::Themes::Base
Author: Brasten Sager

Date: August 14th, 2006

Compare to
$ ri Themes::Base .... several seconds later ...
Nothing known about Themes::Base

A note about RubyGems + FastRI
==============================
RubyGems adds a noticeable overhead to fri, making it run slower than if you
installed it directly from the tarball with setup.rb.

Compare the execution time when installed with RubyGems:
$ time fri -f plain String > /dev/null

real 0m0.385s
user 0m0.244s
sys 0m0.036s

to the time fri actually takes to run, without the overhead introduced by
RubyGems:
$ time ruby bin/fri -f plain String > /dev/null

real 0m0.088s
user 0m0.040s
sys 0m0.008s

If you care about those extra 300ms (and there are situations where they will
matter, e.g. when using fri for method completion), get FastRI from the
tarballs.

Thanks
======
rubikitch
---------
* command-line options to load/build the index
* testing and suggestions

Kristof Bastiaensen
-------------------
For showing the way with ri-emacs.rb.

License
=======
FastRI is licensed under the same terms as Ruby. See LICENSE.

Feedback
========
Bug reports, patches, comments... are appreciated.
You can contact the author via <mfp@acm.org>. Please add "fastri" to the
subject in order to bypass the spam filters.

--
Mauricio Fernandez - http://eige... - singular Ruby

23 Answers

Leslie Viljoen

11/8/2006 3:31:00 PM

Thanks for this. I see it's not working so well on Windows yet.
The script is looking for directory '~' and there are some display problems.

I also saw another bug when looking for things that are not indexed,
but I'll test it on Linux/BSD before I say any more.

Les

Mauricio Fernández

11/8/2006 7:33:00 PM

On Thu, Nov 09, 2006 at 12:31:29AM +0900, Leslie Viljoen wrote:
> Thanks for this. I see it's not working so well on Windows yet.
> The script is looking for directory '~' and there are some display problems.

I have pushed a patch to the 0.1 branch
(http://eige.../repos/f... ) that solves the HOME issue. Until
that gets released, specifying the file as in
fastri-server --index-file c:/tmp/fastri-index
should work.

As for the "display problems", are they due to ANSI escape sequences? If so,
fri -f plain should do. If win32 can't handle ANSI escape codes, I can set
the default format based on the platform at runtime.

BTW, I'd appreciate feedback regarding execution times, both for indexing and
lookups (and if possible, both with cold & hot cache).

On an older K7 XP 1800+ running Linux, it takes under 10 seconds to index 36
gems (out of 88 installed ones --- I grabbed a bunch for testing purposes;
multiple versions are coalesced and some packages have no documentation) with
over 13000 entries (classes/modules & methods), and under 2 seconds when the
OS' cache is doing its work. I bet it'll be slower on win32, IO is much slower
in general (ri+RubyGems must be The Pain).

(note: the docs will only be indexed once, unless you remove the index file or
specify another one; otherwise it will be reused in successive runs)

$ fastri-server
Indexing RI docs for ParseTree version 1.5.0.
Indexing RI docs for Rubilicious version 0.2.0.
Indexing RI docs for ZenTest version 3.4.1.
Indexing RI docs for actionmailer version 1.2.5.
Indexing RI docs for actionpack version 1.12.5.
Indexing RI docs for actionwebservice version 1.1.6.
Indexing RI docs for activerecord version 1.14.4.
Indexing RI docs for activesupport version 1.3.1.
Indexing RI docs for builder version 2.0.0.
Indexing RI docs for call_stack version 0.1.0.0.
Indexing RI docs for cmdparse version 2.0.2.
Indexing RI docs for color-tools version 1.3.0.
Indexing RI docs for dev-utils version 1.0.1.
Indexing RI docs for extensions version 0.6.0.
Indexing RI docs for fastercsv version 0.2.1.
Indexing RI docs for fastri version 0.1.0.1.
Indexing RI docs for highline version 1.2.1.
Indexing RI docs for hoe version 1.1.2.
Indexing RI docs for mailfactory version 1.2.3.
Indexing RI docs for mime-types version 1.15.
Indexing RI docs for pdf-writer version 1.1.3.
Indexing RI docs for rails_analyzer_tools version 1.1.0.
Indexing RI docs for rake version 0.7.1.
Indexing RI docs for rcov version 0.7.0.1.
Indexing RI docs for rspec version 0.6.4.
Indexing RI docs for ruby-breakpoint version 0.5.1.
Indexing RI docs for ruby-debug version 0.4.2.
Indexing RI docs for rubyforge version 0.3.1.
Indexing RI docs for rubyful_soup version 1.0.4.
Indexing RI docs for ruport version 0.5.3.
Indexing RI docs for scruffy version 0.2.2.
Indexing RI docs for sqlite3-ruby version 1.1.0.
Indexing RI docs for starfish version 1.2.0.
Indexing RI docs for sys-proctable version 0.7.3.
Indexing RI docs for tidy version 1.1.2.
Indexing RI docs for transaction-simple version 1.3.0.
Building index.
Indexed:
* 11131 methods
* 2287 classes/modules
Needed 9.553382 seconds
Looking for Ring server...
Located Ring server at druby://127.0.0.1:50275
fastri-server 0.0.1 (FastRI 0.1.0) listening on druby://127.0.0.1:42913
ACL:
deny all
allow 127.0.0.1

$ fastri-server --index-file /tmp/Index2
Indexing RI docs for ParseTree version 1.5.0.
[...]
Building index.
Indexed:
* 11131 methods
* 2287 classes/modules
Needed 1.761658 seconds
Looking for Ring server...

--
Mauricio Fernandez - http://eige... - singular Ruby

Peña, Botp

11/9/2006 1:48:00 AM

:fr Mauricio Julio Fernández Pradier
# FastRI 0.1.0: faster, smarter RI docs for Ruby, DRb-enabled

another cool lib/app fr Mauricio. Thanks.

btw, Mauricio, i installed using tar setup.rb in windows and it can't find fastri-server and fri. I had to rename both files by appending the rb extension.

it's working like a charm now and indeed it IS VERY FAST. no more pause. thanks again!

kind regards -botp

btw, how do i time it windows? use ruby and benchmark perhaps like,

time_start = Time.now
system ARGV.join(" ")
puts "#{Time.now - time_start} seconds"

?

feature request: i hope fri remove the case sensitivity.
eg,

C:\Documents and Settings\peñaijm>fri string.split
nil

Craig Beck

11/9/2006 3:08:00 AM

I'm trying to serve my docs from one machine and have them available
on the local network, but its failing to start...

~$ fastri-server -a 192.168.1.0 -s 192.168.1.100
Looking for Ring server...
No Ring server found, starting my own.
/usr/local/lib/site_ruby/1.8/rubygems/custom_require.rb:21:in
`require__': Insecure operation - require__ (SecurityError)
from /usr/local/lib/site_ruby/1.8/rubygems/custom_require.rb:
21:in `require'
from /usr/lib/ruby/gems/1.8/gems/fastri-0.1.0.1/bin/fastri-
server:142
from /usr/bin/fastri-server:18

So are there any more docs available? What am I missing from the above?

--
Craig Beck

AIM: kreiggers

Louis J Scoras

11/9/2006 3:47:00 AM

Mauricio;

Very nice! Ri definitely can use the speedup.

One thing I noticed is that it blows up if you don't have rubygems
installed--I know, probably not very likely at this point; I only
noticed because I'm still setting up a new laptop and I saw your note
about the tarball being faster.

lou@aurora:~/src/fastri-0.1.0$ fastri-server
Building index.
/usr/local/lib/site_ruby/1.8/fastri/ri_index.rb:192:in
`rebuild_index': uninitialized constant FastRI::RiIndex::Gem
(NameError)
from /usr/local/lib/site_ruby/1.8/fastri/ri_index.rb:189:in
`rebuild_index'
from /usr/local/lib/site_ruby/1.8/fastri/ri_index.rb:177:in
`new_from_paths'
from /usr/bin/fastri-server:43:in `make_index'
from /usr/bin/fastri-server:93

So I installed gems and now it's working correctly.

Anyway, thanks again for the great work.

--
Lou.

Leslie Viljoen

11/9/2006 8:56:00 AM

On 11/8/06, Mauricio Fernandez <mfp@acm.org> wrote:
> On Thu, Nov 09, 2006 at 12:31:29AM +0900, Leslie Viljoen wrote:

> As for the "display problems", are they due to ANSI escape sequences? If so,
> fri -f plain should do. If win32 can't handle ANSI escape codes, I can set
> the default format based on the platform at runtime.

Yes, it was escape codes. I'll try plain.

> BTW, I'd appreciate feedback regarding execution times, both for indexing and
> lookups (and if possible, both with cold & hot cache).

Here you go, on a 1.6GHz Pentium-M:

Indexing RI docs for ZenTest version 3.4.1.
Indexing RI docs for actionmailer version 1.2.5.
Indexing RI docs for actionpack version 1.12.5.
Indexing RI docs for actionwebservice version 1.1.6.
Indexing RI docs for activerecord version 1.14.4.
Indexing RI docs for activesupport version 1.3.1.
Indexing RI docs for color-tools version 1.3.0.
Indexing RI docs for composite_primary_keys version 0.7.2.
Indexing RI docs for fastercsv version 0.2.1.
Indexing RI docs for fastri version 0.1.0.1.
Indexing RI docs for flexmock version 0.4.3.
Indexing RI docs for fxruby-1.2.6 version mswin32.
Indexing RI docs for fxruby-1.6.0 version mswin32.
Indexing RI docs for gruff version 0.1.2.
Indexing RI docs for hoe version 1.1.2.
Indexing RI docs for log4r version 1.0.5.
Indexing RI docs for mailfactory version 1.2.3.
Indexing RI docs for mime-types version 1.15.
Indexing RI docs for net-ping version 1.1.1.
Indexing RI docs for pdf-writer version 1.1.3.
Indexing RI docs for rake version 0.7.1.
Indexing RI docs for ruby-breakpoint version 0.5.1.
Indexing RI docs for rubyforge version 0.3.1.
Indexing RI docs for ruport version 0.4.21.
Indexing RI docs for transaction-simple version 1.3.0.
Indexing RI docs for win32-clipboard version 0.4.0.
Indexing RI docs for win32-dir-0.3.0 version mswin32.
Indexing RI docs for win32-eventlog-0.4.1 version mswin32.
Indexing RI docs for win32-file-0.5.2 version mswin32.
Indexing RI docs for win32-file-stat-1.2.2 version mswin32.
Indexing RI docs for win32-process-0.4.2 version mswin32.
Indexing RI docs for win32-sapi-0.1.3 version mswin32.
Indexing RI docs for win32-sound version 0.4.0.
Indexing RI docs for windows-pr-0.5.1 version mswin32.
Building index.
Indexed:
* 8391 methods
* 1384 classes/modules
Needed 13.209 seconds
Looking for Ring server...
No Ring server found, starting my own.
fastri-server 0.0.1 (FastRI 0.1.0) listening on druby://127.0.0.1:1310
ACL:
deny all
allow 127.0.0.1

Leslie Viljoen

11/9/2006 9:06:00 AM

I get some strange responses on Windows. I'll test on Linux shortly.

c:\Program Files\Microsoft Visual Studio 8\VC>fri -f plain assert
------------------------------------------------------ Breakpoint#assert
assert(context = nil, &condition) {|| ...}
------------------------------------------------------------------------
This asserts that the block evaluates to true. If it doesn't
evaluate to true a breakpoint will automatically be created at that
execution point.

You can disable assert checking in production code by setting
Breakpoint.optimize_asserts to true. (It will still be enabled when
Ruby is run via the -d argument.)

Example:

person_name = "Foobar"
assert { not person_name.nil? }

Note: If you want to use this method from an unit test, you will
have to call it by its full name, Breakpoint.assert.

c:\Program Files\Microsoft Visual Studio 8\VC>fri -f plain assert_not_same
nil

c:\Program Files\Microsoft Visual Studio 8\VC>fri -f plain Test::Unit
(druby://127.0.0.1:1310) c:/ruby/lib/ruby/1.8/rdoc/ri/ri_descriptions.rb:99:in `
concat': can't convert nil into Array (TypeError)
from (druby://127.0.0.1:1310) c:/ruby/lib/ruby/1.8/rdoc/ri/ri_descriptio
ns.rb:99:in `merge_in'
from (druby://127.0.0.1:1310) c:/ruby/lib/ruby/gems/1.8/gems/fastri-0.1.
0.1/lib/fastri/ri_index.rb:323:in `get_class'
from (druby://127.0.0.1:1310) c:/ruby/lib/ruby/gems/1.8/gems/fastri-0.1.
0.1/lib/fastri/ri_index.rb:319:in `get_class'
from (druby://127.0.0.1:1310) c:/ruby/lib/ruby/gems/1.8/gems/fastri-0.1.
0.1/lib/fastri/ri_service.rb:174:in `info'
from (druby://127.0.0.1:1310) c:/ruby/lib/ruby/gems/1.8/gems/fastri-0.1.
0.1/lib/fastri/ri_service.rb:259:in `capture_stdout'
from (druby://127.0.0.1:1310) c:/ruby/lib/ruby/gems/1.8/gems/fastri-0.1.
0.1/lib/fastri/ri_service.rb:173:in `info'
from (druby://127.0.0.1:1310) c:/ruby/lib/ruby/1.8/drb/drb.rb:1552:in `p
erform_without_block'
from (druby://127.0.0.1:1310) c:/ruby/lib/ruby/1.8/drb/drb.rb:1512:in `p
erform'
... 7 levels...
from (druby://127.0.0.1:1310) c:/ruby/lib/ruby/gems/1.8/gems/fastri-0.1.
0.1/bin/fastri-server:113
from (druby://127.0.0.1:1310) c:/ruby/bin/fastri-server:18
from c:/ruby/lib/ruby/gems/1.8/gems/fastri-0.1.0.1/bin/fri:66
from c:/ruby/bin/fri:18

Leslie Viljoen

11/9/2006 9:38:00 AM

Ah, on Linux the output is a bit different. No error for the third line:

root@kadima:/etc# fri assert
------------------------------------------------------ Multiple choices:

Test::Rails::ControllerTestCase#assert_assigned,
Test::Rails::ControllerTestCase#assert_content_type,
Test::Rails::ControllerTestCase#assert_flash,
Test::Rails::ViewTestCase#assert_error_on,
Test::Rails::ViewTestCase#assert_field,
Test::Rails::ViewTestCase#assert_image,
Test::Rails::ViewTestCase#assert_input,
Test::Rails::ViewTestCase#assert_label,
Test::Rails::ViewTestCase#assert_links_to,
Test::Rails::ViewTestCase#assert_multipart_form,
Test::Rails::ViewTestCase#assert_post_form,
Test::Rails::ViewTestCase#assert_select_tag,
Test::Rails::ViewTestCase#assert_submit,
Test::Rails::ViewTestCase#assert_tag_in_form,
Test::Rails::ViewTestCase#assert_text_area,
Test::Unit::Assertions#assert_empty,
Test::Unit::Assertions#assert_includes,
TestRailsViewTestCase#assert_tag
root@kadima:/etc# fri assert_not_nil
nil
root@kadima:/etc# fri Test::Unit
------------------------------------------------------ Class: Test::Unit
(no description...)
------------------------------------------------------------------------

Peña, Botp

11/9/2006 9:48:00 AM

:fr Leslie Viljoen [mailto:leslieviljoen@gmail.com]
# c:\Program Files\Microsoft Visual Studio 8\VC>fri -f plain Test::Unit
# (druby://127.0.0.1:1310)
# c:/ruby/lib/ruby/1.8/rdoc/ri/ri_descriptions.rb:99:in `
# concat': can't convert nil into Array (TypeError)
# from (druby://127.0.0.1:1310)

fyi, it does not fail here:

C:\Documents and Settings\botp>fri -f plain Test::Unit
------------------------------------------------------ Class: Test::Unit

TEST::UNIT - RUBY UNIT TESTING FRAMEWORK
========================================

Introduction
------------
Unit testing is making waves all over the place, largely due to the
fact that it is a core practice of XP. While XP is great, unit
testing has been around for a long time and has always been a good
idea. One of the keys to good unit testing, though, is not just
writing tests, but having tests. What's the difference? Well, if
you just _write_ a test and throw it away, you have no guarantee
that something won't change later which breaks your code. If, on
the other hand, you _have_ tests (obviously you have to write them
first), and run them as often as possible, you slowly build up a
wall of things that cannot break without you immediately knowing

[snip long text]

C:\Documents and Settings\botp>ver
Microsoft Windows XP [Version 5.1.2600]

C:\Documents and Settings\botp>ruby -v
ruby 1.8.5 (2006-08-25) [i386-mswin32]

kind regards -botp

Mauricio Fernández

11/9/2006 10:08:00 AM

On Thu, Nov 09, 2006 at 10:47:40AM +0900, Peña, Botp wrote:
> :fr Mauricio Julio Fernández Pradier
> # FastRI 0.1.0: faster, smarter RI docs for Ruby, DRb-enabled
> btw, Mauricio, i installed using tar setup.rb in windows and it can't find
> fastri-server and fri. I had to rename both files by appending the .rb
> extension.

I'll try to fix that in one of setup.rb's hooks, I'm always bumping into it :)

> btw, how do i time it windows? use ruby and benchmark perhaps like,
>
> time_start = Time.now
> system ARGV.join(" ")
> puts "#{Time.now - time_start} seconds"
>

What about something like this:

$ cat runner.rb
#!/usr/bin/env ruby

program = ARGV.shift

unless program
puts <<E
ruby runner.rb <progname> [arguments]
E
exit
end

found = false
t0 = Time.new

at_exit do
tms = Process.times
$stderr.puts <<EOF

real #{Time.new - t0}
user #{tms.cutime}
sys #{tms.cstime}
EOF
end

system("ruby", "-S", program, *ARGV)

__END__

I don't know if cutime and cstime are meaningful on win32, though.

> feature request: i hope fri remove the case sensitivity.
> eg,
>
> C:\Documents and Settings\peñaijm>fri string.split
> nil

Seems convenient, I'll implement it.

--
Mauricio Fernandez - http://eige... - singular Ruby

comp.lang.ruby

[ANN] FastRI 0.1.0: faster, smarter RI docs for Ruby, DRb-enabled

Mauricio Fernández

Leslie Viljoen

Mauricio Fernández

Peña, Botp

Craig Beck

Louis J Scoras

Leslie Viljoen

Leslie Viljoen

Leslie Viljoen

Peña, Botp

Mauricio Fernández

x Login to ForumsZone