Asp Forum - Why is rake really slow?

richpoirier

8/14/2007 5:05:00 AM

Is rake supposed to be really slow?

I'm using InstantRails and running rake in a project with one unit
test and two functional tests and it takes 33 seconds total. Is that
normal? It seems really slow to me.

When I turn on trace I can see that it spends most of its time in
"Execute environment" and loading the test files:

C:/InstantRails/ruby/bin/ruby -Ilib;test "C:/InstantRails/ruby/lib/
ruby/gems/1.8/gems/rake-0.7.2/lib/rake/rake_test_loader.rb" "test/unit/
user_test.rb"

Anyone else have this problem or is it normal? Thanks.

15 Answers

Ryan Davis

8/14/2007 5:36:00 AM

On Aug 13, 2007, at 22:04 , richpoirier@gmail.com wrote:

> Is rake supposed to be really slow?

no, not that it is supposed to be really fast... but it isn't what
gets in your way.

> ...
> C:/InstantRails/ruby/bin/ruby -Ilib;test "C:/InstantRails/ruby/lib/
> ruby/gems/1.8/gems/rake-0.7.2/lib/rake/rake_test_loader.rb" "test/
> unit/
> user_test.rb"

My guess is that your test is slow. To start, take rake out of the
equation:

ruby -rtest/unit test/unit/user_test.rb

After that, you can do something simple like:

def setup
@start_time = Time.now
end

def teardown
puts "** TIME: #{self.name} : #{Time.now - @start_time} seconds
end

And see what's going so slow a little clearer.

> Anyone else have this problem or is it normal? Thanks.

I run thousands of tests/assertions in just seconds many times a day:

ruby: Finished in 0.105172 seconds. 252 tests, 735 assertions, 0
failures, 0 errors
ruby: Finished in 1.275824 seconds. 750 tests, 2220 assertions, 0
failures, 0 errors
rails: Finished in 81.681246 seconds. 1924 tests, 12195 assertions, 0
failures, 0 errors

Jim Weirich

8/14/2007 10:41:00 PM

unknown wrote:
> Is rake supposed to be really slow?
>
> I'm using InstantRails and running rake in a project with one unit
> test and two functional tests and it takes 33 seconds total. Is that
> normal? It seems really slow to me.
>
> When I turn on trace I can see that it spends most of its time in
> "Execute environment" and loading the test files:
>
> C:/InstantRails/ruby/bin/ruby -Ilib;test "C:/InstantRails/ruby/lib/
> ruby/gems/1.8/gems/rake-0.7.2/lib/rake/rake_test_loader.rb" "test/unit/
> user_test.rb"
>
> Anyone else have this problem or is it normal? Thanks.

Rake in and of itself is quite fast. There are two types of things that
can slow you down.

(1) Executing the Rakefile. If you have lots of tasks or complex
dependencies, then whatever time it takes to resolve those can build up
and slow down every rake command.

(2) Slow tasks. Some tasks in rake are naturely slow. Slow takes make
anything that depends upon them slow as well.

I assume you are working in Rails? Add this to your top level Rakefile:

task :noop

Then time the "rake noop" command. This will be the time it takes rake
to run essentially a do nothing task and will give you a sense of the
general Rakefile overhead.

Then time "rake environment". This task loads the rails environment and
can take some time. The environment task is a dependency of most rails
related rake tasks in a Rails project.

In the project I'm currently working on, "rake noop" takes under a
second. "rake environment" takes nearly 3 seconds. ("rake noop" in a
lightweight non-rails project takes under 0.2 seconds).

It sounds like the environment task on your system is painfully slow.
Is it doing anything unusual?

-- Jim Weirich

--
Posted via http://www.ruby-....

richpoirier

8/15/2007 4:41:00 AM

On Aug 14, 6:40 pm, Jim Weirich <j...@weirichhouse.org> wrote:
> unknown wrote:
> > Israkesupposed to be reallyslow?
>
> > I'm using InstantRails and runningrakein a project with one unit
> > test and two functional tests and it takes 33 seconds total. Is that
> > normal? It seems reallyslowto me.
>
> > When I turn on trace I can see that it spends most of its time in
> > "Execute environment" and loading the test files:
>
> > C:/InstantRails/ruby/bin/ruby -Ilib;test "C:/InstantRails/ruby/lib/
> > ruby/gems/1.8/gems/rake-0.7.2/lib/rake/rake_test_loader.rb" "test/unit/
> > user_test.rb"
>
> > Anyone else have this problem or is it normal? Thanks.
>
> Rakein and of itself is quite fast. There are two types of things that
> canslowyou down.
>
> (1) Executing the Rakefile. If you have lots of tasks or complex
> dependencies, then whatever time it takes to resolve those can build up
> andslowdown everyrakecommand.
>
> (2)Slowtasks. Some tasks inrakeare naturelyslow. Slowtakes make
> anything that depends upon themslowas well.
>
> I assume you are working in Rails? Add this to your top level Rakefile:
>
> task :noop
>
> Then time the "rakenoop" command. This will be the time it takesrake
> to run essentially a do nothing task and will give you a sense of the
> general Rakefile overhead.
>
> Then time "rakeenvironment". This task loads the rails environment and
> can take some time. The environment task is a dependency of most rails
> relatedraketasks in a Rails project.
>
> In the project I'm currently working on, "rakenoop" takes under a
> second. "rakeenvironment" takes nearly 3 seconds. ("rakenoop" in a
> lightweight non-rails project takes under 0.2 seconds).
>
> It sounds like the environment task on your system is painfullyslow.
> Is it doing anything unusual?
>
> -- Jim Weirich
>
> --
> Posted viahttp://www.ruby-....

Thanks for the replies. Yes I'm working with rails. I think you are
both right. Running a single unit test without rake takes about 11
seconds (even though it says it finished in 0.118 seconds). The test
simply creates a new user in my User model. So maybe it's taking a
while to connect to my mysql db? I'll have to investigate that (not
sure how to though). rake environment takes about 13 seconds. Where is
the environment task defined? I'd like to take a look at it to see
what it's doing. Using rake to run that one unit test takes 24
seconds, so it's definitely a combination of the environment task
being painfully slow and the test itself being slow too.

Thanks again for the help Jim and Ryan.

richpoirier

8/16/2007 2:58:00 AM

On Aug 15, 12:41 am, richpoir...@gmail.com wrote:
> On Aug 14, 6:40 pm, Jim Weirich <j...@weirichhouse.org> wrote:
>
>
>
> > unknown wrote:
> > > Israkesupposed to be reallyslow?
>
> > > I'm using InstantRails and runningrakein a project with one unit
> > > test and two functional tests and it takes 33 seconds total. Is that
> > > normal? It seems reallyslowto me.
>
> > > When I turn on trace I can see that it spends most of its time in
> > > "Execute environment" and loading the test files:
>
> > > C:/InstantRails/ruby/bin/ruby -Ilib;test "C:/InstantRails/ruby/lib/
> > > ruby/gems/1.8/gems/rake-0.7.2/lib/rake/rake_test_loader.rb" "test/unit/
> > > user_test.rb"
>
> > > Anyone else have this problem or is it normal? Thanks.
>
> > Rakein and of itself is quite fast. There are two types of things that
> > canslowyou down.
>
> > (1) Executing the Rakefile. If you have lots of tasks or complex
> > dependencies, then whatever time it takes to resolve those can build up
> > andslowdown everyrakecommand.
>
> > (2)Slowtasks. Some tasks inrakeare naturelyslow. Slowtakes make
> > anything that depends upon themslowas well.
>
> > I assume you are working in Rails? Add this to your top level Rakefile:
>
> > task :noop
>
> > Then time the "rakenoop" command. This will be the time it takesrake
> > to run essentially a do nothing task and will give you a sense of the
> > general Rakefile overhead.
>
> > Then time "rakeenvironment". This task loads the rails environment and
> > can take some time. The environment task is a dependency of most rails
> > relatedraketasks in a Rails project.
>
> > In the project I'm currently working on, "rakenoop" takes under a
> > second. "rakeenvironment" takes nearly 3 seconds. ("rakenoop" in a
> > lightweight non-rails project takes under 0.2 seconds).
>
> > It sounds like the environment task on your system is painfullyslow.
> > Is it doing anything unusual?
>
> > -- Jim Weirich
>
> > --
> > Posted viahttp://www.ruby-....
>
> Thanks for the replies. Yes I'm working with rails. I think you are
> both right. Running a single unit test withoutraketakes about 11
> seconds (even though it says it finished in 0.118 seconds). The test
> simply creates a new user in my User model. So maybe it's taking a
> while to connect to my mysql db? I'll have to investigate that (not
> sure how to though).rakeenvironment takes about 13 seconds. Where is
> the environment task defined? I'd like to take a look at it to see
> what it's doing. Usingraketo run that one unit test takes 24
> seconds, so it's definitely a combination of the environment task
> being painfullyslowand the test itself beingslowtoo.
>
> Thanks again for the help Jim and Ryan.

Ok I've narrowed down the environment part of the problem. What's
taking long is the require_frameworks method in rails' Initializer.
The active_record, action_controller and action_web_services
frameworks take a while (a few seconds each) to load. So now the
question is why? I'm not sure where to look to dig deeper into this,
i.e., see what the require method is actually doing. Any ideas?

I still have to look into why the test itself is slow too.

hemant

8/16/2007 3:51:00 AM

On 8/16/07, Rich <richpoirier@gmail.com> wrote:
> On Aug 15, 12:41 am, richpoir...@gmail.com wrote:
> > On Aug 14, 6:40 pm, Jim Weirich <j...@weirichhouse.org> wrote:
> >
> >
> >
> > > unknown wrote:
> > > > Israkesupposed to be reallyslow?
> >
> > > > I'm using InstantRails and runningrakein a project with one unit
> > > > test and two functional tests and it takes 33 seconds total. Is that
> > > > normal? It seems reallyslowto me.
> >
> > > > When I turn on trace I can see that it spends most of its time in
> > > > "Execute environment" and loading the test files:
> >
> > > > C:/InstantRails/ruby/bin/ruby -Ilib;test "C:/InstantRails/ruby/lib/
> > > > ruby/gems/1.8/gems/rake-0.7.2/lib/rake/rake_test_loader.rb" "test/unit/
> > > > user_test.rb"
> >
> > > > Anyone else have this problem or is it normal? Thanks.
> >
> > > Rakein and of itself is quite fast. There are two types of things that
> > > canslowyou down.
> >
> > > (1) Executing the Rakefile. If you have lots of tasks or complex
> > > dependencies, then whatever time it takes to resolve those can build up
> > > andslowdown everyrakecommand.
> >
> > > (2)Slowtasks. Some tasks inrakeare naturelyslow. Slowtakes make
> > > anything that depends upon themslowas well.
> >
> > > I assume you are working in Rails? Add this to your top level Rakefile:
> >
> > > task :noop
> >
> > > Then time the "rakenoop" command. This will be the time it takesrake
> > > to run essentially a do nothing task and will give you a sense of the
> > > general Rakefile overhead.
> >
> > > Then time "rakeenvironment". This task loads the rails environment and
> > > can take some time. The environment task is a dependency of most rails
> > > relatedraketasks in a Rails project.
> >
> > > In the project I'm currently working on, "rakenoop" takes under a
> > > second. "rakeenvironment" takes nearly 3 seconds. ("rakenoop" in a
> > > lightweight non-rails project takes under 0.2 seconds).
> >
> > > It sounds like the environment task on your system is painfullyslow.
> > > Is it doing anything unusual?
> >
> > > -- Jim Weirich
> >
> > > --
> > > Posted viahttp://www.ruby-....
> >
> > Thanks for the replies. Yes I'm working with rails. I think you are
> > both right. Running a single unit test withoutraketakes about 11
> > seconds (even though it says it finished in 0.118 seconds). The test
> > simply creates a new user in my User model. So maybe it's taking a
> > while to connect to my mysql db? I'll have to investigate that (not
> > sure how to though).rakeenvironment takes about 13 seconds. Where is
> > the environment task defined? I'd like to take a look at it to see
> > what it's doing. Usingraketo run that one unit test takes 24
> > seconds, so it's definitely a combination of the environment task
> > being painfullyslowand the test itself beingslowtoo.
> >
> > Thanks again for the help Jim and Ryan.
>
> Ok I've narrowed down the environment part of the problem. What's
> taking long is the require_frameworks method in rails' Initializer.
> The active_record, action_controller and action_web_services
> frameworks take a while (a few seconds each) to load. So now the
> question is why? I'm not sure where to look to dig deeper into this,
> i.e., see what the require method is actually doing. Any ideas?
>
> I still have to look into why the test itself is slow too.
>
>
>

Its normally a good idea to take out actual db connections when
running tests. Did you look into mocha or something?

Damjan Rems

8/16/2007 6:24:00 AM

Hemant Kumar wrote:
> On 8/16/07, Rich <richpoirier@gmail.com> wrote:
>> > > > normal? It seems reallyslowto me.
>> > > Rakein and of itself is quite fast. There are two types of things that
>> >
>> > > In the project I'm currently working on, "rakenoop" takes under a
>> >
>> >
>>
>>
>>
>
> Its normally a good idea to take out actual db connections when
> running tests. Did you look into mocha or something?

Could this be your problem too.
http://forums.mysql.com/read.php?35,6...

I had same problem on a Linux box.

by
TheR
--
Posted via http://www.ruby-....

richpoirier

8/16/2007 12:58:00 PM

On Aug 15, 11:50 pm, hemant <gethem...@gmail.com> wrote:
> On 8/16/07, Rich <richpoir...@gmail.com> wrote:
>
>
>
> > On Aug 15, 12:41 am, richpoir...@gmail.com wrote:
> > > On Aug 14, 6:40 pm, Jim Weirich <j...@weirichhouse.org> wrote:
>
> > > > unknown wrote:
> > > > > Israkesupposed to be reallyslow?
>
> > > > > I'm using InstantRails and runningrakein a project with one unit
> > > > > test and two functional tests and it takes 33 seconds total. Is that
> > > > > normal? It seems reallyslowto me.
>
> > > > > When I turn on trace I can see that it spends most of its time in
> > > > > "Execute environment" and loading the test files:
>
> > > > > C:/InstantRails/ruby/bin/ruby -Ilib;test "C:/InstantRails/ruby/lib/
> > > > > ruby/gems/1.8/gems/rake-0.7.2/lib/rake/rake_test_loader.rb" "test/unit/
> > > > > user_test.rb"
>
> > > > > Anyone else have this problem or is it normal? Thanks.
>
> > > > Rakein and of itself is quite fast. There are two types of things that
> > > > canslowyou down.
>
> > > > (1) Executing the Rakefile. If you have lots of tasks or complex
> > > > dependencies, then whatever time it takes to resolve those can build up
> > > > andslowdown everyrakecommand.
>
> > > > (2)Slowtasks. Some tasks inrakeare naturelyslow. Slowtakes make
> > > > anything that depends upon themslowas well.
>
> > > > I assume you are working in Rails? Add this to your top level Rakefile:
>
> > > > task :noop
>
> > > > Then time the "rakenoop" command. This will be the time it takesrake
> > > > to run essentially a do nothing task and will give you a sense of the
> > > > general Rakefile overhead.
>
> > > > Then time "rakeenvironment". This task loads the rails environment and
> > > > can take some time. The environment task is a dependency of most rails
> > > > relatedraketasks in a Rails project.
>
> > > > In the project I'm currently working on, "rakenoop" takes under a
> > > > second. "rakeenvironment" takes nearly 3 seconds. ("rakenoop" in a
> > > > lightweight non-rails project takes under 0.2 seconds).
>
> > > > It sounds like the environment task on your system is painfullyslow.
> > > > Is it doing anything unusual?
>
> > > > -- Jim Weirich
>
> > > > --
> > > > Posted viahttp://www.ruby-....
>
> > > Thanks for the replies. Yes I'm working with rails. I think you are
> > > both right. Running a single unit test withoutraketakes about 11
> > > seconds (even though it says it finished in 0.118 seconds). The test
> > > simply creates a new user in my User model. So maybe it's taking a
> > > while to connect to my mysql db? I'll have to investigate that (not
> > > sure how to though).rakeenvironment takes about 13 seconds. Where is
> > > the environment task defined? I'd like to take a look at it to see
> > > what it's doing. Usingraketo run that one unit test takes 24
> > > seconds, so it's definitely a combination of the environment task
> > > being painfullyslowand the test itself beingslowtoo.
>
> > > Thanks again for the help Jim and Ryan.
>
> > Ok I've narrowed down the environment part of the problem. What's
> > taking long is the require_frameworks method in rails' Initializer.
> > The active_record, action_controller and action_web_services
> > frameworks take a while (a few seconds each) to load. So now the
> > question is why? I'm not sure where to look to dig deeper into this,
> > i.e., see what the require method is actually doing. Any ideas?
>
> > I still have to look into why the test itself isslowtoo.
>
> Its normally a good idea to take out actual db connections when
> running tests. Did you look into mocha or something?

I'm running rails tests though. Don't most rails tests require a db
connection?

richpoirier

8/16/2007 12:59:00 PM

On Aug 16, 2:24 am, Damjan Rems <d_r...@yahoo.com> wrote:
> Hemant Kumar wrote:
> > On 8/16/07, Rich <richpoir...@gmail.com> wrote:
> >> > > > normal? It seems reallyslowto me.
> >> > > Rakein and of itself is quite fast. There are two types of things that
>
> >> > > In the project I'm currently working on, "rakenoop" takes under a
>
> > Its normally a good idea to take out actual db connections when
> > running tests. Did you look into mocha or something?
>
> Could this be your problem too.http://forums.mysql.com/read.php?35,6...
>
> I had same problem on a Linux box.
>
> by
> TheR
> --
> Posted viahttp://www.ruby-....

I'm just connecting to my local db, that post has to do with
connecting to a remote one. Thanks though.

Jay Levitt

8/16/2007 1:14:00 PM

On Thu, 16 Aug 2007 02:57:30 -0000, Rich wrote:

> Ok I've narrowed down the environment part of the problem. What's
> taking long is the require_frameworks method in rails' Initializer.
> The active_record, action_controller and action_web_services
> frameworks take a while (a few seconds each) to load. So now the
> question is why? I'm not sure where to look to dig deeper into this,
> i.e., see what the require method is actually doing. Any ideas?

Probably useless data point:

On my 3GHz dual-core Athlon, under either Cygwin or virtualized Ubuntu,
with RAID-0 10,000-RPM drives, initializing the rails environment takes
7-10 seconds.

On my 2.16GHz dual-core MacBook Pro, with a single 5200-RPM drive,
initializing the rails environment takes a second or two.

You can imagine my workaround.

Jay Levitt

Alexey Verkhovsky

8/16/2007 3:04:00 PM

On 8/16/07, Jay Levitt <jay+news@jay.fm> wrote:
> On my 3GHz dual-core Athlon, under either Cygwin or virtualized Ubuntu,
> with RAID-0 10,000-RPM drives, initializing the rails environment takes
> 7-10 seconds.
>
> On my 2.16GHz dual-core MacBook Pro, with a single 5200-RPM drive,
> initializing the rails environment takes a second or two.
>
> You can imagine my workaround.

Err... is it to run One-Click Installer Ruby instead of Cygwin?

--
Alexey Verkhovsky
CruiseControl.rb [http://cruisecontrolrb.though...]
RubyWorks [http://rubyworks.though...]

comp.lang.ruby

Why is rake really slow?

richpoirier

Ryan Davis

Jim Weirich

richpoirier

richpoirier

hemant

Damjan Rems

richpoirier

richpoirier

Jay Levitt

Alexey Verkhovsky

x Login to ForumsZone