[lnkForumImage]
TotalShareware - Download Free Software
Confronta i prezzi di migliaia di prodotti.
Asp Forum
 Home | Login | Register | Search 


 

Forums >

comp.lang.ruby

Slow RAILS testing

kevin cline

9/13/2007 8:15:00 PM

My RAILS unit tests take approximately one minute to run. This may
not sound like much, but it's a real drag on incremental development.
I imagine most of the slowness in spent in applying the database
schema to the test database. Is there a memory-resident database that
would speed up my tests?
9 Answers

George Malamidis

9/13/2007 8:55:00 PM

0

Hello,

Do all your tests need to access the database? I find this approach
( http://nutrun.com/weblog/rails-fast-t... ) useful in terms
of instant feedback and application code responsibility boundaries.

Thanks,
George

On 13 Sep 2007, at 21:15, kevin cline wrote:

> My RAILS unit tests take approximately one minute to run. This may
> not sound like much, but it's a real drag on incremental development.
> I imagine most of the slowness in spent in applying the database
> schema to the test database. Is there a memory-resident database that
> would speed up my tests?
>
>

Alex Young

9/13/2007 9:22:00 PM

0

kevin cline wrote:
> My RAILS unit tests take approximately one minute to run. This may
> not sound like much, but it's a real drag on incremental development.
> I imagine most of the slowness in spent in applying the database
> schema to the test database. Is there a memory-resident database that
> would speed up my tests?
I think you can run SQLite in-memory. The Rails people would know more,
though:

http://groups.google.com/group/rubyon...

I tend to factor tests so that they don't use Rails' built-in fixtures,
though - you really don't need to test the database interface, so it's
pure overhead.

--
Alex

Travis D Warlick Jr

9/13/2007 11:38:00 PM

0

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

kevin cline wrote:
> My RAILS unit tests take approximately one minute to run. This may
> not sound like much, but it's a real drag on incremental development.
> I imagine most of the slowness in spent in applying the database
> schema to the test database. Is there a memory-resident database that
> would speed up my tests?

You can use Heap tables with MySQL if you wish. This SQL statement will convert
a table into a Heap (memory-resident) table:

ALTER TABLE tablename TYPE=HEAP;
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.4.7 (MingW32)
Comment: Using GnuPG with Mozilla - http://enigmail....

iD8DBQFG6cnYWvapaOIz2YYRAuuvAJ4wwlypxThIaHCkVy7t4V1iDdgEHQCfZ5w+
wYBYFf9cGzqs56/O04qhG9I=
=JLqS
-----END PGP SIGNATURE-----

Gaspard Bucher

9/14/2007 8:24:00 AM

0

I could more then double the speed of my tests by avoiding fixtures
reload on each test and using transactions. Have a look at
http://dev.zenadmin.org/browser/trunk/test/test... and
http://dev.zenadmin.org/browser/trunk/test/ze....

It sure helped me (and I will now try Travis' HEAP idea on top of that).

Gaspard

2007/9/14, Travis D Warlick Jr <warlickt@operissystems.com>:
> -----BEGIN PGP SIGNED MESSAGE-----
> Hash: SHA1
>
> kevin cline wrote:
> > My RAILS unit tests take approximately one minute to run. This may
> > not sound like much, but it's a real drag on incremental development.
> > I imagine most of the slowness in spent in applying the database
> > schema to the test database. Is there a memory-resident database that
> > would speed up my tests?
>
> You can use Heap tables with MySQL if you wish. This SQL statement will convert
> a table into a Heap (memory-resident) table:
>
> ALTER TABLE tablename TYPE=HEAP;
> -----BEGIN PGP SIGNATURE-----
> Version: GnuPG v1.4.7 (MingW32)
> Comment: Using GnuPG with Mozilla - http://enigmail....
>
> iD8DBQFG6cnYWvapaOIz2YYRAuuvAJ4wwlypxThIaHCkVy7t4V1iDdgEHQCfZ5w+
> wYBYFf9cGzqs56/O04qhG9I=
> =JLqS
> -----END PGP SIGNATURE-----
>
>

Gaspard Bucher

9/14/2007 8:28:00 AM

0

HEAP does not support TEXT columns... Too bad.

Gaspard
2007/9/14, Gaspard Bucher <gaspard@teti.ch>:
> I could more then double the speed of my tests by avoiding fixtures
> reload on each test and using transactions. Have a look at
> http://dev.zenadmin.org/browser/trunk/test/test... and
> http://dev.zenadmin.org/browser/trunk/test/ze....
>
> It sure helped me (and I will now try Travis' HEAP idea on top of that).
>
> Gaspard
>
> 2007/9/14, Travis D Warlick Jr <warlickt@operissystems.com>:
> > -----BEGIN PGP SIGNED MESSAGE-----
> > Hash: SHA1
> >
> > kevin cline wrote:
> > > My RAILS unit tests take approximately one minute to run. This may
> > > not sound like much, but it's a real drag on incremental development.
> > > I imagine most of the slowness in spent in applying the database
> > > schema to the test database. Is there a memory-resident database that
> > > would speed up my tests?
> >
> > You can use Heap tables with MySQL if you wish. This SQL statement will convert
> > a table into a Heap (memory-resident) table:
> >
> > ALTER TABLE tablename TYPE=HEAP;
> > -----BEGIN PGP SIGNATURE-----
> > Version: GnuPG v1.4.7 (MingW32)
> > Comment: Using GnuPG with Mozilla - http://enigmail....
> >
> > iD8DBQFG6cnYWvapaOIz2YYRAuuvAJ4wwlypxThIaHCkVy7t4V1iDdgEHQCfZ5w+
> > wYBYFf9cGzqs56/O04qhG9I=
> > =JLqS
> > -----END PGP SIGNATURE-----
> >
> >
>

Phlip

9/15/2007 10:44:00 AM

0

Travis D Warlick Jr wrote:

> ALTER TABLE tablename TYPE=HEAP;

Ain't that ENGINE=HEAP?


Gaspard Bucher wrote:

> HEAP does not support TEXT columns... Too bad.

Suppose one wrote a rake task db:heap_mode, to switch the test environment
DB to the faster system. Could that task programmatically access schema.rb,
find the TEXT columns, and downgrade them to long strings?

--
Phlip
http://www.oreilly.com/catalog/9780...
"Test Driven Ajax (on Rails)"
assert_xpath, assert_javascript, & assert_ajax

Travis D Warlick Jr

9/15/2007 12:30:00 PM

0

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

Phlip wrote:
> Travis D Warlick Jr wrote:
>
>> ALTER TABLE tablename TYPE=HEAP;
>
> Ain't that ENGINE=HEAP?

Both is valid.

> Gaspard Bucher wrote:
>
>> HEAP does not support TEXT columns... Too bad.
>
> Suppose one wrote a rake task db:heap_mode, to switch the test
> environment DB to the faster system. Could that task programmatically
> access schema.rb, find the TEXT columns, and downgrade them to long
> strings?

One possible rails issue: I believe the entire test database structure is erased
and recreated everytime, so it couldn't be a rake task, it would have to be
integrated into the test task.

A database issue: you would have to beware of your table size. Every HEAP table
with at least 1 VARCHAR is at least 50KB due to how the VARCHARs are stored in
memory, and the larger you make your VARCHARs the larger your tables get (eg. a
table with 1 field of VARCHAR(20000) gives a table size of ~170KB).

If anyone wants to do this, note that the largest record size using the HEAP
engine is 64KB, and the largest single VARCHAR is ~21000.
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.4.7 (MingW32)
Comment: Using GnuPG with Mozilla - http://enigmail....

iD8DBQFG69BFWvapaOIz2YYRAv5cAJ0Z29aPLUVD8q/coTpULjzkzMahuACfUxqy
Sbr0ZNzCUHCFbrGI4FBQJMk=
=6GQA
-----END PGP SIGNATURE-----

Phlip

9/15/2007 4:25:00 PM

0

Travis D Warlick Jr wrote:

> One possible rails issue: I believe the entire test database structure is
> erased
> and recreated everytime, so it couldn't be a rake task, it would have to
> be
> integrated into the test task.

Yup; I just got that.

Then I put the conversion into the top of the source file of the first test
suite in alphabetic order (please nobody comment on the sustainability!),
and got a huge spew of test faults, as if I had no fixtures. The converter
was just this:

require File.dirname(__FILE__) + '/../test_helper'

def setup_database
tables = %w( accessories chats fighter_images props rings rounds users )
tables.each do |table|
ActiveRecord::Base.connection.execute("
ALTER TABLE #{ table } ENGINE=HEAP
")
end
end
setup_database

class AccessoryTest < Test::Unit::TestCase
...

And note, per the memory consumption issues you mentioned, that you don't
need to put all the fixtures in the list - if it worked!

--
Phlip


Gaspard Bucher

9/15/2007 7:22:00 PM

0

I tried this, but HEAP does not support transactions either. From what
I see, it feels like it's going to be much slower to reload the
fixtures then to do a ROLLBACK...

2007/9/15, Phlip <phlip2005@gmail.com>:
> Travis D Warlick Jr wrote:
>
> > ALTER TABLE tablename TYPE=HEAP;
>
> Ain't that ENGINE=HEAP?
>
>
> Gaspard Bucher wrote:
>
> > HEAP does not support TEXT columns... Too bad.
>
> Suppose one wrote a rake task db:heap_mode, to switch the test environment
> DB to the faster system. Could that task programmatically access schema.rb,
> find the TEXT columns, and downgrade them to long strings?
>
> --
> Phlip
> http://www.oreilly.com/catalog/9780...
> "Test Driven Ajax (on Rails)"
> assert_xpath, assert_javascript, & assert_ajax
>
>
>