[lnkForumImage]
TotalShareware - Download Free Software
Confronta i prezzi di migliaia di prodotti.
Asp Forum
 Home | Login | Register | Search 


 

Forums >

comp.lang.ruby

A better benchmarking syntax (was: Automatic Benchmark Iterations

Daniel Schierbeck

3/7/2007 1:27:00 PM

Reading Phrogz' post about automatic benchmark iterations, and then
seeing Mauricios' lovely Adaptative Benchmark[1], I came to think we
might need a friendlier syntax for benchmarks all together. Minutes
later, I discovered that someone had almost the same idea as me[2].
Anyway, I'd like to just throw it out here, and hear what you people
think.

The idea is to make benchmarks syntactically similar to the current
Test::Unit test cases, e.g.

class SortBenchmark < Benchmark
def setup
@array = [1, 6, 2, 9, 4, 6, 2]
end

def teardown
@array = nil
end

def report_quicksort
@array.quicksort
end

def report_mergesort
@array.mergesort
end
end

Automatic iteration could be added, either as a class method call or by
creating a different base class (IterativeBenchmark?)

So, what do y'all think? If I'm not the only one liking this, I might
whip something up when I get some spare time...


Cheers,
Daniel



1. http://eigenclass.org/hiki.rb?cmd=view&p=...
+benchmark&key=TDD
2. http://djberg96.livejournal.com/...
21 Answers

Daniel Berger

3/7/2007 1:45:00 PM

0

On Mar 7, 6:26 am, Daniel Schierbeck <daniel.schierb...@gmail.com>
wrote:
> Reading Phrogz' post about automatic benchmark iterations, and then
> seeing Mauricios' lovely Adaptative Benchmark[1], I came to think we
> might need a friendlier syntax for benchmarks all together. Minutes
> later, I discovered that someone had almost the same idea as me[2].
> Anyway, I'd like to just throw it out here, and hear what you people
> think.
>
> The idea is to make benchmarks syntactically similar to the current
> Test::Unit test cases, e.g.
>
> class SortBenchmark < Benchmark
> def setup
> @array = [1, 6, 2, 9, 4, 6, 2]
> end
>
> def teardown
> @array = nil
> end
>
> def report_quicksort
> @array.quicksort
> end
>
> def report_mergesort
> @array.mergesort
> end
> end
>
> Automatic iteration could be added, either as a class method call or by
> creating a different base class (IterativeBenchmark?)
>
> So, what do y'all think? If I'm not the only one liking this, I might
> whip something up when I get some spare time...

+1 - please do. :)

Dan

Trans

3/7/2007 2:19:00 PM

0



On Mar 7, 8:26 am, Daniel Schierbeck <daniel.schierb...@gmail.com>
wrote:
> Reading Phrogz' post about automatic benchmark iterations, and then
> seeing Mauricios' lovely Adaptative Benchmark[1], I came to think we
> might need a friendlier syntax for benchmarks all together. Minutes
> later, I discovered that someone had almost the same idea as me[2].
> Anyway, I'd like to just throw it out here, and hear what you people
> think.
>
> The idea is to make benchmarks syntactically similar to the current
> Test::Unit test cases, e.g.
>
> class SortBenchmark < Benchmark
> def setup
> @array = [1, 6, 2, 9, 4, 6, 2]
> end
>
> def teardown
> @array = nil
> end
>
> def report_quicksort
> @array.quicksort
> end
>
> def report_mergesort
> @array.mergesort
> end
> end
>
> Automatic iteration could be added, either as a class method call or by
> creating a different base class (IterativeBenchmark?)
>
> So, what do y'all think? If I'm not the only one liking this, I might
> whip something up when I get some spare time...

Cool! Go BDD with it:

benchmark "compare sorting methods" do

compare "quick sort"
@array.quicksort
end

compare "merge sort" do
@array.mergesort
end

end

Or something like that.

T.

pat eyler

3/7/2007 2:34:00 PM

0

On 3/7/07, Trans <transfire@gmail.com> wrote:
>
>
> Cool! Go BDD with it:
>
> benchmark "compare sorting methods" do
>
> compare "quick sort"
> @array.quicksort
> end
>
> compare "merge sort" do
> @array.mergesort
> end
>
> end
>
> Or something like that.

I'd like to see a way to specify with or without rehearsals, and with or without
statistics (see
http://on-ruby.bl.../2006/12/benchmarking-lies-and-stati...
for more info).

Maybe:

benchmark "compare sorting methods" do
with
tsort
rehearsal
end

compare "quick sort"
@array.quicksort
end

compare "merge sort" do
@array.mergesort
end

end

>
> T.
>

--
thanks,
-pate
-------------------------
http://on-ruby.bl...
http://on-ruby....
http://mtnwe...

Daniel Schierbeck

3/7/2007 4:54:00 PM

0

On Wed, 2007-03-07 at 23:18 +0900, Trans wrote:
>
> On Mar 7, 8:26 am, Daniel Schierbeck <daniel.schierb...@gmail.com>
> wrote:
> > Reading Phrogz' post about automatic benchmark iterations, and then
> > seeing Mauricios' lovely Adaptative Benchmark[1], I came to think we
> > might need a friendlier syntax for benchmarks all together. Minutes
> > later, I discovered that someone had almost the same idea as me[2].
> > Anyway, I'd like to just throw it out here, and hear what you people
> > think.
> >
> > The idea is to make benchmarks syntactically similar to the current
> > Test::Unit test cases, e.g.
> >
> > class SortBenchmark < Benchmark
> > def setup
> > @array = [1, 6, 2, 9, 4, 6, 2]
> > end
> >
> > def teardown
> > @array = nil
> > end
> >
> > def report_quicksort
> > @array.quicksort
> > end
> >
> > def report_mergesort
> > @array.mergesort
> > end
> > end
> >
> > Automatic iteration could be added, either as a class method call or by
> > creating a different base class (IterativeBenchmark?)
> >
> > So, what do y'all think? If I'm not the only one liking this, I might
> > whip something up when I get some spare time...
>
> Cool! Go BDD with it:
>
> benchmark "compare sorting methods" do
>
> compare "quick sort"
> @array.quicksort
> end
>
> compare "merge sort" do
> @array.mergesort
> end
>
> end
>
> Or something like that.

I must admit to being a great fan of RSpec, although the pace of API
changes has thrown me off for now, but I'm not sure this path is the
best. I did actually think about it at first. The strength of Test::Unit
is its simple syntax: just remember #setup, #teardown, and #test_*. I'd
like to add the same simplicity to any new benchmarking system. I'll
give it some more thought :)

Any idea for a name? "benchmark" is unfortunately already taken...


Cheers,
Daniel

Daniel Berger

3/7/2007 4:57:00 PM

0

On Mar 7, 9:54 am, Daniel Schierbeck <daniel.schierb...@gmail.com>
wrote:
> On Wed, 2007-03-07 at 23:18 +0900, Trans wrote:
>
> > On Mar 7, 8:26 am, Daniel Schierbeck <daniel.schierb...@gmail.com>
> > wrote:
> > > Reading Phrogz' post about automatic benchmark iterations, and then
> > > seeing Mauricios' lovely Adaptative Benchmark[1], I came to think we
> > > might need a friendlier syntax for benchmarks all together. Minutes
> > > later, I discovered that someone had almost the same idea as me[2].
> > > Anyway, I'd like to just throw it out here, and hear what you people
> > > think.
>
> > > The idea is to make benchmarks syntactically similar to the current
> > > Test::Unit test cases, e.g.
>
> > > class SortBenchmark < Benchmark
> > > def setup
> > > @array = [1, 6, 2, 9, 4, 6, 2]
> > > end
>
> > > def teardown
> > > @array = nil
> > > end
>
> > > def report_quicksort
> > > @array.quicksort
> > > end
>
> > > def report_mergesort
> > > @array.mergesort
> > > end
> > > end
>
> > > Automatic iteration could be added, either as a class method call or by
> > > creating a different base class (IterativeBenchmark?)
>
> > > So, what do y'all think? If I'm not the only one liking this, I might
> > > whip something up when I get some spare time...
>
> > Cool! Go BDD with it:
>
> > benchmark "compare sorting methods" do
>
> > compare "quick sort"
> > @array.quicksort
> > end
>
> > compare "merge sort" do
> > @array.mergesort
> > end
>
> > end
>
> > Or something like that.
>
> I must admit to being a great fan of RSpec, although the pace of API
> changes has thrown me off for now, but I'm not sure this path is the
> best. I did actually think about it at first. The strength of Test::Unit
> is its simple syntax: just remember #setup, #teardown, and #test_*. I'd
> like to add the same simplicity to any new benchmarking system.

I agree. Keep it simple.

> Any idea for a name? "benchmark" is unfortunately already taken...

Assuming you keep the test-unit like syntax, I vote for bench-unit.

Regards,

Dan

pat eyler

3/7/2007 5:00:00 PM

0

On 3/7/07, Daniel Schierbeck <daniel.schierbeck@gmail.com> wrote:
> I must admit to being a great fan of RSpec, although the pace of API
> changes has thrown me off for now, but I'm not sure this path is the
> best. I did actually think about it at first. The strength of Test::Unit
> is its simple syntax: just remember #setup, #teardown, and #test_*. I'd
> like to add the same simplicity to any new benchmarking system. I'll
> give it some more thought :)

Whichever way you go, please do consider configurable (and
extensible) additons like rehearsals and statistics.


>
> Any idea for a name? "benchmark" is unfortunately already taken...
>

what about BenchBench (in a play on "Benchmarkers' Tool
Bench")


>
> Cheers,
> Daniel
>
>
>


--
thanks,
-pate
-------------------------
http://on-ruby.bl...
http://on-ruby....
http://mtnwe...

Daniel Schierbeck

3/7/2007 5:03:00 PM

0

On Wed, 2007-03-07 at 23:33 +0900, pat eyler wrote:
> On 3/7/07, Trans <transfire@gmail.com> wrote:
> >
> >
> > Cool! Go BDD with it:
> >
> > benchmark "compare sorting methods" do
> >
> > compare "quick sort"
> > @array.quicksort
> > end
> >
> > compare "merge sort" do
> > @array.mergesort
> > end
> >
> > end
> >
> > Or something like that.
>
> I'd like to see a way to specify with or without rehearsals, and with or without
> statistics (see
> http://on-ruby.blogspot.com/2006/12/benchmarking-lies-and-stati...
> for more info).

I think statistics may be beyond the immediate scope of this -- I'd need
the help of someone better at it than me, that's for sure!

One thing on my priority list is a nice Rake task that makes it easy to
run the benchmarks and set different options.


Daniel

Daniel Schierbeck

3/7/2007 5:11:00 PM

0

On Thu, 2007-03-08 at 01:59 +0900, pat eyler wrote:
> On 3/7/07, Daniel Schierbeck <daniel.schierbeck@gmail.com> wrote:
> > I must admit to being a great fan of RSpec, although the pace of API
> > changes has thrown me off for now, but I'm not sure this path is the
> > best. I did actually think about it at first. The strength of Test::Unit
> > is its simple syntax: just remember #setup, #teardown, and #test_*. I'd
> > like to add the same simplicity to any new benchmarking system. I'll
> > give it some more thought :)
>
> Whichever way you go, please do consider configurable (and
> extensible) additons like rehearsals and statistics.
>
>
> >
> > Any idea for a name? "benchmark" is unfortunately already taken...
> >
>
> what about BenchBench (in a play on "Benchmarkers' Tool
> Bench")

I'm not sure... the other guy[1] suggested BenchUnit, which I think
reads well, but I not sure how descriptive it is.


Cheers,
Daniel


. http://djberg96.livejournal.com/...

David Chelimsky

3/7/2007 5:17:00 PM

0

On 3/7/07, Daniel Schierbeck <daniel.schierbeck@gmail.com> wrote:
> On Wed, 2007-03-07 at 23:18 +0900, Trans wrote:
> >
> > On Mar 7, 8:26 am, Daniel Schierbeck <daniel.schierb...@gmail.com>
> > wrote:
> > > Reading Phrogz' post about automatic benchmark iterations, and then
> > > seeing Mauricios' lovely Adaptative Benchmark[1], I came to think we
> > > might need a friendlier syntax for benchmarks all together. Minutes
> > > later, I discovered that someone had almost the same idea as me[2].
> > > Anyway, I'd like to just throw it out here, and hear what you people
> > > think.
> > >
> > > The idea is to make benchmarks syntactically similar to the current
> > > Test::Unit test cases, e.g.
> > >
> > > class SortBenchmark < Benchmark
> > > def setup
> > > @array = [1, 6, 2, 9, 4, 6, 2]
> > > end
> > >
> > > def teardown
> > > @array = nil
> > > end
> > >
> > > def report_quicksort
> > > @array.quicksort
> > > end
> > >
> > > def report_mergesort
> > > @array.mergesort
> > > end
> > > end
> > >
> > > Automatic iteration could be added, either as a class method call or by
> > > creating a different base class (IterativeBenchmark?)
> > >
> > > So, what do y'all think? If I'm not the only one liking this, I might
> > > whip something up when I get some spare time...
> >
> > Cool! Go BDD with it:
> >
> > benchmark "compare sorting methods" do
> >
> > compare "quick sort"
> > @array.quicksort
> > end
> >
> > compare "merge sort" do
> > @array.mergesort
> > end
> >
> > end
> >
> > Or something like that.
>
> I must admit to being a great fan of RSpec, although the pace of API
> changes has thrown me off for now,

Thanks for being a fan. FWIW, the recent API changes are the last
significant ones. What's new in 0.8 will be the basis of 1.0.

Cheers,
David

> but I'm not sure this path is the
> best. I did actually think about it at first. The strength of Test::Unit
> is its simple syntax: just remember #setup, #teardown, and #test_*. I'd
> like to add the same simplicity to any new benchmarking system. I'll
> give it some more thought :)
>
> Any idea for a name? "benchmark" is unfortunately already taken...
>
>
> Cheers,
> Daniel
>
>
>

Tim Pease

3/7/2007 5:20:00 PM

0

On 3/7/07, Daniel Schierbeck <daniel.schierbeck@gmail.com> wrote:
>
> I must admit to being a great fan of RSpec, although the pace of API
> changes has thrown me off for now, but I'm not sure this path is the
> best. I did actually think about it at first. The strength of Test::Unit
> is its simple syntax: just remember #setup, #teardown, and #test_*. I'd
> like to add the same simplicity to any new benchmarking system. I'll
> give it some more thought :)
>
> Any idea for a name? "benchmark" is unfortunately already taken...
>
>

Algorithm for generating product names

1) locate random object on desk
2) pick random character from alphabet
3) replace first character of object (from step 1) with the chosen
character (from step 2)

Example

1) apricot
2) 'h'
3) hpricot

From my own desk

1) light bulb
2) 'f'
3) fight_bulb

See, piece of cake!

Blessings,
TwP