[lnkForumImage]
TotalShareware - Download Free Software
Confronta i prezzi di migliaia di prodotti.
Asp Forum
 Home | Login | Register | Search 


 

Forums >

comp.lang.ruby

Help interpreting benchmark results

Juan Alvarez

2/24/2009 10:35:00 PM

I ran this simple benchmark:

Benchmark.bm do |x|
x.report('Curl::Easy.http_get') do
Curl::Easy.http_get(text)
end

x.report('RestClient.get') do
RestClient.get(text)
end
end

RestClient uses the standard Net:HTTP library so the benchmark is really
trying to compare Net::HTTP vs curb (libcurl bindings).

These are the results I get:

user system total real
Curl::Easy.http_get 0.070000 0.130000 0.200000 ( 26.319104)
RestClient.get 0.970000 0.580000 1.550000 ( 23.557923)

I get these results consistently where curb's user, system and total
times are dramatically lower than Net::HTTP's. However, Net:HTTP is
reported to run faster by the real column. Shouldn't the reported real
time be consistent with the other columns?

I'm running this benchmark on a 2.4Ghz Core 2 Duo with 2GB RAM MacBook
Pro, if that helps.

Thanks,
Juan
--
Posted via http://www.ruby-....
3 Answers

Sandor Szücs

2/25/2009 11:35:00 AM

0


On 24.02.2009, at 23:35, Juan Alvarez wrote:

> These are the results I get:
>
> user system total =20
> real
> Curl::Easy.http_get 0.070000 0.130000 0.200000 ( 26.319104)
> RestClient.get 0.970000 0.580000 1.550000 ( 23.557923)
>
> I get these results consistently where curb's user, system and total
> times are dramatically lower than Net::HTTP's. However, Net:HTTP is
> reported to run faster by the real column. Shouldn't the reported real
> time be consistent with the other columns?

No. "real" measures the time that is passed from start to finish.

Pseudocode
real_start =3D Time.now
yield # run test
real_end =3D Time.now
real_start.to_f - real_end.to_f


If the Kernel of your OS schedules another processes for, say 20 =20
seconds,
then the difference between total and real is >=3D 20 seconds.
I think the time called "real" should not be trusted in an evaluation.

regards, Sandor Sz=FCcs
--

Robert Klemme

2/25/2009 12:21:00 PM

0

2009/2/25 Sandor Sz=FCcs <sandor.szuecs@fu-berlin.de>:
>
> On 24.02.2009, at 23:35, Juan Alvarez wrote:
>
>> These are the results I get:
>>
>> =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 user =A0=
=A0 system =A0 =A0 =A0total =A0 =A0 =A0 =A0real
>> Curl::Easy.http_get =A0 0.070000 =A0 0.130000 =A0 0.200000 ( 26.319104)
>> RestClient.get =A0 =A0 =A0 =A0 =A00.970000 =A0 0.580000 =A0 1.550000 ( 2=
3.557923)
>>
>> I get these results consistently where curb's user, system and total
>> times are dramatically lower than Net::HTTP's. However, Net:HTTP is
>> reported to run faster by the real column. Shouldn't the reported real
>> time be consistent with the other columns?
>
> No. "real" measures the time that is passed from start to finish.
>
> Pseudocode
> =A0real_start =3D Time.now
> =A0yield # run test
> =A0real_end =3D Time.now
> =A0real_start.to_f - real_end.to_f
>
>
> If the Kernel of your OS schedules another processes for, say 20 seconds,
> then the difference between total and real is >=3D 20 seconds.
> I think the time called "real" should not be trusted in an evaluation.

Not so fast. If another process has been running that may well be a
process spawned by our process.

13:19:09 Temp$ ruby19 bb.rb
user system total real
other process 0.000000 0.000000 0.139000 ( 10.375000)
in process 4.594000 0.000000 4.594000 ( 4.765000)
13:19:30 Temp$ cat bb.rb
require 'benchmark'

Benchmark.bm 20 do |b|
b.report 'other process' do
system *%w{sleep 10}
end

b.report 'in process' do
10_000_000.times {|i| i + 1}
end
end
13:19:35 Temp$

I'd check the curl lib - I would assume that it uses the command line
utility of the same name - which then is another process.

Kind regards

robert

--=20
remember.guy do |as, often| as.you_can - without end

Sandor Szücs

2/25/2009 8:56:00 PM

0


On 25.02.2009, at 13:20, Robert Klemme wrote:

> 2009/2/25 Sandor Sz=FCcs <sandor.szuecs@fu-berlin.de>:
>>
>> No. "real" measures the time that is passed from start to finish.
>>
>> Pseudocode
>> real_start =3D Time.now
>> yield # run test
>> real_end =3D Time.now
>> real_start.to_f - real_end.to_f
>>
>>
>> If the Kernel of your OS schedules another processes for, say 20 =20
>> seconds,
>> then the difference between total and real is >=3D 20 seconds.
>> I think the time called "real" should not be trusted in an =20
>> evaluation.
>
> Not so fast. If another process has been running that may well be a
> process spawned by our process.

Full ack that's right, if a process forks, like system() do you can =20
measure
that one too.

Benchmark.bm(20) do |b|
b.report 'fork' do
pid =3D Process.fork { sleep 3 }
Process.waitpid pid
end
end
user system total real
fork 0.000000 0.010000 0.020000 ( 3.008273)


regards, Sandor Sz=FCcs
--