[lnkForumImage]
TotalShareware - Download Free Software
Confronta i prezzi di migliaia di prodotti.
Asp Forum
 Home | Login | Register | Search 


 

Forums >

comp.lang.c

A new classification method for RNGs: Significance Level

joe

7/11/2008 12:39:00 PM

My experiments show that the random number generator
in Microsoft's VC++6 compiler is a statistical RNG with a
significance level > 1.0%.
Statistical testing at SL >1.0% (for example 1.001%) passes the test,
but 1.0% does not pass...

Can anybody confirm this finding?

The RNG function of the various SW products can be
analyzed and classified better using its significance level
as shown above.
I think this IMO important finding deserves a deeper research... :-)

For the testing method see:
http://en.wikipedia.org/wiki/Bin...
http://en.wikipedia.org/wiki/Binomial_di...
31 Answers

David Kerber

7/11/2008 1:13:00 PM

0

In article <g57kb1$47r$1@aioe.org>, joe@iamnotathome.org.invalid says...
> My experiments show that the random number generator
> in Microsoft's VC++6 compiler is a statistical RNG with a
> significance level > 1.0%.
> Statistical testing at SL >1.0% (for example 1.001%) passes the test,
> but 1.0% does not pass...
>
> Can anybody confirm this finding?
>
> The RNG function of the various SW products can be
> analyzed and classified better using its significance level
> as shown above.
> I think this IMO important finding deserves a deeper research... :-)
>
> For the testing method see:
> http://en.wikipedia.org/wiki/Bin...
> http://en.wikipedia.org/wiki/Binomial_di...

What are you using for your sample size and null hypothesis?

--
/~\ The ASCII
\ / Ribbon Campaign
X Against HTML
/ \ Email!

Remove the ns_ from if replying by e-mail (but keep posts in the
newsgroups if possible).

joe

7/11/2008 2:42:00 PM

0

"David Kerber" wrote:
> joe@iamnotathome.org.invalid says:
> >
> > My experiments show that the random number generator
> > in Microsoft's VC++6 compiler is a statistical RNG with a
> > significance level > 1.0%.
> > Statistical testing at SL >1.0% (for example 1.001%) passes the test,
> > but 1.0% does not pass...
> >
> > Can anybody confirm this finding?
> >
> > The RNG function of the various SW products can be
> > analyzed and classified better using its significance level
> > as shown above.
> > I think this IMO important finding deserves a deeper research... :-)
> >
> > For the testing method see:
> > http://en.wikipedia.org/wiki/Bin...
> > http://en.wikipedia.org/wiki/Binomial_di...
>
> What are you using for your sample size and null hypothesis?

Here the details:
Sample size is 500 (ie. calling rand() 500 times),
rnd range is 37 (ie. 0 to 36; yes, a roulette simulation).
Doing the above mentioned StatTest after each rand() call.
The above said is called more than 30 times in a loop,
each time initializing the freq stats anew.
srand(time(0)) done only once at pgmstart.

joe

7/11/2008 2:49:00 PM

0

"joe" wrote:
> "David Kerber" wrote:
> > joe@iamnotathome.org.invalid says:
> > >
> > > My experiments show that the random number generator
> > > in Microsoft's VC++6 compiler is a statistical RNG with a
> > > significance level > 1.0%.
> > > Statistical testing at SL >1.0% (for example 1.001%) passes the test,
> > > but 1.0% does not pass...
> > >
> > > Can anybody confirm this finding?
> > >
> > > The RNG function of the various SW products can be
> > > analyzed and classified better using its significance level
> > > as shown above.
> > > I think this IMO important finding deserves a deeper research... :-)
> > >
> > > For the testing method see:
> > > http://en.wikipedia.org/wiki/Bin...
> > > http://en.wikipedia.org/wiki/Binomial_di...
> >
> > What are you using for your sample size and null hypothesis?

H0 = RNG passes the randomness test only at a significance level >1%
ie. try out 1% and 1.001% for example and you will
see it always fails at the <=1% level and always passes at >1% sig level.

> Here the details:
> Sample size is 500 (ie. calling rand() 500 times),
> rnd range is 37 (ie. 0 to 36; yes, a roulette simulation).
> Doing the above mentioned StatTest after each rand() call.
> The above said is called more than 30 times in a loop,
> each time initializing the freq stats anew.
> srand(time(0)) done only once at pgmstart.

David Kerber

7/11/2008 3:07:00 PM

0

In article <g57rsv$5ad$1@aioe.org>, joe@iamnotathome.org.invalid says...
> "joe" wrote:
> > "David Kerber" wrote:
> > > joe@iamnotathome.org.invalid says:
> > > >
> > > > My experiments show that the random number generator
> > > > in Microsoft's VC++6 compiler is a statistical RNG with a
> > > > significance level > 1.0%.
> > > > Statistical testing at SL >1.0% (for example 1.001%) passes the test,
> > > > but 1.0% does not pass...
> > > >
> > > > Can anybody confirm this finding?
> > > >
> > > > The RNG function of the various SW products can be
> > > > analyzed and classified better using its significance level
> > > > as shown above.
> > > > I think this IMO important finding deserves a deeper research... :-)
> > > >
> > > > For the testing method see:
> > > > http://en.wikipedia.org/wiki/Bin...
> > > > http://en.wikipedia.org/wiki/Binomial_di...
> > >
> > > What are you using for your sample size and null hypothesis?
>
> H0 = RNG passes the randomness test only at a significance level >1%
> ie. try out 1% and 1.001% for example and you will
> see it always fails at the <=1% level and always passes at >1% sig level.
>
> > Here the details:
> > Sample size is 500 (ie. calling rand() 500 times),
> > rnd range is 37 (ie. 0 to 36; yes, a roulette simulation).
> > Doing the above mentioned StatTest after each rand() call.
> > The above said is called more than 30 times in a loop,
> > each time initializing the freq stats anew.
> > srand(time(0)) done only once at pgmstart.

I think your sample size is too small for such a conclusion to be
statistically valid.

--
/~\ The ASCII
\ / Ribbon Campaign
X Against HTML
/ \ Email!

Remove the ns_ from if replying by e-mail (but keep posts in the
newsgroups if possible).

joe

7/11/2008 3:30:00 PM

0

"David Kerber" wrote:
> joe@iamnotathome.org.invalid says:
> > "joe" wrote:
> > > "David Kerber" wrote:
> > > > joe@iamnotathome.org.invalid says:
> > > > >
> > > > > My experiments show that the random number generator
> > > > > in Microsoft's VC++6 compiler is a statistical RNG with a
> > > > > significance level > 1.0%.
> > > > > Statistical testing at SL >1.0% (for example 1.001%) passes the test,
> > > > > but 1.0% does not pass...
> > > > >
> > > > > Can anybody confirm this finding?
> > > > >
> > > > > The RNG function of the various SW products can be
> > > > > analyzed and classified better using its significance level
> > > > > as shown above.
> > > > > I think this IMO important finding deserves a deeper research... :-)
> > > > >
> > > > > For the testing method see:
> > > > > http://en.wikipedia.org/wiki/Bin...
> > > > > http://en.wikipedia.org/wiki/Binomial_di...
> > > >
> > > > What are you using for your sample size and null hypothesis?
> >
> > H0 = RNG passes the randomness test only at a significance level >1%
> > ie. try out 1% and 1.001% for example and you will
> > see it always fails at the <=1% level and always passes at >1% sig level.
> >
> > > Here the details:
> > > Sample size is 500 (ie. calling rand() 500 times),
> > > rnd range is 37 (ie. 0 to 36; yes, a roulette simulation).
> > > Doing the above mentioned StatTest after each rand() call.
> > > The above said is called more than 30 times in a loop,
> > > each time initializing the freq stats anew.
> > > srand(time(0)) done only once at pgmstart.
>
> I think your sample size is too small for such a conclusion to be
> statistically valid.

The minimum neccessary is given in these relations
(cf. the above wiki pages and also stats books):

n*p >= 5 AND n*(1-p) >= 5

ie. for the above example:
n >= 5 / (1/37) >= 185
Above we have 500, ie. from 185 to 500 the test can be applied.

David Kerber

7/11/2008 4:04:00 PM

0

In article <g57ub5$gmq$1@aioe.org>, joe@iamnotathome.org.invalid says...
> "David Kerber" wrote:
> > joe@iamnotathome.org.invalid says:
> > > "joe" wrote:
> > > > "David Kerber" wrote:
> > > > > joe@iamnotathome.org.invalid says:
> > > > > >
> > > > > > My experiments show that the random number generator
> > > > > > in Microsoft's VC++6 compiler is a statistical RNG with a
> > > > > > significance level > 1.0%.
> > > > > > Statistical testing at SL >1.0% (for example 1.001%) passes the test,
> > > > > > but 1.0% does not pass...
> > > > > >
> > > > > > Can anybody confirm this finding?
> > > > > >
> > > > > > The RNG function of the various SW products can be
> > > > > > analyzed and classified better using its significance level
> > > > > > as shown above.
> > > > > > I think this IMO important finding deserves a deeper research... :-)
> > > > > >
> > > > > > For the testing method see:
> > > > > > http://en.wikipedia.org/wiki/Bin...
> > > > > > http://en.wikipedia.org/wiki/Binomial_di...
> > > > >
> > > > > What are you using for your sample size and null hypothesis?
> > >
> > > H0 = RNG passes the randomness test only at a significance level >1%
> > > ie. try out 1% and 1.001% for example and you will
> > > see it always fails at the <=1% level and always passes at >1% sig level.
> > >
> > > > Here the details:
> > > > Sample size is 500 (ie. calling rand() 500 times),
> > > > rnd range is 37 (ie. 0 to 36; yes, a roulette simulation).
> > > > Doing the above mentioned StatTest after each rand() call.
> > > > The above said is called more than 30 times in a loop,
> > > > each time initializing the freq stats anew.
> > > > srand(time(0)) done only once at pgmstart.
> >
> > I think your sample size is too small for such a conclusion to be
> > statistically valid.
>
> The minimum neccessary is given in these relations
> (cf. the above wiki pages and also stats books):
>
> n*p >= 5 AND n*(1-p) >= 5
>
> ie. for the above example:
> n >= 5 / (1/37) >= 185
> Above we have 500, ie. from 185 to 500 the test can be applied.

But you said you're initializing the frequency stats before each loop
iteration? To me, that would imply that your sample size is really only
36 for each of 30 or so runs. Or am I misunderstanding how you are
accumulating the results?


--
/~\ The ASCII
\ / Ribbon Campaign
X Against HTML
/ \ Email!

Remove the ns_ from if replying by e-mail (but keep posts in the
newsgroups if possible).

user923005

7/11/2008 6:54:00 PM

0

On Jul 11, 5:39 am, "joe" <j...@iamnotathome.org.invalid> wrote:
> My experiments show that the random number generator
> in Microsoft's VC++6 compiler is a statistical RNG with a
> significance level > 1.0%.
> Statistical testing at SL >1.0% (for example 1.001%) passes the test,
> but 1.0% does not pass...
>
> Can anybody confirm this finding?
>
> The RNG function of the various SW products can be
> analyzed and classified better using its significance level
> as shown above.
> I think this IMO important finding deserves a deeper research... :-)
>
> For the testing method see:http://en.wikipedia.org/wiki/Binomial_testhttp://en.wikipedia.org/wiki/Binomial_di...

I guess that what you really want is the statistics group.
news:sci.stat.math

Follow-up added.

joe

7/11/2008 10:25:00 PM

0

"David Kerber" wrote:
> joe@iamnotathome.org.invalid says:
> > "David Kerber" wrote:
> > > joe@iamnotathome.org.invalid says:
> > > > "joe" wrote:
> > > > > "David Kerber" wrote:
> > > > > > joe@iamnotathome.org.invalid says:
> > > > > > >
> > > > > > > My experiments show that the random number generator
> > > > > > > in Microsoft's VC++6 compiler is a statistical RNG with a
> > > > > > > significance level > 1.0%.
> > > > > > > Statistical testing at SL >1.0% (for example 1.001%) passes the test,
> > > > > > > but 1.0% does not pass...
> > > > > > >
> > > > > > > Can anybody confirm this finding?
> > > > > > >
> > > > > > > The RNG function of the various SW products can be
> > > > > > > analyzed and classified better using its significance level
> > > > > > > as shown above.
> > > > > > > I think this IMO important finding deserves a deeper research... :-)
> > > > > > >
> > > > > > > For the testing method see:
> > > > > > > http://en.wikipedia.org/wiki/Bin...
> > > > > > > http://en.wikipedia.org/wiki/Binomial_di...
> > > > > >
> > > > > > What are you using for your sample size and null hypothesis?
> > > >
> > > > H0 = RNG passes the randomness test only at a significance level >1%
> > > > ie. try out 1% and 1.001% for example and you will
> > > > see it always fails at the <=1% level and always passes at >1% sig level.
> > > >
> > > > > Here the details:
> > > > > Sample size is 500 (ie. calling rand() 500 times),
> > > > > rnd range is 37 (ie. 0 to 36; yes, a roulette simulation).
> > > > > Doing the above mentioned StatTest after each rand() call.
> > > > > The above said is called more than 30 times in a loop,
> > > > > each time initializing the freq stats anew.
> > > > > srand(time(0)) done only once at pgmstart.
> > >
> > > I think your sample size is too small for such a conclusion to be
> > > statistically valid.
> >
> > The minimum neccessary is given in these relations
> > (cf. the above wiki pages and also stats books):
> >
> > n*p >= 5 AND n*(1-p) >= 5
> >
> > ie. for the above example:
> > n >= 5 / (1/37) >= 185
> > Above we have 500, ie. from 185 to 500 the test can be applied.
>
> But you said you're initializing the frequency stats before each loop
> iteration? To me, that would imply that your sample size is really only
> 36 for each of 30 or so runs. Or am I misunderstanding how you are
> accumulating the results?

The two loops I used are not that important.
I just simulate 30 days with each day 500 rand's (ie. spins, draws etc.).
At the beginning of each day the frequency stats are cleared.
Ie. the tests are done for intraday stats only.
I hope this makes it clear.

BTW, there is also a parallel discussion on this in sci.math
under the subject "Detecting biased random number generators".

Jerry Coffin

7/12/2008 1:28:00 AM

0

In article <g58mkt$rem$1@aioe.org>, joe@iamnotathome.org.invalid says...

[ ... ]

> The two loops I used are not that important.
> I just simulate 30 days with each day 500 rand's (ie. spins, draws etc.).
> At the beginning of each day the frequency stats are cleared.
> Ie. the tests are done for intraday stats only.
> I hope this makes it clear.
>
> BTW, there is also a parallel discussion on this in sci.math
> under the subject "Detecting biased random number generators".

Out of curiosity, what exact code are you using to get from the original
range of the generator to the 0-35 that you're using?

--
Later,
Jerry.

The universe is a figment of its own imagination.

joe

7/12/2008 9:55:00 PM

0

"Jerry Coffin" <jcoffin@taeus.com> wrote:
> joe@iamnotathome.org.invalid says...
>
> [ ... ]
>
> > The two loops I used are not that important.
> > I just simulate 30 days with each day 500 rand's (ie. spins, draws etc.).
> > At the beginning of each day the frequency stats are cleared.
> > Ie. the tests are done for intraday stats only.
> > I hope this makes it clear.
> >
> > BTW, there is also a parallel discussion on this in sci.math
> > under the subject "Detecting biased random number generators".
>
> Out of curiosity, what exact code are you using to get from the original
> range of the generator to the 0-35 that you're using?

Jerry, it is 0 to 36.
As recommended in many books (for example Phillip Good
"Permutation, Parametric and Bootstrap Tests of Hypotheses")
I use the following:

int genrand(int lo, int hi)
{
int z = rand() % (hi - lo + 1) + lo;
return z;
}
....
int r = genrand(0, 36);