Make rand.t vanishingly unlikely to give false failure

Make rand.t vanishingly unlikely to give false failure

On Wed, 7 May 1997, Larry Schwimmer wrote:

> Subject: FYI: perl5.00399/t/op/rand.t test 7

> 	I know this sounds perverse, but it did happen to fail test 7
> of op/rand.t the very first time I ran make test on a Solaris 2.5.1
> machine, and the test does say to mail the developers if that
> happened.  (-:

And thank you for doing so. Ya done good. :-)

When I wrote that test, I had thought (erroneously, as it turns out) that
that test would never fail, or virtually never. Actually, on Solaris, it
can report a false positive about one time in two-to-the-15th tests.

That test attempts to ensure that srand's default seed isn't the same
twice in a row, which it shouldn't be. But was your test result falsely
positive, or was it a bug for real? We have no way to know.

> It worked fine the next 100 times I ran it and on the other seven
> builds,

Okay, if you had success the next 100 times, it's _probably_ a fluke.
There's no way to know for sure, though, short of finding a bug in the
srand code. :-(

I'm supplying a patch which makes the test more reliable without reducing
the sensitivity to bugs. This should effectively eliminate this problem,
except for unavoidable coincidences.

> but it might be nice to run the test file multiple times to
> reduce the likelihood of a false failure while still catching errant
> builds.

Actually, that wouldn't do the trick. If we ran it five times, and one of
those attempts gets the same srand seed twice, that's _still_
unacceptable. The program has to notify a human, since recompiling perl,
checking the source, and asking for advice are things that humans still do
better than machines. But this patch will make the machine a little better
at knowing when to cry "Wolf!" :-)

Thanks!

p5p-msgid: Pine.GSO.3.96.970510190846.23340K-100000@kelly.teleport.com