Make rand.t vanishingly unlikely to give false failure

On Wed, 7 May 1997, Larry Schwimmer wrote:

> Subject: FYI: perl5.00399/t/op/rand.t test 7

> 	I know this sounds perverse, but it did happen to fail test 7
> of op/rand.t the very first time I ran make test on a Solaris 2.5.1
> machine, and the test does say to mail the developers if that
> happened.  (-:

And thank you for doing so. Ya done good. :-)

When I wrote that test, I had thought (erroneously, as it turns out) that
that test would never fail, or virtually never. Actually, on Solaris, it
can report a false positive about one time in two-to-the-15th tests.

That test attempts to ensure that srand's default seed isn't the same
twice in a row, which it shouldn't be. But was your test result falsely
positive, or was it a bug for real? We have no way to know.

> It worked fine the next 100 times I ran it and on the other seven
> builds,

Okay, if you had success the next 100 times, it's _probably_ a fluke.
There's no way to know for sure, though, short of finding a bug in the
srand code. :-(

I'm supplying a patch which makes the test more reliable without reducing
the sensitivity to bugs. This should effectively eliminate this problem,
except for unavoidable coincidences.

> but it might be nice to run the test file multiple times to
> reduce the likelihood of a false failure while still catching errant
> builds.

Actually, that wouldn't do the trick. If we ran it five times, and one of
those attempts gets the same srand seed twice, that's _still_
unacceptable. The program has to notify a human, since recompiling perl,
checking the source, and asking for advice are things that humans still do
better than machines. But this patch will make the machine a little better
at knowing when to cry "Wolf!" :-)

Thanks!

p5p-msgid: Pine.GSO.3.96.970510190846.23340K-100000@kelly.teleport.com

diff --git a/t/op/rand.t b/t/op/rand.t
index 23a09b7..c779f9d 100755 (executable)
--- a/t/op/rand.t
+++ b/t/op/rand.t
@@ -210,7 +210,7 @@ sub bits ($) {
 {
     srand;             # These three lines are for test 7
     my $time = time;   # It's just faster to do them here.
-    my $rand = rand;
+    my $rand = join ", ", rand, rand, rand;
 
     # Hints for TEST 5
     # 
@@ -251,7 +251,7 @@ sub bits ($) {
     #
     while ($time == time) { }  # Wait for new second, just in case.
     srand;
-    if (rand == $rand) {
+    if ((join ", ", rand, rand, rand) eq $rand) {
        print "not ok 7\n";
        print "# srand without args isn't varying.\n";
     } else {