PHP 5.3.4 Changes rand(), Filled My Error Log, Spikes Load

I ran into a peculiar situation with a PHP web application that went from working for several years without incident to suddenly resulting in timeouts and spiking the load on my server. Some investigation traced it back to a seemingly benign and obscure change to PHP’s rand() implementation between 5.3.3 and 5.3.4.

To summarize several hundred lines of code: it gets a value from an array where the index is a random number between X and Y. X and Y are highly unpredictable by nature of the application. It keeps trying with different values until something is returned. Something like:

See it? If you don’t, you shouldn’t feel bad, I didn’t see it initially either.

Prior to PHP 5.3.4 mt_/rand did not check if the max is greater than the min. This has changed as a result of bug 46587. That 4 line change made an impact.

Take this example code:

In PHP 5.3.3 you’d get:

$ php test.php
right: 3
wrong: 4

$ php test.php
right: 2
wrong: 5

Despite the incorrect order of max/min it actually worked just fine. It had done so at least since PHP 4.3 (circa 2003) as far as I’m aware.

In PHP 5.3.4:

$ php test.php
right: 2
PHP Warning:  mt_rand(): max(1) is smaller than min(5) in /test/test.php on line 4
wrong: 

As a result, this while(){} never terminated until the timeout was reached.

The solution is obviously trivial once you actually trace this bug back:

This resulted in several GB’s worth of warnings in my error log in a matter of hours. You can also see how it (the brown area) dropped off once the fix was deployed as measured by % of wall clock time:

It’s the little things sometimes that cause all the trouble.