I ran into a peculiar situation with a PHP web application that went from working for several years without incident to suddenly resulting in timeouts and spiking the load on my server. Some investigation traced it back to a seemingly benign and obscure change to PHP’s rand()
implementation between 5.3.3 and 5.3.4.
To summarize several hundred lines of code: it gets a value from an array where the index is a random number between X and Y. X and Y are highly unpredictable by nature of the application. It keeps trying with different values until something is returned. Something like:
See it? If you don’t, you shouldn’t feel bad, I didn’t see it initially either.
Prior to PHP 5.3.4 mt_/rand
did not check if the max is greater than the min. This has changed as a result of bug 46587. That 4 line change made an impact.
Take this example code:
In PHP 5.3.3 you’d get:
$ php test.php right: 3 wrong: 4 $ php test.php right: 2 wrong: 5
Despite the incorrect order of max/min it actually worked just fine. It had done so at least since PHP 4.3 (circa 2003) as far as I’m aware.
In PHP 5.3.4:
$ php test.php right: 2 PHP Warning: mt_rand(): max(1) is smaller than min(5) in /test/test.php on line 4 wrong:
As a result, this while(){}
never terminated until the timeout was reached.
The solution is obviously trivial once you actually trace this bug back:
This resulted in several GB’s worth of warnings in my error log in a matter of hours. You can also see how it (the brown area) dropped off once the fix was deployed as measured by % of wall clock time:
It’s the little things sometimes that cause all the trouble.