Moodle development traffic 12/2010

Latest stable version is 1.9.8

On 25th March 2010, Moodle 1.9.8 was released. See Release notes for details. As usually, admins are advised to upgrade production servers.

There are 6 commits into MOODLE_19_STABLE from the last week done by humans. As always in this blog, commits by Moodle Robot are excluded from these statistics. Moodle Robot tirelessly increases the build number in version.php every day (yes, paradoxically that line with “Human-friendly version name” comment).  By the way – thanks to this job, Moodle Robot is the 10th most active contributor into Moodle code and even has reached Kudo Rank 8 at ohloh.net

In the last commit before Moodle 1.9.8 was released, Petr Skoda fixed proper handling of HTTPS wwwroot in Flash version detection (MDL-21910).

Unstable development version 2.0dev

There were 89 commits into the main development branch last week (from Tuesday 00:00 to Monday 23:59 in my git clone). Among other core developers, Sam Hemelryk committed his recent work on new themes layout (MDL-21862). In Moodle 2.0, there is kind of system theme called “base” which defines just very basic CSS. All other themes (including the new “standard”) are built upon this base theme. Web designers should follow http://docs.moodle.org/en/Development:Theme_changes_in_2.0 when preparing the themes for the incoming major release. I committed a series of patches that move language files into their plugin space (MDL-21694). In Moodle 2.0, English strings for activity modules and other plugin types are stored in the plugin space so there is no difference between core plugins and contributed plugins.

Quotes of the week

“2.0 beta is so much a priority right now I’ve stopped eating”
– Martin Dougiamas

“I think reinventing the wheel is usually a good idea personally, but that’s just me. ”
– Sam Marshall

“In PHP, every dog and his master wants to write their own framework.”
– Penny Leach

Eins, Zwei, Polizei

If you ask a mother how many children she had, you do not expect her actually counting them. She just knows. Similarly, if you ask PHP array how many items it contains using count() function, you expect it will be very cheap call. Knowing the number of contained items is so natural thing that array should return the value almost immediately with almost no cost. Well, apparently not in PHP. The following trivial simple script demonstrates the performance difference between using count() function compared with keeping the number of items in a separate counter:

define('MAXITEMS', 10000);

function using_array_count() {
    $a = array();
    while (count($a) < MAXITEMS) {
        $a[] = 1;
    }
}

function using_own_counter() {
    $a = array();
    $i = 0;
    while ($i < MAXITEMS) {
        $a[] = 1;
        $i++;
    }
}

$start = microtime(true);
using_array_count();
$t1 = microtime(true) - $start;

$start = microtime(true);
using_own_counter();
$t2 = microtime(true) - $start;

printf("%f %f %f \n", $t1, $t2, $t1/$t2);

At my notebook, the variant using count() is about five times slower than the one using the own counter. Not big deal, right? But Jens Eremie from Humboldt Universität in Berlin realized that in more complicated scenario (two arrays involved, resetting the array, keeping the number of items under a given limit etc.) the difference may be significantly higher – even hundreds times. Such a scenario that Jens was dealing with is Moodle cache_context() function. In MDL-19702 you can find a test script showing quite interesting figures. At large Moodle installations with many courses and categories (therefore many context) using count() in a loop makes caching a pain. Therefore, places like MyMoodle page run into a unacceptable poor performance and can easily reach PHP max_execution limit. After some not-so-complicated changes, things just run.

During a nice discussion at MoodleMoot in Berlin, Jens explained me several caching improvements he proposes. Getting rid of using count() is just one part of them. Jens realized that it does not make sense to shift the internal cache array (that is to remove the first cached item) after reaching the limit of cached items. As the contexts come into the cache in some order and they may be requested in the same order, the cache may actually never hit if the number of contexts is higher than cache limit. It is better to randomly pick instead of using the first item always. Then the cache works better in first-in first-out scenarios. Jens also proposes to follow the common wisdom (used for example in microprocessor caches) to prune items down to 80% of cache capacity at once instead of freeing items one by one when needed. And last, but not least, one the cached context is hit, it will be probably used more than once in Moodle code. Therefore the items should stay as long as possible in the cache once they have been hit so the random picking should prefer items with not hit so far.

Disclaimer: if I am not right or even if I am wrong, there is a bug in my understanding of the issue. I write here what I remember from the discussion with Jens and chances are I misinterpreted something. Blame me, not him. Praise him, not me.

Post scriptum

PostgreSQL’s EXPLAIN ANALYSE rocks.