The Competition

Development and evaluation of the initial three sites, representing Drupal, Joomla!, and WordPress, occurred in February and March 2009 in preparation for the Ultimate Showdown of Content Management System Destiny panel at South by Southwest Interactive on March 16, 2009.

The specification, to develop a Web-based collaboration platform for community leadership programs, was developed by George DeMet and Tiffany Farriss of Palantir.net with input from Marybeth Schroeder of the Evanston Community Foundation and Don Wise of Park University. The site's design site's design concepts were developed by renowned interactive designer Mark Boulton, author of the recently published “Five Simple Steps: A Practical Guide to Designing for the Web”.

The three teams included:

Drupal

• Colleen Carroll, Palantir.net
• Larry Garfield, Palantir.net
• Jeff Eaton, Lullabot.com

Joomla!

• Steve Fisher, Idea Market
• Amy Stephen, Tamka.org
• Arno Zijlstra, Alvanna Creative
• Kevin Devine, PICnet
• Tibor Toth, Joomla! Association Hungary

WordPress

• Matt Mullenweg, Automattic
• Beau Lebens, Dented Reality
• Noel Jackson, Automattic
• Andrew Ozz, Automattic

The project specification and designs were delivered to all three teams on February 16, and they were given a deadline of 11:59pm CST on March 2 to develop their sites, which also had to conform to the following requirements:

• No team could spend more than 100 hours working on the project.
• Only freely-available software can be used, and all sites must in turn be freely-available.
• Site must function on the provided (Linux/Apache/MySQL) shared hosting space.

The design implementation for all three sites was evaluated by Mark Boulton:

Each team has applied the visual language across elements that I didn't supply a design for - overall, this
was great and something I was pleased to see. It does indeed look like all the teams were not limited by their choice of CMS in realising the design....I actually think that it's a close tie between WordPress and Drupal. From a typographic standpoint, the Joomla! team were off slightly. Am allowed to say a tie from that standpoint? :)

All three sites were also evaluated by Marybeth Schroeder, who looked at both the front-end and the administrative back-end of each site using the login information provided by each team, and offered her opinion on each one's ease of use. She did not view any documentation or training material, and the order of the sites was determined by random draw: Drupal, then Joomla!, then WordPress.

"I think that Drupal was my favorite, but Joomla! was a close second, and WordPress was a distant third. I found the WordPress one really difficult, whereas I think Drupal edged out Joomla!, but I saw some elements of Joomla! that I liked, and both of them I think if I worked with them more some of the things I found frustrating I would figure out."

Understanding that the circumstances under which she tested all three sites was less than ideal, and desiring to learn more about them, Marybeth agreed to conduct a re-evaluation of all three sites after she had an opportunity to review documentation and/or training material provided by all three teams.

Information was also collected about the number of hours used by each team, Web standards validation, page weight, and number of lines of custom PHP/Javascript code used:

In the case of HTML and CSS validation, the errors for all three sites were minor. The amount of custom code used for the WordPress site was due largely to the custom installer created by that team and additional theme-level code.

The audience at SXSW refused to render a verdict in the competition, saying they wanted to see more of the sites before coming to a decision. While time did not permit walkthroughs of all three sites, the source code and installers for all three sites will soon be made available from this site so that people can download them and try them out for themselves.