Case study: Ushahidi

 

“Load Impact is by far the easiest tool we have found for load testing our products”

 

Summary:

Ushahidi used Load Impact to greatly improve the performance of its software. Through comparing “before” and “after” test results it was possible to see the impact of performance optimizations made.

About the customer:

Ushahidi is a non-profit tech company that specializes in developing free and open source software for information collection, visualization and interactive mapping. Such software is deployed during disasters so that real time information can be shared across the web. Like Wordpress, the software can be self hosted or hosted on the company’s server.

Case:

Ushahidi software is generally used for crisis and disaster situations so optimization is absolutely crucial. An earthquake reporting site based on Ushahidi software (http://www.sinsai.info/) received a spike in traffic after the earthquake and tsunami in Japan and it went down several times, causing service outage at the time the service was needed the most.

Ushahidi were interested in using a load testing tool to test the performance of their software before and after optimization efforts, to determine what effect the optimizatons had had. 

Test setup:

There were four load tests run on two different versions of the Ushahidi software. The software was hosted on Ushahidi’s servers. The first two test runs used ramp-up configurations up to 500 concurrent users on the test sites to test performance differences between Ushahidi 2.0.1 and Ushahidi 2.1. The results were revealing, showing performance graphs that were practically identical. There hadn’t been any change in performance from 2.0.1 to 2.1.

From these tests, it was also found out that the theoretical total number of concurrent users for Ushahidi on a typical webserver is about 330 clients but may be lower, depending on configuration. Load times at the 330-client level were very high, however, and defining the largest acceptable page load time to be 10 seconds meant that a more realistic figure would be 100 concurrent users on the typical webserver.

Finally, Ushahidi wanted to measure the potential performance gain when using a CDN (content delivery network). The Ushahidi 2.1 software was modified so that static resources were loaded from Rackspace’s CDN service instead of the Ushahidi server, then the previous load test was executed again. The result was a major increase in the number of concurrent users the system could handle. Where previous tests had shown a significant slowdown after 60-100 concurrent users, and an absolute max limit of about 330 concurrent users, the CDN-enabled site could handle more than 300 concurrent users before even starting to slow down. To find out the extreme limit of the site with CDN enabled, a final test was run with even higher load levels, and it was found that the server now managed to serve content at load levels up to 1,500 concurrent users, although with the same high load times as in the 330-client case with no CDN.

Service environment:

  • Apache
  • PHP
  • MySQL
  • Linux (CentOS 5.0) 

Challenges:

  • Find load limits for 2 different software versions
  • Find load limits with/without CDN enabled for static files
  • Detect potential problems in the infrastructure or web app before they affect customers 

Solution:

  • Run ramp-up tests with identical configurations on the 2.01 and the 2.1 software. See which one performs better or worse
  • Run ramp-up tests with identical configurations on the 2.1 software with CDN enabled, and without CDNenabled. See which performs better or worse.
  • Run final, large-volume ramp-up test for the CDN-enabled software, to find out its theoretical maximum concurrent user limit. 

Results:

  • Ushahidi found out that there was a significant performance gain when using CDN to serve their static files.
  • Load test measured that performance increased by 300% – 400% when using the CDN
  • Load times started to increase only after 334 concurrent users when using the CDN, and the server timed out at around 1500 concurrent users.
  • Faster time to verify CDN deployment. Test also quantified % increase in performance which leads to justification for additional cost of CDN service.
  • Test showed no changes in load time between version 2.01 and 2.10.

Feedback and Knowledge Base