Case study: J. Walter Thompson

“Load Impact’s service allowed us to load and stress test an advertising campaign and helped us successfully prepare for high traffic volumes”


Summary:

JWT, one of the largest advertising agencies in the United States and the fourth-largest in the world, used Load Impact to perform load tests to verify that their new campaign site could handle up to120,000 visitors/hour. 

The test revealed that the website could handle the expected traffic and that response times were well within an acceptable range. The campaign was successfully launched on YouTube. 

Background:

According to an independent global research study undertaken by Vanson Bourne, even minor delays to website response times can have a sizable impact on customer satisfaction, page views, conversion rates and site abandonment. Despite this, an astonishing 68% of website owners experienced performance or stability problems and 32% of organizations do not know if their website is monitored on a 24x7 basis*. To make matters worse, 47% of PC visitors, 69% of tablet visitors and 34% of smartphone visitors expect response times equal to or below 2 seconds**. 

In an effort to ensure quality performance of a new campaign website built for a client in the pharmaceutical industry, Load Impact was commissioned to establish that the backend for the website could handle the expected traffic – 120,000 visitors per hour – while exhibiting load times that were within acceptable limits. 

The campaign-site was built with a user signup/login procedure and offers an interactive online game. The backend for the service is hosted in Texas, and all static content is distributed through a CDN which makes calls to the backend servers. There is also an external partner which handles the user database including registration, signup etc.

Test setup:

For the purpose of testing the backend only, a set of specific user actions were defined, such as “user registration”, “user sign-in”, and other actions where the backend systems had to be involved. These actions were activated through requesting certain URLs, one for each type of action, that were created specifically for the load test. In practice it meant that a simple API was created only for running the load test.

The simulated users in the test were configured to perform a series of these predefined user actions, resulting in corresponding calls to the backend systems. The static content, normally served through CDN operators, was ignored in the test.

The test was run as a series of 5 minute ramp-up tests (simulating approximately 8.33 clients per second), individually tuned depending on results of previous tests and designed to find out the breaking point of the target system. 

Service environment:

  • Apache
  • PHP
  • MySQL
  • Linux

The tested environment consisted of an HTTP load balancer plus web-, application- and database servers.

Challenges:


There were numerous challenges that the test was designed to detect. First of all, there was a need to validate that the system could handle the expected amount of traffic and establish a performance baseline. The test was also set-up to detect potential problems in the infrastructure and web app. 
  • Validate that the system could handle the expected amount of traffic
  • Detect potential problems in the infrastructure or web app
  • Establish a performance baseline

Solution:


The solution agreed upon was to generate and measure load from multiple geographic locations as well as to measure response times, throughput and customer experience.  
  • Load generation and measurements from multiple geographic locations
  • Application response time, throughput and customer experience analysis provided for JWT

Results:


The results of the load test revealed that the campaign website could withstand the expected traffic and there were no specific performance problems with the site. Therefore, a baseline was established at about the required level of 120k visitors/hour. 






The external service provider of user registration and sign-in functionality had no problems and their response times remained constant during the tests while the other backend services exhibited response times that were stable until just over the required level of 120,000 visitors/hour, after which response times started to increase rapidly and exponentially. 

Specifically, the response times for the start page were under 1 second for up to 2,000 concurrent visitors.
 Response times for the configured tests, which included the set of specific user actions, were under 2 seconds for up to 2,000 concurrent visitors. Considering that the average response time for similar campaigns of this size is above 4 seconds*, these results were impressive. The campaign site was launched successfully on Youtube.

The campaign site was launched successfully on Youtube on August 31, 2011

*Source: State of Web Readiness Report, Load Impact, 2013 

**Source: How To Deliver Fast, Engaging Responsive Web Design Sites, Akamai, 2012

Feedback and Knowledge Base