Calendar Load Testing

Testing Process

To perform the load testing we used the Apache HTTP server benchmarking tool, also known as ab. This tool is distributed with Apache and its documentation can be found here http://httpd.apache.org/docs/2.2/programs/ab.html.

We tested version 2.0.2 of Kinetic Calendar.

We tested using the following architectures: a single tomcat instance, two load balanced tomcat instances, and four load balanced tomcat instances. Each tomcat instance was installed on its own virtual machine with the following specifications: Windows 2008 RC2, 64bit, 4GB Ram, 2x 2.4GHz, Java 7u21 (64bit), Tomcat 7.0.39 (64bit) 1GB.

In addition to testing with multiple architectures we also performed testing with a variety of calendars. Because there is processing of events on the server-side before they are sent to the client, calendars that contain more events will take more time for the application to process.

Finally, to simulate different amounts of load on the application we varied the concurrency value of ab. Raising the concurrency value simulates higher load on the application.

Additional details about the environment. We used HAProxy on Ubuntu (2x2.4GHz) as a load balancer and a single Remedy 7.6 server (4x2.4 GHZ). The single remedy server did not appear to be a bottleneck for any of the tests.

Note About Concurrency

Concurrency in these test cases does not mean number of users. Concurrency in these test cases means the number of requests being handled by the application at the same point in time. You can think of it as a number of users that click a calendar link within the same second or less.

Results

The results are included below, there is a graph and a table for each type of calendar tested.

In the tables, the value along the top is the concurrency value given ab for that test case and the value along the left was the number of tomcat instances. The value in the table is the average response time in milliseconds for that test case.

In the graphs, the value along the x-axis is the concurrency value and the value along the y-axis is the response time in milliseconds. Note that there is a line for each of the server setups.

Small Calendar

In this test case the calendar contains 25 events.

Concurrent

5

10

25

50

75

100

1 Server

40

59

162

173

799

1450

2 Servers

37

56

341

930

1222

1183

4 Servers

36

55

230

1012

1250

1183

Medium Calendar

In this test case the calendar contains 100 events.

Concurrent

5

10

25

50

75

100

1 Server

97

179

452

876

1325

1770

2 Servers

63

105

228

450

718

914

4 Servers

53

80

195

498

751

1078

Big Calendar

In this test case the calendar contains 1000 events.

Concurrent

5

10

25

50

75

100

1 Server

1170

2331

5838

11758

17735

23895

2 Servers

595

1177

2942

5913

8888

11892

4 Servers

396

749

1947

3925

5840

7761

Multi Calendar

In this test case the calendar contains 100 events, but there are 4 event types. This means the calendar application must make 4 separate queries to retrieve its data. The purpose was to see the performance impact of this case to retrieving 100 events from a single source (Medium Calendar).

Concurrent

5

10

25

50

75

100

1 Server

127

235

570

1163

1773

2526

2 Servers

146

247

476

710

1143

1511

3 Servers

91

131

310

599

1004

1290


Did this page help you?