Open Source solutions for Call Centre performance testing. SIPp: pros and cons

Projects always have hidden issues, 100% of the time. Those issues, if left unresolved, can result in major deployment delays, frustrations, impacts on user experience and lost revenues. Testing reduces the risk of negative surprises and is more effective in terms of both costs and time. The sooner you find the issues the cheaper it is to get it fixed. Contact Centre projects are no exception to that rule. Modern telephony is usually based on VoIP technologies and requires higher processing horsepower compared to a typical web application. That complexity creates a series of unique architectural challenges and involves thorough Load and Performance testing. Here's a short list of possible issues, If you wonder what might go wrong in your system while under load:

• Your customers aren't able to navigate IVR menus any more

• There's poor audio quality on calls

• Some calls have echo

• Registration of agent's phone is constantly dropping

• Your softphone gets timeout errors

• The incoming calls don’t reach your agents

• Only one phone gets to ring when call hunting

• Your agents wait for extended periods of time in Ready state before customer's call is delivered

So many possible problems. The reality is that if something can possibly go wrong, it will go wrong when using IP based communications. The reverse is that everything can go right. If you've read that far, you probably decided that Load and Performance testing of your Contact Centre solution is highly important, but you still might wonder if there are any tools that would assist you in achieving that. Let’s make a brief comparison between open source solutions like SIPp with the extension Sippy Cup and Data Synergy Voice.

From the SIPp documentation:

SIPp is a performance testing tool for the SIP (VoIP) protocol. It includes a few basic SipStone user agent scenarios (UAC and UAS) and establishes and releases multiple calls with the INVITE and BYE methods. It can also read XML scenario files describing any performance testing configuration. It features the dynamic display of statistics about running tests (call rate, round trip delay, and message statistics), periodic CSV statistics dumps, TCP and UDP over multiple sockets or multiplexed with retransmission management, regular expressions and variables in scenario files, and dynamically adjustable call rates.

SIPp can be used to test many real SIP equipment like SIP proxies, B2BUAs, SIP media servers, SIP/x gateways, SIP PBX, etc. It is also very useful in emulating thousands of user agents calling your SIP system. In short, SIPp can simulate one or more calls to your system in an automated fashion by directly leveraging the SIP and RTP protocols, testing the SIP dialogs and generating statistics. It can run a specified number of concurrent calls, ramping up to a maximum number of calls, and many other uses.

As great as SIPp is, however, it’s not the easiest thing to harness. It requires test scenarios to be written in a fairly verbose XML format. But even if you love XML, the real pain is lining up the call media. In our world, load tests usually need to interact like a human caller might. This primarily means sending DTMF button presses and voice utterances to the voice application. SIPp doesn’t send media by default, and it gets upset if you try to re-use/loop a single silent audio file (at least, it does without a third-party patch). But even then, it’s very easy to sleep just a little too long and have your telephony engine hang up for lack of received media. Usually this forces us to record an interaction with the application, then use SIPp to play that interaction back. But that makes setting up new test scenarios rather time consuming.

Sippy Cup is a tool to generate SIPp load test profiles and the corresponding media in PCAP format. It takes an input document that describes a load test in a simple way (call this number, wait this many seconds, send this digit, wait a few more seconds, etc) and generates inputs required by SIPp for execution of the test. It definitely makes life of performance tester easier, but still requires quite intimate knowledge of SIP internals.

Data Synergy Voice, on other hand, allows your performance tester to continue using their favourite tool, be it HP LoadRunner or Microfocus SilkPerformer. It provides state of the art easy to use and intuitive test call flow designer. It integrates all captured performance metrics into standard reports and allows you to capitalize on a rich range of natively supported protocols to cover additional aspects of the Contact Centre solution such as WFM, Operational Reports, Wallboards, ...

In the next post we will review process of creating basic test scenario in HP LoadRunner, Microfocus SilkPerformer and compare it with SIPp.