Solutions

Stress & load testing SOA-based systems

Business Problem

SOA application performance and reliability must be verified

CalPERS is implementing a new system that will extend the self-service capabilities of their existing site for both members and employers. The new system will integrate two internal systems using web services. The performance and reliability of these interfaces is critical to the success of the project.

Solution

SOA application performance testing

CloverLeaf developed a performance test strategy and scripts for testing the SOA-based application and interfaces.

Nested service calls

Web services that call each other (sometimes recursively) complicate the performance test strategy. Knowing if the system will perform correctly is only one part of the equation - knowing where the problem(s) lie is equally important. Page-level testing only tells the latency for the page and the services it calls. If a service calls another service, the latency of the called service is often difficult to determine.

Performance test orchestration

Due to the complexity of the interface relationships, the testing strategy called for a combination of LoadRunner scripts to exercise the web pages (simulating real users) in conjunction with SOA Test scripts to test the web service interfaces under load.

Page level testing

While it is typical to only script a representative sampling of heavy hit pages, it was necessary to script a larger sample in order to provide proper load simulation. Standard LoadRunner transactions were established for page level analysis.

Interface level testing

SOA Test scripts were developed to test the web service interfaces. In cases where one system (A) would call a web service on the other system (B), which would in turn call a web service on the calling system (A), determining latencies for each call was critical in understanding where bottlenecks occur.

Instrumentation

Instrumentation was added on the web pages calling the services to log events when the response time for a web service call exceeded established thresholds.

CloverLeaf Added Value

CloverLeaf scripted and executed interim performance testing for a current production patch deployment. Before and after analysis was done to ensure the patch did not negatively impact performance.

Benefits

• Re-usable automated test scripts are invaluable for regression testing. As new fixes and enhancements are developed, re-executing the scripts verifies the new deployment will not break anything.
• Various scenarios can be run to determine performance under different conditions (number of users, number and frequency of page hits, etc.)
• Statistical analysis can be automatically generated after each test.