The importance of performance, load and stress testing.

Performance testing is a practical approach of ensuring the quality characteristics of speed, stability and scalability of a software application. It can be further considered as the superset of more critical testing techniques such as load testing, stress testing endurance testing, spike testing and isolation testing. All these methods help to determine that the developed system is meeting its performance criteria. 

Load testing focuses on evaluating the response time of the system for transactions for a period of time. This type of testing is carried out to determine the behavior of the system under anticipated load in normal conditions. 

Stress Testing is performed in order to evaluate the performance characteristics of an application under test when it is subjected to workload models and load volumes which are beyond anticipation during production operations. It is also performed to determine the application behavior when it is subjected to load volumes while it is subjected to additional stressful conditions such as server failure, insufficient disk space or limited memory. 

Users expect webpages to load as quickly as possible. When this does not happen user satisfaction decreases. Websites that load faster have a competitive advantage. Having competitors just a click away makes it vital to keep load times quick in order to retain customers.

After speed, website scalability is another performance factor. It is important to determine how much a website can be enhanced with new features and configuration so that it can perform equally efficiently with additional enhancements.

There will be times, when a website will have to endure stressful situations like Black Friday, New Years Eve, etc., a time when crashing would not be  a good time. Even  a few minutes of downtime is extremely costly and may generate invalid records in the database.  Stress testing here proves to be very important as they determine the threshold limits of a webpage to  help the development team beforehand in addressing such exceptions if and when they occur.

Although many companies are hesitant to do so, the most accurate way to ensure the performance of an application, is to test it in the production system. This can be done by following some practices which can minimize its impact on the real users.

-      Test the application sufficiently before making it available to the users at all. This can be planned as a release management plan 

 

-      Plan for a scheduled maintenance and restrict all user interaction during that period of time. 

-      If none of the above two options are possible, then it is best to test the application on the off hours of off days like midnight on Friday, Saturday and Sunday.

-      Increase test load on the system gradually, so that the test transactions along with the real user transactions remain within the threshold limits. This will prevent any negative impact on the real users.

Conclusion

With advancement in technology and complexity in software features, performance testing has become more difficult. Part of it can also be attributed to the complexity of modern web browsers and HTML5. The performance metrics after testing should be very clearly informed to the stakeholders. Any defects should be checked and corrections should be immediately made.

Stress Testing of Web Applications

Stress Testing is performed to identify the issues of the application that can arise only in cases of extreme software and hardware conditions. These conditions may be heavy loads, extreme concurrency or limited resources. Stress Testing identifies the breaking point of the application and related system discrepancies that may occur just before that. Such inconsistencies may be:

- Loss or corruption of data.

- Unnecessary high resource utilization

.- Failure of application components to respond.

- Surfacing of new and un-handled exceptions.

Stress on an application can be due to one or more of the following reasons:

- Excessive user or data

- Denial of service

- Sudden spike in user transactions

- Unexpected outages

Steps for Stress Testing -

Step 1: Identify the test objectives

- The different ways that the system can possibly fail.

- Parameters to consider for building defenses against possible failures.

- Identify the behavior of the system under challenging situations.

- Ensure the stability of the system under stressful conditions.     

   

Step 2: Identify the main scenarios

-  Select scenarios critical to the overall application performance.

- Try operations that affect performance.

- Define scenarios based on potential bottlenecks through load testing and stress test in these areas.

Step 3: Identify the workload

- Incrementally increase the load and record the application performance under such conditions. Here, the key is to test the system with various workloads until a significant failure can be created.

The following activities help to identify the correct workload for stress testing:

- Observe the distribution of work

- Estimate the peak user load

- Simulate the anti-profile workload

Step 4: Identify the metrics

Metrics help to identify the potential problem areas in a system or application.The different types of metric categories are:

- Processor Utilization

- Memory Consumption

- Process recycles

- Available memory

- Utilized memory

- Disk utilization

- Network utilization

- Transaction Rate

- Successful transactions

- Failed transactions

- Successful orders

- Failed orders

- Rate of contention

- Deadlocks

- Thread allocation

- Transaction time

Step 5: Creation of Test Cases

In addition to workload profiles and key scenarios, additional parameters such as performance objectives, workload characteristics, test environment, test data and identified metrics are required to design a test case. The expected results should be mentioned for each test in such a way that the test can be marked as a pass or fail after its execution.

Step 6: Load Simulation

- Validate the test environment as expected.

- Ensure the correct configuration of the test and the test environment for metrics collection.

- Quickly execute a smoke test to ensure that the test script and the performance counters are correctly working.

- Reset and start the system.

Step 7: Result Analysis

Analyze and compare the results as per the accepted level for each metric. In case of any issues:

- Review the design.

- Review the code.

- Run the stress tests in environments where debugging is possible.

Conclusion

In practice, stress testing in generally applied in application testing, transaction testing and system testing. Exploratory stress testing is applied by subjecting the system to conditions that are very unlikely to occur. A stress test is sometimes confused with loadtest, which is a milder form of testing. Stress testing very brutally emphasizes on the robustness, reliability, stability and scalability of a web application.

An understanding of Performance Testing

Performance Testing of an application ensures its responsiveness, reliability, throughput and future scalability under a certain workload. Performance testing determines the following for an application.

-           Assess readiness for production.

-           Compare performance against performance criteria.

-           Find performance issues.

-           Determine throughput levels.

Performance Testing can be divided into the following test activities:

-           Performance Test: It is a technical investigation that determines characteristics such as responsiveness, scalability and throughput of the application system.

-           Load Test: It verifies the performance of the application under different conditions of work load ranging from normal to peak conditions. It helps to determine the response time, resource utilization levels, and throughput rates and further identifies the breaking point of the application on an assumption that the breaking point occurs within the peak load condition.

-           Endurance Test is classified under load testing. It validates the performance of the application when it is subjected to volumes of workload over a period of time. This helps to calculate the Mean Time between Failure and Mean Time to Failure for the application. 

-           Stress Test: it verifies the performance of the system when it is subjected to work beyond peak load conditions. The bugs generated here include synchronization problems, memory leaks etc. Stress test determines how the system functions under extreme loads.

-           Spike Test is classified under stress test. It validates the performance of a system by subjecting it repeatedly to load models beyond anticipation for short periods of time.

-           Capacity Test: This test determines how many transactions the application will support and still meet its performance parameters. It basically determines the future scalability of the system.

Risks Addressed by Performance Testing

-           Speed related risks

-           Is the application fast enough?

-           Is the application able to process the data before it becomes outdated?

-           Is the application able to provide the latest information?

-           Is the application able to respond within time before an error occurs?

-       Scalability related risk

-       Is the application able to provide consistent response time?

-           Is the application able to store all the data in its life time?

-           Is there any warning flag that shows it is reaching its peak capacity?

-           Will the application secure under heavy usage?

-           Will the application get compromised under heavy usage?

-           Will the application be able to handle heavy work loads?

-     Stability related risk

-           Is the application able to run for longer periods of time without inconsistencies?

-           What happens to incomplete transactions during abrupt system failure?

-           Will the application still function seamlessly once rebooted?

-           What are the probabilities of a system crash?

-           Can the system be scaled up or down without putting it down?

The main activities of performance testing are as follows:

-           Test Environment Identification

-           Identification of the Performance Criteria

-           Planning and designing of tests

-           Configuration of the test environment

-           Implementing the test design

-           Executing the tests

-           Analyzing results, reporting and retesting.

Conclusion

The goal of carrying out performance testing is not to find out bugs but to eliminate them. It is necessary before releasing the application into the market. There are a number of performance testing tools available. HP Loadrunner, HTTP Load and Proxy Sniffer are a few.

The different types of Performance Testing

Performance Testing of an application determines its responsiveness, throughput, reliability and scalability under a given workload. This process evaluates the software against its performance criteria. It compares the software performance on multiple system configurations and quantifies the levels of throughput.

Performance testing usually identifies the bottlenecks in a software application and helps to identify its compliance its initial requirements. The results generated by performance testing also helps to assess the hardware configurations that will be required while deployment. 

Types of Testing:

The types of performance testing can be categorized into the following three categories.

-      Performance Testing: This process validates the responsiveness, speed, scalability and stability of the software. It measures the response times, throughput and the level of resources utilized by the software. This category is the super set of all the performance related testing. 

-      Load Testing: This validates the performance characteristics of the software application when it is subjected from normal to peak load conditions that can be expected during production. It aims at identifying the breaking point of the application within peak load conditions. Endurance test can be considered as a subset of load testing. It determines the performance of the system under load conditions over an extended period of time. Endurance level of a system can be defined by Mean Time Between Failure(MTBF) and Mean Time to Failure(MTTF).

-      Stress Testing: This process is an extension of load testing. It verifies the performance characteristics of the system when it is subjected to conditions beyond the anticipated workloads during production.  It also tests the system under other stressful conditions such as limited memory, insufficient disk space and server failure. These tests help to estimate the conditions under which the application is liable to fail. It reveals bugs that can occur only under extreme load conditions. This helps to identify the indications which need to be monitored to avoid failures. Spike testing can be considered as a variation of stress testing that verifies the performance of a system on repeated extreme loads for short periods of time. 

-      Capacity Testing: It determines the upper limit of the users and transactions that can be supported by the software. It is conducted in conjunction with capacity planning. This can be used to anticipate the additional resources required to support increased user base or increased data volumes. It also helps to determine whether the system should be scaled up or scaled down in future.

Automated Performance Testing Tool:

JMeter is an automated open source testing tool in Java used to perform load and stress testing of application software. It can be used to test the performance of both static and dynamic resources of an application. It can simulate heavy loads on a server or group of servers to test overall performance under different load types. It generates the test results in graphical formats like charts and tables which are very easy to understand. It can be installed by running the JMeter Shell script in Linux or by starting the .bat file in Windows.

 

Conclusion: 

Performance Testing is an indispensable activity for assessing business risks. Besides identifying business risks, it reveals information about usability, functionality and security of the system. It is a qualitative and a quantitative evaluation of the software under test. It generally occurs at the end of the test plan. However, it should be also incorporated in the earlier stages of the development system when important logical structures are being decided.