The importance of performance, load and stress testing.

Performance testing is a practical approach of ensuring the quality characteristics of speed, stability and scalability of a software application. It can be further considered as the superset of more critical testing techniques such as load testing, stress testing endurance testing, spike testing and isolation testing. All these methods help to determine that the developed system is meeting its performance criteria. 

Load testing focuses on evaluating the response time of the system for transactions for a period of time. This type of testing is carried out to determine the behavior of the system under anticipated load in normal conditions. 

Stress Testing is performed in order to evaluate the performance characteristics of an application under test when it is subjected to workload models and load volumes which are beyond anticipation during production operations. It is also performed to determine the application behavior when it is subjected to load volumes while it is subjected to additional stressful conditions such as server failure, insufficient disk space or limited memory. 

Users expect webpages to load as quickly as possible. When this does not happen user satisfaction decreases. Websites that load faster have a competitive advantage. Having competitors just a click away makes it vital to keep load times quick in order to retain customers.

After speed, website scalability is another performance factor. It is important to determine how much a website can be enhanced with new features and configuration so that it can perform equally efficiently with additional enhancements.

There will be times, when a website will have to endure stressful situations like Black Friday, New Years Eve, etc., a time when crashing would not be  a good time. Even  a few minutes of downtime is extremely costly and may generate invalid records in the database.  Stress testing here proves to be very important as they determine the threshold limits of a webpage to  help the development team beforehand in addressing such exceptions if and when they occur.

Although many companies are hesitant to do so, the most accurate way to ensure the performance of an application, is to test it in the production system. This can be done by following some practices which can minimize its impact on the real users.

-      Test the application sufficiently before making it available to the users at all. This can be planned as a release management plan 

 

-      Plan for a scheduled maintenance and restrict all user interaction during that period of time. 

-      If none of the above two options are possible, then it is best to test the application on the off hours of off days like midnight on Friday, Saturday and Sunday.

-      Increase test load on the system gradually, so that the test transactions along with the real user transactions remain within the threshold limits. This will prevent any negative impact on the real users.

Conclusion

With advancement in technology and complexity in software features, performance testing has become more difficult. Part of it can also be attributed to the complexity of modern web browsers and HTML5. The performance metrics after testing should be very clearly informed to the stakeholders. Any defects should be checked and corrections should be immediately made.

Performance testing through the cloud

Performance has been the most important quality attribute with respect to the business efficiency and user acceptance. In terms of modern cloud based solutions, the response times and utilization rates required are crucial from the end users point of view. 

Performance testing is not only important for traditional IT systems, but also for cloud in various respects. Clouds are highly attractive IT systems. They are highly scalable. With increasing number of users, it utilizes a greater number of computers in order to ensure high performance. Performance testing has always been dependant on sophisticated tools. 

It needs to generate high volumes of test data to create a substantial workload and be more realistic. Hence, performance testing requires a complex and expensive test infrastructure.

    

The cloud is perfectly suitable in providing this type of infrastructure. So, even if the components behind performance testing are independent of the location of the system under test, the process itself benefits considerably from  cloud. 

The different kinds of interaction between cloud and testing can be mapped into four types. As per utilization cloud can be distinguished into four types:

-      Cloud as system under test: The cloud itself as a software product needs to be tested. This implies that the cloud itself as a system under test is deployed and executed within the cloud.

-      Cloud as testware utility: The cloud or services which accessible via the cloud can be utilized for testing other other IT systems.

-      Cloud as test environment: A software product or an application can be executed within cloud for testing purposes. Here the system under test is deployed and tested within the cloud, but subsequent operations may not be continued inside the cloud.

-      Cloud as test logistics: Here the cloud is used as a service in the sense of its typical cloud characteristics.

The cloud is becoming increasingly important as a production platform. Hence systems are now directly developed directly in the cloud. As cloud computing continues to develop, performance and load based testing can be counted as cloud offerings. Cloud based testing changes the way tests are performed.  Cloud based testing services change the testing methodologies of http and https applications through its following three characteristics:

-      Testing at scale: the cloud based testing provides a cost-effective means of testing applications at scale as compared to lab environment that can only simulate a small subset of the production environment. Thus, cloud based testing can test the application against the actual number of expected users.

-      Testing globally: Could bast testing enables testing the application on a global scale, thereby recreating the regions from which the users are accessing the application. This is usually done through partnerships with other cloud providers which allow customers to run loadtests constituting millions of concurrent users from different continents.

-      Testing production apps: Could testing can also be used for testing production apps. Testing in production implies testing at maximum capacity. Different problems then arise which do not show up in medium capacity.     This gives us a realistic picture of of what can actually go wrong and the potential adjustments that can be made before they occur.

Conclusion:

Cloud testing can be utilized as IaaS, PaaS or SaaS regarding the type of service the cloud provides. They are not necessarily independent, but can they can be modeled into a layered architecture as per requirement. Generally, clients use  SaaS via Web Browsers, Mobile apps and thin clients. Companies like Google, Netsuite, salesforce.com have been the real drivers behind the cloud computing revolution.

Stress Testing of Web Applications

Stress Testing is performed to identify the issues of the application that can arise only in cases of extreme software and hardware conditions. These conditions may be heavy loads, extreme concurrency or limited resources. Stress Testing identifies the breaking point of the application and related system discrepancies that may occur just before that. Such inconsistencies may be:

- Loss or corruption of data.

- Unnecessary high resource utilization

.- Failure of application components to respond.

- Surfacing of new and un-handled exceptions.

Stress on an application can be due to one or more of the following reasons:

- Excessive user or data

- Denial of service

- Sudden spike in user transactions

- Unexpected outages

Steps for Stress Testing -

Step 1: Identify the test objectives

- The different ways that the system can possibly fail.

- Parameters to consider for building defenses against possible failures.

- Identify the behavior of the system under challenging situations.

- Ensure the stability of the system under stressful conditions.     

   

Step 2: Identify the main scenarios

-  Select scenarios critical to the overall application performance.

- Try operations that affect performance.

- Define scenarios based on potential bottlenecks through load testing and stress test in these areas.

Step 3: Identify the workload

- Incrementally increase the load and record the application performance under such conditions. Here, the key is to test the system with various workloads until a significant failure can be created.

The following activities help to identify the correct workload for stress testing:

- Observe the distribution of work

- Estimate the peak user load

- Simulate the anti-profile workload

Step 4: Identify the metrics

Metrics help to identify the potential problem areas in a system or application.The different types of metric categories are:

- Processor Utilization

- Memory Consumption

- Process recycles

- Available memory

- Utilized memory

- Disk utilization

- Network utilization

- Transaction Rate

- Successful transactions

- Failed transactions

- Successful orders

- Failed orders

- Rate of contention

- Deadlocks

- Thread allocation

- Transaction time

Step 5: Creation of Test Cases

In addition to workload profiles and key scenarios, additional parameters such as performance objectives, workload characteristics, test environment, test data and identified metrics are required to design a test case. The expected results should be mentioned for each test in such a way that the test can be marked as a pass or fail after its execution.

Step 6: Load Simulation

- Validate the test environment as expected.

- Ensure the correct configuration of the test and the test environment for metrics collection.

- Quickly execute a smoke test to ensure that the test script and the performance counters are correctly working.

- Reset and start the system.

Step 7: Result Analysis

Analyze and compare the results as per the accepted level for each metric. In case of any issues:

- Review the design.

- Review the code.

- Run the stress tests in environments where debugging is possible.

Conclusion

In practice, stress testing in generally applied in application testing, transaction testing and system testing. Exploratory stress testing is applied by subjecting the system to conditions that are very unlikely to occur. A stress test is sometimes confused with loadtest, which is a milder form of testing. Stress testing very brutally emphasizes on the robustness, reliability, stability and scalability of a web application.

What is Load Testing?

Load Testing is a process by which simulated demand is put on the application under test to demonstrate its  behavior under different load conditions. Load Testing generally applies ordinary stress to the application to review its performance under regular anticipated conditions. Its final objective is to establish that the software is able to handle the load it has been designed to handle. 

Load Testing, a key type of performance testing, enables to measure the response times, throughput rate and the level of  resource utilization of the software application. It subjects the associated hardware, server, the network to its load limits. This can be done under lab conditions to accurately get an idea of  its capabilities or it can be conducted in the field to get a qualitative idea of its functions in the real world. 

Endurance testing is a type of loadtesting. It subjects the application to load volumes over an extended period of time. Endurance Testing quantifies the Mean Time to Failure and Mean TIme Between Failure.

Steps for Load Testing-

Step 1: Identify the performance acceptance criteria. This is determined by balancing the business, technology, competition and user requirements. The objectives of test generally include:

- Response Time

- Throughput

- Resource Utilization

- Business Load

- Maximum User Load

Step 2: Identify the anticipated user scenarios with high risk key performance goals. The main steps for identifying these scenarios are:

-  Identify the scenarios for the Web Application

-  Identify the activities in each scenario

-  Identify the commonly executed, most resource intensive scenarios

Step 3: Once the key scenarios are identified, the workload profiles can be created to design load tests. For this, the characteristics of the user scenario need to be defined.

- A user scenario can be thought of as a user session, which can include navigational paths and a number of intermediate steps to complete a task.

- Consider user pause as user delay or think time.

- Define the average time for each session for multiple concurrent views.

- Define the primary users.

Step 4: Identify the target load levels to ensure that the tests can be used to compare a variety of load conditions. The common inputs used for this are:

- Key Scenarios

- Volume of business

- Work distribution

- Session characteristics

Step 5: Identify the metrics that are most relevant to the performance objectives. Only well selected metrics provide valuable information. This can be done by:

- Defining performance questions that can be tested.

- Determine quality goals

- Identify the metrics

- Identify the supporting metrics

- Regularly reevaluate the metrics to be collected

Step 6: Designing tests can be done through the scenarios defined, selected key metrics and the workload analyzed. Each test has its own purpose, and is able to collect its own data and have its different target load levels.  Tests are designed to collect information that helps to understand, evaluate and enhance the application.

Step 7: Load simulations need to reflect the tests designed in order to collect useful information. The following points need to be considered for this:

 

- Configure the test environment as per the production environment.

- Ensure the not interference of performance counters with the simulation.

- Use appropriate tools for load generation.

- Begin with a small number of users, continue to increase the load, record the behavior till the threshold is reached, identify when the system crosses the threshold limit and continue to do so till the SLA limit is violated.

Step 8: Result Analysis

- Analyze the recorded data against the accepted level to determine the inward or outward trend from the performance objective.

- Analyze the data and identify the potential bottlenecks.  

Conclusion:

Load testing enables us to quantify the maximum operating capacity of an application. It should not be confused with Stress testing, which is performed to determine the application's breaking point. It is a more severe form of testing, which helps to record the different warning signals that a software gives out before its actual breakdown.