Performance Testing in the cloud

In simple words, Internet-based-computing can be referred to as Cloud computing. Here, shared hardware and software resources are available for use as per demand. This concept is also based on optimum utilization of the increasing power of servers and virtualization technologies. It is the convergence of computing, networking and storage which create a dynamic infrastructure which makes these functions highly dependant on each other. This forces a change in management strategy. 

Performance generally refers to the statistics regarding response time, throughput and resource utilization of an application. It is not difficult to assess that a performance problems impact customer experience, revenues,  brand perception and overall user satisfaction. 

Let us review the tools for performance testing in the cloud.

- Load Storm:  This allows developers to evaluate how an application will respond under heavy volumes of HTTP traffic. It gives massive cloud resources in the hands of web developers which enables them to improve the performance of their application.

- SOASTA – CloudTest: It provides performance, functional, Web User Interface and AJAX testing as an on-demand service. 

- VMLogic : And Skytap offer scalable test labs along with management software in the cloud. On an average these can cost several thousand  dollars per month.

- TechOut: This tool helps to measure and optimize the speed and availability of  critical online applications. It also provides Website Monitoring, REST/SOAP Web Services Monitoring, Business Transactions Monitoring, Cloud Monitoring, etc. 

- IntroSpectrum: It is a hosted performance monitoring system that makes use of real browsers to provide the most accurate view of website performance. It is hosted in the cloud and hence it is platform/OS agnostic. It works with any public website or any web application.

- LoadIntelligence: It can use JMeter in the cloud. It is an affordable Software as  a Service from Cloud Intelligence. It includes access to software and unlimited hardware resources. Users of JMeter can execute their test scripts in an unlimited, pre-configured distributed environment. It neither requires setup nor installations. Also, it provides immediate access to JMeter logs, reports, test scripts, CSV files and more. 

- Xceptance LoadTest: It is a load testing and regression tool for  Web apps and Java apps load testing. For web-based tests it can emulate Internet Explorer or Firefox behavior. 

- Grinder in the Cloud: It is a cloud based, open source load testing tool which uses resources offered by Amazon Web Services. It leverages Grinder Load Test Framework by putting it in the cloud.

-  Pylot: It is a Web Testing tool from Amazon Elastic Compute Cloud(EC2). It generates concurrent load (HTTP Requests), verifies server responses and produces reports with metrics. Test suites, which are executed are monitored from a GUI or shell/console. 

 -  HPLoadRunner, Keynote and BrowserMob are some additional cloud based performance testing tools.

Conclusion

Performance is critical to revenue and an efficient performance tuning of an application can increase revenue by 10%. Cloud performance, which is measured at the point of application use is the sum of application performance, network performance and cloud infrastructure performance. Cloud based performance testing tools offer a way to test across platforms and ensures capacity even in the most extreme scenarios.

The importance of performance, load and stress testing.

Performance testing is a practical approach of ensuring the quality characteristics of speed, stability and scalability of a software application. It can be further considered as the superset of more critical testing techniques such as load testing, stress testing endurance testing, spike testing and isolation testing. All these methods help to determine that the developed system is meeting its performance criteria. 

Load testing focuses on evaluating the response time of the system for transactions for a period of time. This type of testing is carried out to determine the behavior of the system under anticipated load in normal conditions. 

Stress Testing is performed in order to evaluate the performance characteristics of an application under test when it is subjected to workload models and load volumes which are beyond anticipation during production operations. It is also performed to determine the application behavior when it is subjected to load volumes while it is subjected to additional stressful conditions such as server failure, insufficient disk space or limited memory. 

Users expect webpages to load as quickly as possible. When this does not happen user satisfaction decreases. Websites that load faster have a competitive advantage. Having competitors just a click away makes it vital to keep load times quick in order to retain customers.

After speed, website scalability is another performance factor. It is important to determine how much a website can be enhanced with new features and configuration so that it can perform equally efficiently with additional enhancements.

There will be times, when a website will have to endure stressful situations like Black Friday, New Years Eve, etc., a time when crashing would not be  a good time. Even  a few minutes of downtime is extremely costly and may generate invalid records in the database.  Stress testing here proves to be very important as they determine the threshold limits of a webpage to  help the development team beforehand in addressing such exceptions if and when they occur.

Although many companies are hesitant to do so, the most accurate way to ensure the performance of an application, is to test it in the production system. This can be done by following some practices which can minimize its impact on the real users.

-      Test the application sufficiently before making it available to the users at all. This can be planned as a release management plan 

 

-      Plan for a scheduled maintenance and restrict all user interaction during that period of time. 

-      If none of the above two options are possible, then it is best to test the application on the off hours of off days like midnight on Friday, Saturday and Sunday.

-      Increase test load on the system gradually, so that the test transactions along with the real user transactions remain within the threshold limits. This will prevent any negative impact on the real users.

Conclusion

With advancement in technology and complexity in software features, performance testing has become more difficult. Part of it can also be attributed to the complexity of modern web browsers and HTML5. The performance metrics after testing should be very clearly informed to the stakeholders. Any defects should be checked and corrections should be immediately made.

Performance testing through the cloud

Performance has been the most important quality attribute with respect to the business efficiency and user acceptance. In terms of modern cloud based solutions, the response times and utilization rates required are crucial from the end users point of view. 

Performance testing is not only important for traditional IT systems, but also for cloud in various respects. Clouds are highly attractive IT systems. They are highly scalable. With increasing number of users, it utilizes a greater number of computers in order to ensure high performance. Performance testing has always been dependant on sophisticated tools. 

It needs to generate high volumes of test data to create a substantial workload and be more realistic. Hence, performance testing requires a complex and expensive test infrastructure.

    

The cloud is perfectly suitable in providing this type of infrastructure. So, even if the components behind performance testing are independent of the location of the system under test, the process itself benefits considerably from  cloud. 

The different kinds of interaction between cloud and testing can be mapped into four types. As per utilization cloud can be distinguished into four types:

-      Cloud as system under test: The cloud itself as a software product needs to be tested. This implies that the cloud itself as a system under test is deployed and executed within the cloud.

-      Cloud as testware utility: The cloud or services which accessible via the cloud can be utilized for testing other other IT systems.

-      Cloud as test environment: A software product or an application can be executed within cloud for testing purposes. Here the system under test is deployed and tested within the cloud, but subsequent operations may not be continued inside the cloud.

-      Cloud as test logistics: Here the cloud is used as a service in the sense of its typical cloud characteristics.

The cloud is becoming increasingly important as a production platform. Hence systems are now directly developed directly in the cloud. As cloud computing continues to develop, performance and load based testing can be counted as cloud offerings. Cloud based testing changes the way tests are performed.  Cloud based testing services change the testing methodologies of http and https applications through its following three characteristics:

-      Testing at scale: the cloud based testing provides a cost-effective means of testing applications at scale as compared to lab environment that can only simulate a small subset of the production environment. Thus, cloud based testing can test the application against the actual number of expected users.

-      Testing globally: Could bast testing enables testing the application on a global scale, thereby recreating the regions from which the users are accessing the application. This is usually done through partnerships with other cloud providers which allow customers to run loadtests constituting millions of concurrent users from different continents.

-      Testing production apps: Could testing can also be used for testing production apps. Testing in production implies testing at maximum capacity. Different problems then arise which do not show up in medium capacity.     This gives us a realistic picture of of what can actually go wrong and the potential adjustments that can be made before they occur.

Conclusion:

Cloud testing can be utilized as IaaS, PaaS or SaaS regarding the type of service the cloud provides. They are not necessarily independent, but can they can be modeled into a layered architecture as per requirement. Generally, clients use  SaaS via Web Browsers, Mobile apps and thin clients. Companies like Google, Netsuite, salesforce.com have been the real drivers behind the cloud computing revolution.