Regression testing is the practice of running tests on existing software after a patch, update or change. Regression...
tests are designed to detect if and how any new additions to software affect the program. A common regression testing method is to run the same tests used on previous software versions to see if results differ in any way.
A variety of regression testing software is available from commercial vendors and open source projects. This software allows developers to automate testing procedures. Combining this software with the cloud's on-demand computing and storage capabilities offers the components of a low-cost, high-availability test platform.
Amazon Web Services (AWS) offers three different EC2 instance types -- On-Demand, Spot Instances and Reserved Instances. However, not all EC2 instance types are suitable for such tests. Spot Instances are the best fit for regression tests because they provide a relatively inexpensive, stable environment. Spot instances are extra EC2 instance types that are priced according to the current supply and demand of EC2 instances. When companies purchase Spot Instances, they can request Availability Zone, number of instances needed and the max price they intend to pay.
AWS offers the last 90 days of Spot Instance pricing. Use the ec2-describe-spot-price-history command to automatically collect details about price history and to calculate an appropriate bid (e.g. average price in past 24 hours * 1.5).
One potential problem with using AWS Spot Instances for regression tests is that you will lose your instance if the bid price drops below the current price. In automation scripts, test for the existence of results files or other verification that the regression test has completed. If results do not exist, you should re-run the test. If you find that you repeatedly lose Spot Instances because of low bids, consider increasing your bid. You will only be charged the current Spot Instance price -- even if your bid is higher. Therefore, there is no risk in spending more than the current spot price.
Knowing when to use AWS Spot Instances
How to implement regression testing
Which EC2 instance type offers the highest storage capacity?
Dig Deeper on AWS instances strategy and setup
Related Q&A from Dan Sullivan
Docker's recent upgrade introduced support for hardware signing and in the future, automated security analysis on Docker images. Expert Dan Sullivan ... Continue Reading
Cisco's new project Contiv automates operational policies for containerized applications in the cloud. Expert Dan Sullivan explains the benefits of ... Continue Reading
Dropbox API abused by attackers posing as legitimate users in a huge spear phishing campaign. Expert Dan Sullivan explains how to mitigate the risks ... Continue Reading