Q
Evaluate Weigh the pros and cons of technologies, products and projects you are considering.

Which EC2 instance types are better for regression tests?

When performing regression tests in the cloud, should we choose to run tests on one AWS EC2 instance versus another?

Regression testing is the practice of running tests on existing software after a patch, update or change. Regression...

tests are designed to detect if and how any new additions to software affect the program. A common regression testing method is to run the same tests used on previous software versions to see if results differ in any way.

A variety of regression testing software is available from commercial vendors and open source projects. This software allows developers to automate testing procedures. Combining this software with the cloud's on-demand computing and storage capabilities offers the components of a low-cost, high-availability test platform.

Amazon Web Services (AWS) offers three different EC2 instance types -- On-Demand, Spot Instances and Reserved Instances. However, not all EC2 instance types are suitable for such tests. Spot Instances are the best fit for regression tests because they provide a relatively inexpensive, stable environment. Spot instances are extra EC2 instance types that are priced according to the current supply and demand of EC2 instances. When companies purchase Spot Instances, they can request Availability Zone, number of instances needed and the max price they intend to pay.

AWS offers the last 90 days of Spot Instance pricing. Use the ec2-describe-spot-price-history command to automatically collect details about price history and to calculate an appropriate bid (e.g. average price in past 24 hours * 1.5).

One potential problem with using AWS Spot Instances for regression tests is that you will lose your instance if the bid price drops below the current price. In automation scripts, test for the existence of results files or other verification that the regression test has completed. If results do not exist, you should re-run the test. If you find that you repeatedly lose Spot Instances because of low bids, consider increasing your bid. You will only be charged the current Spot Instance price -- even if your bid is higher. Therefore, there is no risk in spending more than the current spot price.

Next Steps

Knowing when to use AWS Spot Instances

How to implement regression testing

Which EC2 instance type offers the highest storage capacity?

This was last published in April 2015

Dig Deeper on AWS instances strategy and setup

PRO+

Content

Find more PRO+ content and other member only offers, here.

Have a question for an expert?

Please add a title for your question

Get answers from a TechTarget expert on whatever's puzzling you.

You will be able to add details on the next page.

Join the conversation

2 comments

Send me notifications when other members comment.

By submitting you agree to receive email from TechTarget and its partners. If you reside outside of the United States, you consent to having your personal data transferred to and processed in the United States. Privacy

Please create a username to comment.

Do you perform regression testing?
Cancel
"Regression testing is the practice of running tests on existing software after a patch, update or change. Regression tests are designed to detect if and how any new additions to software affect the program. A common regression testing method is to run the same tests used on previous software versions to see if results differ in any way."

That has to be the most obfuscated definition of Regression testing I've ever heard.

Can we be honest here?  We don't test for regression, we test to try and prove there have been no regressions.   Running the same tests over and over again is a strategy that is sometimes utilized, but let's be clear that running the same tests will only verify the procedure works the same to the extent that it is run exactly the same way each time.  (Ever heard of the pesticide paradox)

I honestly don't know why there is so much emphasis on regression testing, how often do regressions (real regressions) actually happen? Barely I imagine.

But if you get past that definition in this article, the suggestion of using instances that can be taken away, sounds absolutely terrible to me :/
Cancel

-ADS BY GOOGLE

SearchCloudApplications

TheServerSide

SearchSoftwareQuality

SearchCloudComputing

Close