News Stay informed about the latest enterprise technology news and product updates.

Developers use AWS spot instances as Chaos Monkey to test apps

Spot instances were introduced by Amazon five years ago, but some AWS shops are just hitting their stride with novel use cases for the instance type.

Amazon Web Services customers put spot instances to use in creative ways, especially for new application development.

Advantages to Amazon Web Services (AWS) spot instances include a very low price point and the ability to spin up capacity quickly. The main disadvantage is that if someone outbids the initial spot price, the original spot instance will disappear, without warning or the ability to save anything.

Thus, a common use for AWS spot instances is high-performance computing (HPC), in which firms performing financial or scientific analysis spin up hundreds or thousands of machines for a short time.

But some advanced AWS users have found other ways to use the service in spite of its availability constraints -- and in some cases even because of them.

"One of the use cases that we get the most out of is testing," said Kent Langley, CEO of Ekho Inc., a Web-based data analytics company based in San Rafael, Calif. "When we've got something new [and] want to see how it performs at a large scale … using spot instances is a no-brainer."

We use spot instances as our own personal Chaos Monkey, in a way, because you never know when they're going to go away.
Kent LangleyCEO, Ekho Inc.

Developers can spin up five or six instances quickly, given a good bid on the spot price, and "they're sort of unshackled from the typical, more restrictive dev environment that they live in," Langley said. "That can be really powerful, especially when you're testing things like, 'Does this algorithm I just wrote scale?'"

AWS spot instances can disappear at any moment, but this also gives developers the ability to test the resiliency of their applications as well.

"We use spot instances as our own personal Chaos Monkey, in a way, because you never know when they're going to go away," Langley said. "If you want to prove that your production service is resilient to instance failure, then there's nothing better than a bunch of spot instances, because you're pretty well guaranteed that they're going to go away."

Spot instances also give orchestration and automation tools "quite a workout," Langley added.

Opscode's Chef configuration automation software has proven itself under this system, as has Jenkins for continuous integration testing.

Spot instances save 'gobs' of money

Cabot Research, a Boston-based financial analytics company, often uses spot instances for HPC to crunch numbers, but one software engineer from Cabot also uses a spot instance to run his Java development machine, precisely because it can disappear at any moment.

This promotes good development hygiene in some ways, according to Brian Tarbox, a software engineer for Cabot who spoke at a monthly Boston Amazon Web Services Meetup Group in Watertown, Mass., last week about his use of spot instances.

"If you're using something like Git … you can pull up Eclipse, and you're current," Tarbox said. "It takes me no time at all [to recover]. And knowing that my Eclipse instance might disappear out from under me at any time, I make sure to commit often -- under Git you should be committing often anyway."

What are AWS spot instances?

AWS spot instances are a way to bid for unused capacity on EC2. Customers enter a maximum hourly price they are willing to pay for a particular instance type, region and availability zone; when the spot price for the desired instance type is at or below the price of the initial bid, the customer's application will run.

Additionally, by using spot instances, "you'll save gobs and gobs of money," Tarbox said. He presented a chart that estimated the yearly cost of on-demand instances for Cabot at $7,104; reserved instances could be as low as $1,422 per year if used judiciously. But spot instances, at $0.07 an hour with no up-front fees, would cost $612 annually.

Spot instances are also good for cloud bursting, according to one consultant.

"The classic example is if you're advertising during the Super Bowl," said Mike Ryan, consultant at European firm Epitech. "You don't want to allocate all of the hardware to meet that demand, and then you can mix and match your public and private cloud."

AWS spot instance wish list

One way Amazon could make spot instances better would be to allow conversion between them and the other instance types, on-demand and reserved, Langley said.

"That would be immensely powerful. … If you don't have really good orchestration and automation, it can be prohibitively expensive operationally to use spot," Langley said. "We fall somewhere in the middle with that now. … Even for us it can be challenging, and we'll sometimes make the decision to bite the bullet and go with the more expensive stuff."

Amazon did not comment as of press time.

Beth Pariseau is senior news writer for SearchCloudComputing. Write to her at or follow @PariseauTT on Twitter.

Dig Deeper on AWS instances strategy and setup

Join the conversation


Send me notifications when other members comment.

Please create a username to comment.

Do you or will you use AWS spot instances?
Price is unbeatable and our use case totally works for it. We also have some unexpected use cases that most people would not guess are great for spots (development machines!)
Smart idea (in theory), if you can figure out a way to introduce Chaos Monkey at the lowest cost possible. What I'd be concerned about isthat they would only do the Chaos testing if they thought the instance cost was low enough, and eventually do less testing. 
Isn't this basically what Netflix Pioneered recently?
Spot instances are a perfect way to get access to low-cost instances for non-critical jobs. I could see a team leveraging spot instances for things like the nightly build and saving quite a bit.