IT pros who battle complex AWS billing each month have found third-party cloud cost analysis tools crucial to keep...
the financial planning process under control.
Tools from emerging companies such as CloudHealth Technologies and Cloudyn Ltd. help customers cut down on waste, identify spending patterns and create financial forecasts.
"It gives me a tool that allows me to … have really healthy conversations with our ops team, for potential cost savings or just understanding why things are increasing," said Cameron Lee, financial analyst at Instructure Inc., a Salt Lake City, Utah-based educational technology company that makes extensive use of AWS infrastructure and services. There isn't an AWS Service the company doesn't use, and at peak times it runs as many as 1,500 instances on Elastic Compute Cloud (EC2).
"There are some things I'll use Amazon for, just to get a quick snapshot of the Amazon bill and how it's laid out … but if I really want to dig in and see how are things trending in a specific region, or a specific part of our environment, then CloudHealth is so much easier to use," Lee said. "With Amazon, to get what I really wanted there'd be a lot of manual data input [in] Excel."
The CloudHealth tool has also been helpful in recommending an upgrade from first-generation instances to the newer C3 line of instances; recommending when to buy Reserved Instances; and identifying instances that weren't tagged properly. Lee also can slice and dice AWS billing statements according to regions, users, roles and applications, as opposed to by instance or by service in Amazon.
Instructure execs want CloudHealth to have more data visualization and forecasting options, according to Wade Billings, director of DevOps for the company.
AWS partner sorts through customer bills with Cloudyn
Mirabeau, the largest full-service Internet company in the Netherlands and an AWS Advanced Consulting Partner, uses CloudHealth's competitor Cloudyn. Mirabeau has some 70 large European firms as customers to whom it resells AWS Services; the company also hosts its own website on AWS.
Mirabeau found the native AWS product, Trusted Advisor, to be prohibitively expensive to use, despite having many of the same features as CloudHealth and Cloudyn, including identifying cost savings opportunities and making configuration change recommendations based on best practices.
AWS makes up to four Trusted Advisor checks available to customers free, but to get the full-fledged tool with 37 different checks, customers must purchase business-level support from AWS, which Fons Van Geelan, IT director at Mirabeau, said would cost between $11,000 and $12,000 per month.
"The business support is at least $100 per account and 10% of revenue, and that makes it very expensive," Von Geelan said.
Big data analytics firm consolidates AWS billing accounts
Trusted Advisor was also a no-go for Redwood Shores, Calif.-based AutoGrid Systems. The company makes software to analyze energy information, has multiple AWS accounts for each of its energy industry customers and would have to buy business support for each to get the full-fledged checks from Trusted Advisor.
Keeping track of all those separate AWS bills on a daily and weekly basis, as well as the monthly schedule AWS release bills, also was difficult enough to prompt AutoGrid to adopt Cloudyn's software.
"If the cost across all of our Amazon accounts exceeds a certain threshold, our ops group gets an email," said Rajeev Singh, VP of engineering and chief software architect for AutoGrid. "So if some engineering group fires up a cluster and leaves it running, all of a sudden the costs will go up, and we'll find out within a day."
Cloudyn's tool, however, isn't perfect.
Both Von Geelan and Singh said they'd seen some discrepancies between Cloudyn's numbers and Amazon's. Von Geelan said the problem in his case had been corrected, and Singh said the discrepancy from his point of view was negligible -- less than 10%. But these differences keep both firms using AWS billing alongside Cloudyn's tool to get a fully accurate picture of cloud costs.
"It’s a good tool for day-to-day, week-to-week cost management," Singh said. "But Amazon provides the better monthly report."
There are several reasons the Amazon's number may differ from the tool's, a Cloudyn spokesperson said.
Cloudyn gets its data from Amazon billing files. At the end of each month, Amazon pauses the generation of the billing files until about the second or third of the next month, but the billing console keeps updating, the company said. During that time, Cloudyn generates a cost estimation, and the difference during these days can be significant. When the billing files are made available, Cloudyn adjusts its numbers to match Amazon.
Cloudyn may also take into account amortized costs paid in the past; the software also has different data display policies, one of which differs significantly from Amazon due to its handling of reserved instance costs.
Cloudyn offers the lite version of its product for free and a premium version for $229 a month. An enterprise version is also available at a custom price. CloudHealth is priced at 2%of the prior month’s Amazon bill.