Guido Vrola - Fotolia

AWS cloud cost management: The buck stops with you

Getting stakeholders to help manage AWS costs shouldn't be like herding cats. Create a cost-conscious culture and follow these steps to control cloud costs.

NEW YORK - Cloud is no longer just one person's responsibility.

Several business units -- engineering teams, operations, finance, marketing and executives -- are all involved in decisions on cloud use and spending. And enterprises with large-scale development teams who deploy instances to the public cloud make micro-decisions on a daily basis that can greatly affect the company's total spending, as well as overall capacity and efficiency of the system.

The key is to make cost management a cultural change and ensure all teams are accountable for cloud expenditure.

"It's difficult to dictate a policy or practice without infusing it culturally across the entire organization," said Toban Zolman, vice president of product development at Cloudability, Inc., during a cost management session at the recent AWS Summit here.

"The more you can tighten that feedback loop to individuals responsible for actually driving costs, the more you can create a change in behavior," Zolman said.

How do you build this culture in an organization? The first step is to put data in the hands of the people. Make sure everyone with the organization who has some cloud responsibility understands exactly what is happening with cloud costs and how they can control them.

Next, enact policies and evangelize best practices. Having a clear approach to how IT teams tag instances, how resources are deployed and what policies are tied to the resources is imperative, Tobin noted. Then evangelize best practices around those criteria so everyone understands what levers they can pull individually in Amazon Web Services (AWS) to reduce costs.

Giving stakeholders daily visibility into costs creates a cadence of consumption.
Toban ZolmanVice president of product development at Cloudability


Finally, incentivize good spending behavior and habits through gamification and competition. According to Tobin, recognizing good behavior or efficiency when it comes to cloud is something that can help dramatically change how people act. Letting one group see another group's data and spending can help teams learn from each other.

Five stages of cost management

Once an enterprise is moving in the right direction toward creating a cost-focused culture when it comes to AWS cloud use, it's time follow these five tactics to manage costs throughout the budget cycle.

1. Create visibility into cloud costs.
Make sure everyone within the organization, regardless of their role, understands what is happening with its AWS expenditures. Organizations typically have two types of people: those who incur costs and those who watch costs. In a traditional data center structure, very few people paid attention to who was spending what after the infrastructure was purchased. But cloud is different. The more resources groups consume, the more the costs continue to increase.

Give everyone a role as a cost watcher, Tobin said. Having more individuals watching costs and understanding why costs increase or decrease changes attitudes and leads to better cost management.

"Giving stakeholders daily visibility into costs creates a cadence of consumption," he said. "Let each team understand how other teams are saving money, what their unit costs are and what their growth curve on the cloud looks like." 

2. Focus on allocating costs so you know who is responsible for what in the AWS bill. Tobin suggests splitting the bill into various cost centers. As the AWS bill grows, this becomes an increasingly challenging task, especially if you have a large organization with a lot of applications and many cost centers.

The key with allocation is to get everyone who incurs costs in the cloud involved in understanding how to allocate those costs back out. For example, have a master account mapped to a business unit. Within that master account, each account represents a product line. Then use tags at a resource level to capture what you want to use as dimensions across business units or across products.

Then get a consensus on taxonomy.  What type of hierarchy will the organization use for cost centers?  Tobin suggests letting someone in the finance department draw up a hierarchy showing how AWS charges will be allocated.

3. Promote efficient behavior to drive down the AWS bill. The first step here is to make sure the organization is as efficient as possible. With cloud, there is a lot of low-hanging fruit that companies can go after to increase efficiency in the infrastructure.

Dig into wasted cloud instances and turn off underutilized instances. "Don't run the cloud like a data center," Tobin said. "Sixty-five percent of the hours in a month are nights and weekends when, most likely, instances that are used for development or test aren't going to be touched."

Using cloud management tools can help pinpoint instances that have low CPUs, low bandwidth and low disk I/O. In most cases, admins can turn off those instances or put them on a watch list to determine use. Resizing instances and properly managing legacy instances can also drive down costs, he added.

VideoBlocks, an online site that provides downloads to stock video, is starting to evaluate third-party tools to manage AWS costs.

"We have been on AWS for about a year and a half now," said Rocky Gray, senior software engineer at the company. "We are currently using the AWS console and intuition to manage our costs; however, we aren't as efficient as we should be."

The company started with five to 10 instances in AWS and now has about 30 instances running full-time -- with 100 in test and development. "We need something to help manage those development instances and costs," he added.

4. Implement cost-savings methods. Once the organization is running AWS instances as efficiently as possible, it's time to look at how to drive down costs. Two ways to do this in AWS are through Spot Instances and Reserved Instances (RIs), Tobin said. Reserved Instances are an easy way to lower costs without any engineering effort, he added. But it's important to analyze current and future usage carefully before buying RIs, otherwise you'll waste more money than you save.

"Paying two months of On-Demand [instances] and then buying an RI is costly," Tobin said. "Do lots of rapid, small [RI] buys and then iterate through it."

5. Focus on unit cost, not overall cost to understand savings. Once more teams are looking at the top line of the budget numbers, they may notice the bill is beginning to increase. That doesn't mean you're losing money.

"The bill could be going up because you're tripling users and you're scaling as you need it," Tobin said. Focusing on unit cost helps normalize the increase and helps determine what's correct. Unit cost is cost per X, which could be cost per subscriber, cost per user, or cost per API. This is an effective way to normalize the organization's actual spend.

Finding Zen in a balanced cloud budget

When all of these pieces are in place, companies can begin to reap the rewards. No one is shocked when the AWS bill arrives -- and IT teams can control growth at a reasonable rate.

In addition, everyone is on the same page. Creating transparency around costs and any associated anomalies eliminates fire drills, questions and reactive justifications, Tobin said.

It also makes it easier to budget for cloud in the future. Even though companies on public cloud longer deal with yearly infrastructure purchases, finance teams and business owners still need some understanding of what the spend will be for the quarter or fiscal year. The more transparency you can create -- with a clearer path around containment and cost savings -- the easier it is to budget.

Michelle Boisvert is the executive site editor for Contact her at [email protected].

Dig Deeper on AWS pricing, cost and ROI