Andrea Danti - Fotolia
Cloud providers like AWS make it easy to rapidly provision resources, and that ease in which enterprises spin up resources can result in bad habits. In doing so, cloud users waste deployed capacity and money.
AWS stands to profit from its customers' carelessness, but it also realizes that when customers see the cloud turning into a money pit, they'll abandon the rental model and return to wholly owned and operated infrastructure. It's in Amazon's long-term interest for customers to view its cloud as a valuable commodity with added benefits like security and agility. To help enterprises control costs, it offers the AWS cost-optimization service Trusted Advisor.
Trusted Advisor scans an AWS environment and performs a set of checks against best practices covering four key areas:
- Cost optimization. AWS Trusted Advisor evaluates if a cloud deployment would benefit from using Elastic Compute Cloud (EC2) On-Demand instances versus Reserved Instances. The service also checks for idle databases, load balancers and elastic IPs, as well as underutilized Elastic Block Store (EBS) volumes and Redshift clusters.
- Performance. The service looks for overutilized EC2 instances, too many rules in an EC2 security group, overutilized EBS volumes and various red flags with Amazon CloudFront content optimizations.
- Security audits. Trusted Advisor checks for potential vulnerabilities in a deployment, including security groups that have unrestricted access or ports and Simple Storage Service (S3) bucket permissions that are not strict enough. The service looks for other access weak points, such as lax AWS Identity and Access Management password and access key rotation policies, as well as use of multifactor authentication on root or other administrative accounts.
- Fault tolerance. This searches for common areas to increase application and data availability, redundancy and security. Such areas include the use of EBS snapshots, redundant configurations for virtual private network tunnels and Elastic Load Balancing (ELB), multizone configurations for Amazon Relational Database Service and Amazon Aurora, and object versioning on S3 buckets.
Trusted Advisor presents results in a color-coded dashboard format, indicating areas to change in red and areas that require further investigation in yellow. Green indicates there are no problems.
Check costs on AWS Trusted Advisor
Improperly configured cloud implementations aren't cost-effective; Trusted Advisor can help enterprises that are heavily invested in AWS save money while ensuring deployments remain secure and resilient. In a case study, AWS claims that Hungama Digital Media Entertainment cut its AWS spending by 33% per month after implementing Trusted Advisor recommendations. The company, based in Mumbai, India, eliminated underused EBS volumes, limited Reserved Instances to only those with consistently heavy workloads and improved instance use with smaller EC2 instances.
Trusted Advisor isn't an entirely free service. All AWS users can access four free -- but limited -- features that cover a subset of security and performance checks. Access to the entire Trusted Advisor suite requires a Business or Enterprise support plan. The former plan includes essential support services and costs at least $100 per month; it could cost more, depending on a business' monthly AWS costs. Enterprise plans, which include a dedicated account manager and infrastructure reviews by AWS Solution Architects, start at $15,000 per month and go up from there.
If you decide Trusted Advisor is worth the cost, adding checks to Amazon CloudWatch events to automate them can help.
Spend more now to pay less later
There are many third-party alternatives that automatically analyze cloud infrastructure for ways to tune performance and improve AWS cost optimization. RightScale originated as a multicloud management product for hybrid cloud users, but it also offers a dedicated cost optimization service called Optima that has a subset of Trusted Advisor features. Other options include ActOnCloud, Cloudability, Cloudamize, CloudCheckr, Cloudyn and CloudHealth Technologies; these tools help AWS buyers get a handle on resource use, costs and configurations. The utilities typically include automated policies and tasks to terminate unused volumes and instances, schedule shutdowns and notify admins of budget overages.
Other products bridge the gap between fully packaged products and customizable tools for quasi-DIY management. For example, GorillaStack offers services that provide scheduled instance cycling, scaling options, scheduled EBS volumes and billing alerts. It also provides an auto-tagging feature that admins can use to categorize AWS resources by project, business unit or application. These features integrate with Slack to simplify the DevOps workflow and enable IT teams to build a chat interface to define and manage CloudTrail workflows.
Geckoboard enables ops teams to build custom dashboards with more than 60 integrations to a variety of external cloud services, including Amazon CloudWatch. The service handles all the API dirty work, giving nonspecialists the ability to create visualizations that mix data from many sources. Geckoboard also provides a DevOps dashboard, making it useful for more than just infrastructure monitoring.
But none of these products are particularly cheap. For example, the Pro plan on CloudCheckr costs nearly $800 for those with $10,000 in monthly AWS spending; both GorillaStack and Geckoboard cost about $200 monthly for a small AWS installation. For that reason, many businesses go the DIY route for AWS resource monitoring and cost management.
If you build it, savings will come
AWS has helped large users create dashboards that track spending, EC2 use and savings from changes based on CloudWatch data. The cloud provider offers the foundation for custom dashboards using CloudWatch, which can monitor resource use for almost any Amazon service.
AWS also publishes a price list API that provides data in JSON and CSV, and integrates with Amazon Simple Notification Service to trigger notifications when prices change. Ops teams can load the data into a spreadsheet and combine it with a billing report to build a cost model.
The cloud provider also instructs enterprise IT on how to create a more sophisticated AWS cost optimization monitor using data from AWS billing reports. Ops can load data into Elasticsearch, where it uses Kibana for data visualization. Although AWS provides a CloudFormation template to build the infrastructure, it still requires customization. And the AWS infrastructure that runs the stack isn't free.
Pay up or partner up
If the DIY route seems a bit overwhelming but you're not sure that AWS Trusted Advisor is worth the support subscription, an AWS consulting partner can evaluate your needs and help build tools. Ops teams can assess the environment and bootstrap the AWS cost optimization exercise:
- Identify always-on instances. Are they always active and what is their average utilization? Then identify instances to downsize.
- Implement storage lifecycle management. Can you move certain data to a cheaper storage tier, such as a cold EBS instance, S3 Infrequent Access or Amazon Glacier?
- Evaluate instance options. How many workloads are in a steady state? Can you reduce costs if these are changed to Reserved Instances? And what, if anything, is keeping you from using Reserved Instances?
- How do you handle workload elasticity and scaling? Instead of overprovisioning EC2 instances, use Auto Scaling, ELB, CloudFront and Elastic Beanstalk for web apps.
After assessing resources, ops can identify cost targets and metrics for availability and performance to build a cost model. The next step is to develop a consistent, repetitive process that collects AWS cost data and loads it into a spreadsheet or program that crunches the numbers. Ideally, ops should feed the results into a dashboard to summarize data, visualize trends and document the cost savings of infrastructure optimization made in response to the information.
Save money in AWS without an IT strategy overhaul
DevOps, monitoring tools can help smooth app delivery
Avoid monthly surprises with cloud cost analysis tools