ra2 studio - Fotolia

What Amazon CloudWatch Logs can and can't do

There are a lot of tools available to manage AWS resources, including the native CloudWatch Logs. So, what exactly does it do and is this monitoring tool right for your enterprise?

I am looking for one tool to monitor my AWS cloud services. What is Amazon CloudWatch Logs and how does it wor...


Amazon CloudWatch monitors many AWS Services such as Elastic Compute Cloud (EC2), Relational Database Service, Elastic Block Storage and several others. CloudWatch alerts can identify events such as CPU spikes and latency issues. When used with AWS AutoScaling, administrators can seamlessly increase EC2 capacity as needed.

But since its inception, Amazon CloudWatch has had some limitations. One issue was that it lacked the ability to process huge amounts of custom log data that cloud platforms create, and to react to events in near real time. CloudWatch Logs, a new feature introduced at the AWS Summit in New York last month, is an extension of CloudWatch that admins can use to detect patterns and take action to resolve issues that arise.

With CloudWatch Logs, admins can capture any type of log -- from applications, on networks, from devices and so on -- and process it in Amazon Kinesis. From there, admins can then monitor application health.

But CloudWatch Logs can’t do everything.

Logs provide an incredible window into problems that affect a cloud platform and act as a business intelligence resource. Many vendors, such as Loggly and Logentries, have carved out niches, offering tools that interpret AWS logs. These companies don't appear too threatened by CloudWatch Logs, as their services are already several generations ahead in their ability to process log data, provide intelligent alerting and help AWS customers identify trends.

With CloudWatch Logs, admins can analyze system logs to search for malicious login attempts over Secure Shell or pinpoint a botnet attempting a denial-of-service attack on the Web server. The monitoring tool looks for recurring IP patterns from your Web server logs and blocks access from attacking IP addresses. You also can query applications logs to identify customer usage patterns to improve the customer experience.

CloudWatch Logs can identify events that are unique to your specific platform -- not just the canned metrics that CloudWatch delivers. There are operating system logs, application logs, Web server logs or logs from any type of endpoint device.

And setting up CloudWatch Logs is fairly straightforward. Simply install and configure a CloudWatch Logs agent on an EC2 instance and access results through the Amazon CloudWatch console. Costs associated with Cloudwatch Logs are $0.50 per 1 GB of ingested log data per day, plus $0.03 per GB archived per month, in addition to the cost of Amazon Kinesis.

About the author:
Russ Vanderpool, MSCS and MBA, is a technologist interested in using cloud technology to deliver solutions, help companies better serve customers and to identify new businesses. He has hands-on experience as an architect/developer and a business adviser across the finance, energy, education, technology and nonprofit sectors. Russ has architected and built a cloud infrastructure for a green tech company and while working for Japan's largest system's integration firm, he developed proprietary object-oriented database visualization software for that market.


Dig Deeper on AWS CloudWatch and application performance monitoring