ra2 studio - Fotolia


Police your public cloud with AWS CloudTrail

Many AWS customers are unaware of the vast security options AWS CloudTrail provides. Use this native tool to meet your security and compliance needs.

Enterprises know it is crucial to keep a close eye on resources, including who has access to what. Many Amazon Web Services customers, however, may be unaware of all the security capabilities that AWS CloudTrail offers, including forensics records and policy compliance audits about activities within their AWS cloud environment.

CloudTrail records API calls and monitors access to the Amazon Web Services (AWS) Management Console, command-line interface (CLI) use and programmatic access to other Amazon services. CloudTrail provides key input for security audits, recording administrator activity such as policy changes on an S3 bucket, starts and stops on Amazon Elastic Compute Cloud instances and any changes to user groups or roles.

AWS CloudTrail records data to an S3 bucket in JavaScript Object Notation (JSON) format to facilitate parsing, filtering and data analysis. Then the service can trigger alerts via the Amazon Simple Notification Service (SNS). The SNS is accessed via custom applications using APIs and can feed other logging and operational analysis systems like AWS CloudWatch as well as third-party tools from Alert Logic, Loggly and Splunk.

CloudTrail workflow

CloudTrail is configured using the AWS Management Console or CLI. Configuring the service primarily entails specifying an S3 bucket to store the logs -- by default the user interface will create a new bucket but you can select an existing one. Once enabled, CloudTrail starts recording events that can be viewed in the AWS Management Console and programmatically queried using the LookupEvents API. Users can create an SNS topic that receives notifications when a new log file has arrived.

CloudTrail stores log files in a gzip archive using a standard, hierarchical naming scheme organized by day, making it easy to pull entries individually or for specific time periods. Users retrieve log entries using any S3 access method: the management console, CLI or API.

Entries are written in JSON format to simplify post-processing; admins can also view entries directly in the browser via an add-on extension like JSON View. JSON format allows third-party log analysis tools to aggregate, parse and analyze CloudTrail data.

Configuring AWS CloudTrail with Amazon SNS allows users to subscribe to a particular log and receive notifications when it's updated. However, topic subscriptions are still managed through the SNS console or API. Some log files can be quite active, leading to a large number of messages. AWS CloudTrail documentation recommends using Amazon SQS to handle these notifications programmatically.

Permissions and access controls

AWS Identity and Access Management (IAM) maintains access to logs and other CloudTrail resources, such as SNS topics, S3 buckets and message queues. IAM grants administrators control over who can create, configure or delete CloudTrail entries and who can start and stop logging and access the buckets that contain log information.

The IAM policy generator provides an easy interface for creating and editing CloudTrail permissions, including templates for full and read-only access. With IAM, it's a best practice to first create IAM groups like "Administrators" and "Viewers," then add users to the appropriate group. You can also create custom policies using the IAM JSON syntax. For example, an administrator might allow users to read CloudTrail logs and objects in an S3 bucket, but not allow them to create, update or delete those logs.

Using CloudTrail with other services

CloudTrail's standard log format and API means it can feed third-party log analysis tools or a custom application. For example, admins can use CloudTrail, AWS Lambda and SNS to generate email notifications when certain APIs in the AWS infrastructure are used. In this scenario, Lambda watches the CloudTrail S3 bucket and triggers an SNS notification when specified APIs are logged. SNS then sends a message to every topic subscriber via email, SMS or mobile push.

Popular log management and analysis products can also combine CloudTrail logs with data from other AWS offerings, such as Config or OpsWorks, as well as on-premises infrastructures, to produce comprehensive usage and security reports. Tracking changes across services and infrastructure allows a product like DataDog to correlate change events with performance metrics to help identify the cause of any degradation and highlight the source of any security incidents.

CloudTrail works with every major AWS offering and regularly adds support for new products. AWS only charges for S3 storage with CloudTrail, which the company estimates to be less than $3 per account for most customers.

Next Steps

Using DevOps to improve AWS security

Stay one step ahead of AWS cloud security issues

Think like a hacker with security threat modeling

Dig Deeper on AWS security