Knowing what is going on in your environment is security 101. It applies as much to AWS Cloud environments as it does to any other environment (including SaaS solutions you may use). It’s why it’s the number one pick in our AWS Security Foundations series! Fortunately AWS has a solution, AWS CloudTrail.
For your on-premises environment, do you have a record or every action or request? Does that include every asset or service you have (or may no longer have)? Do you record read only and change actions? Are all users included? Does that include service access and external users? CloudTrail covers all this and more. The first step in any AWS setup should be to create a baseline configuration for CloudTrail.
Sounds great! How do I get started?
The good news is that AWS CloudTrail is enabled by default in your account and it’s free! To get started, log into the AWS Console and go to the CloudTrail service.
In the CloudTrail console, the first thing you will see is an invitation to create a Trail. We will come back to that in a minute as it’s an key step. Fortunately, even without a Trail, CloudTrail is still recording all events in your AWS account and retaining them for 90 days. Click on the burger icon, top left, and select Event History. You will see a bunch of events in the main panel.
If you click on an event you will see details about the action, such as who performed it, from where and what the result was. It’s a very powerful record. Later on you can use this to alert on certain actions or on failed actions, for example. Right now, almost every AWS service logs events to CloudTrail. With the event view you can filter by a bunch of fields such as event type, user ID and event source. You can also set a timeframe in which to search for events.
Without a Trail, you can query events from the last 90 days. But what if you want to go back further or you want to see events across multiple regions? That’s where you need to create a Trail. Note that the CloudTrail service remains free when creating a single Trail (although there is a small S3 storage cost). Additional Trails or enabling extra capabilities do incur extra charges.
Creating a long term audit trail
To create a Trail, click on Trails and then Create trail. Trails are logged to a S3 bucket. You can then query the contents of the audit logs in the S3 bucket.
- Provide the name of the Trail;
- Either create a new S3 bucket name or use an existing bucket. Note that a S3 bucket name needs to be globally unique. We recommend creating a seperate bucket so you can carefully manage access to the event history via bucket policies (more of that in a coming blog post);
- Enable encryption and provide an existing or new KMS key for the encryption;
- Enable Log file validation;
- Click on Next;
- Leave the Choose log events at the default detailed for now and click on Next;
- On the review page, select Create trail.
The Trail will only take a few seconds to be created. Go to the Trails tab and click on the newly created Trail to see the details. You can see that it has been created as a multi-region Trail by default.
Let’s go and have a look at the S3 bucket. Note that it can take up to 15 minutes for log files to be delivered into the bucket.
You will find the log files delivered into the bucket under the following structure:
In our example:
If you open one of the files, you will see a JSON object for each of the events for that time period. That objects mirrors the information seen the in the event viewer.
Monitoring AWS CloudTrail events
Collecting CloudTrail events into an S3 bucket is the first step in keeping an eye on actions in your AWS account. The next step is to actively look at the events that are occurring and alert as required. There are a several ways to do this.
- Use CloudTrail Insights to identify anomalous activity. This requires that you enable logging of insight events on your Trail. You can do that by simply editing the Insight events configuration for your new trail, putting a check on the Insight events box and clicking Save changes. Easy! Additional charge apply when you do so. With Insights you can query unusual behaviour in the CloudTrail console;
- Set up CloudWatch Logs alarms to alert you when certain CloudTrail events occur, such as a console log in by the root user (don’t log in to the console as the root user, except in exceptional circumstances!). You can follow the multi-step process to do this. Note that sending events to CloudWatch will incur standard CloudWatch costs;
- Query the S3 log files in place using Amazon Athena. Athena allows you to use SQL style queries on objects in S3 without have to move the data anywhere else!
- Amazon GuardDuty (more on that in a forthcoming blog) will alert you on certain events reported through CloudTrail;
- Use a third party tool such as Sumo Logic to analyse the event logs and alert on certain events or sequence of events.
What events should I look out for?
At RedBear, we often get asked about CloudTrail events to monitor. Below are the basic events that we recommend to look for as these are deemed as high risk and potentially rare events.
|Changes to CloudTrail such as deletion of a Trail|
|Anomalous level of failed actions from a network location or a user|
|Changes to network ACLs, SecurityGroups and RouteTables|
|Changes to IAM users, groups, policies and role permissions|
|Role assumption from an external account|
|Console login without MFA|
|Console login from outside of Australia|
|Changes to KMS keys such as delete, rotate and policy change|
|S3 bucket policy changes|
|Console login by the root user|
|First use of a service|
|First use of a region|
|Sharing of snapshots or private AMIs|
Getting more advanced
Once you have the data being logged to S3 and the ability to look for or be alerted on events, where should you go next?
- You can add additional monitoring of data events for S3 and for Lambda. For S3, CloudTrail will record details of access to objects in S3 buckets. For Lambda, data events gives you insight into when and by whom your Lambda functions are being invoked. Logging of these data events does incur extra charges;
- Protect your S3 Trail bucket by creating a bucket policy. We will cover this in more detail later in this series;
- If you have multiple accounts, consider consolidating the CloudTrail logs from each account into a central S3 bucket. This will simplify the process of an AWS wide view of CloudTrail events;
- Enable actions for certain events. Once you have notification of events enabled, you can kick off actions to remediate or perform analysis. You can enable this through CloudWatch to call a Lambda function for example. This could be your first step into security automation!
AWS CloudTrail is one of the most fundamental and critical security service that you should actively use in your AWS accounts. It’s very simple to setup. As a minimum, make sure that you are securely logging events to a S3 bucket for long term retention. When you have done that, look at monitoring events and enabling notifications of key events.
If you would like to dive deeper into CloudTrail, please get in contact with us at RedBear. Look out for our next instalment in our AWS Security Foundations series next week!