Today we’ll be talking about one of AWS’s more recent addition to the EC2 landscape, the AWS Resource Data Sync. What it is, what does it do and why you should care?. Get ready for some terrible sinking puns. Glub, glub, glub!
Time is a real drain…
With COVID and the end of 2020, it seems more and more of our day is eaten up behind screens. With the rise of working from home, working time can bleed into personal time. As IT professionals, it can be tough to separate the two.
One of the keys to successful security management is, of course, monitoring and observability. However, it can be difficult to keep on top of your environments and their inventory of resources. Digging into logs or leafing through emails is a thing of the past. Compiling our data into clear metrics is critical to managing at scale. The first step in achieving this is the aggregation and collection of data.
Enter the AWS Resource Data Sync. Resource date sync collects and aggregates a whole lot of data for your fleets of EC2 instances conveniently into one place.
More than just the kitchen sync
One of the great things about Resource data sync is the huge range of data it gathers. The sync includes pretty much everything you might need to know about your instances.
This includes patching info (such as hot fixes for windows), granular information about the applications and software packages installed, billing info, compliance data and instance metadata. It also collects detailed network interface data, resource groups, windows roles and even the processes and services running on each instance.
With this pretty comprehensive list of data, the keen folks in operations can know just about everything there is to know about your precious little servers as they tick along with a quick glance at the at the resource data sync. This is gold for data enrichment when it comes to qualifying potential security incidents.
Data sync provides you, and anyone else in your organisation, with a very wide array of data. It might be that a particularly vicious strain of malware is on the loose. Depending on your SIEM of choice, aggregating the data on exactly which instances have which KB updates installed is just a query away. This can take a huge burden off IT managers, who need to ensure that their organisations are current with software updates and immediately patch any known vulnerabilities.
Perhaps someone has been using instances for things they shouldn’t. Surreptitiously mining bitcoin on those spot instances when they should be shut down. Data sync can tell you exactly what applications are installed on current and past instances.
Am I compliant?
These features make data sync ideal for all things compliance related. You can rest easy knowing your fleet is patched, secure and up to date. The versioning options of S3 also mean there is a preserved bread crumb audit trail in the event of a breach or other security incident. Its low cost means that the storage costs are negligible and there is little need for leveraging S3’s tiered storage model and moving data out to Amazon Glacier.
All drains lead to one place…
And it’s not the sewer! Although it does start with S. Resource Data Sync aggregates this data into S3. It supports multiple regions. Cross account support for Organizations comes out of the box and it is CloudFormation ready. However cross-account setup will take a little more configuration.
All the data on all your instances, across multiple regions, for the whole organisation in one place! This ain’t just for the folks in operations. There’s something tasty in this pie for everyone – security folks, billing, compliance. It’s all there and ready to go.
The resource syncs can be viewed directly in the Systems Manager console. With a couple of clicks they can be visualised in AWS Quicksight. For the more technical, they can be sliced and diced with Athena – or pushed out to a SIEM of your choice.
How to Sync?
As mentioned above, the service is support by CloudFormation. There are a few permissions required on the bucket side but for the most part setup is pretty straightforward, including setting up a cross-account role.
However there are a couple of tripwires for the unwary.
- A CloudFormation stack failure in any capacity can result in a tangled web of failure-to-delete’s if your bucket is included in your template solution;
- At the time of writing, there are also some known issues with regions, such as this example.
Once you are up and running, you can even setup notification via Amazon EventBridge for resource inventory state changes.
As always, your instances will require that the SSM agent is installed for the solution to work. All the AWS standard images for the last few years already have the agent installed. You will also need an instance role with the appropriate permissions for SSM.
That’s it!
If you would like to understand more about AWS Resource Data Sync, please get in contact with us at RedBear.
Happy syncing!
