The re:Invent keynotes for 2019 have wrapped up with Werner Vogels, CTO of Amazon, focussing on the platform they have build for the builders. It was light on announcements but heavy on fascinating insight. You can read about the tsunami of major announcements here.
Werner focussed on how they re-imagined virtualisation. He also covered how they architected serverless compute for scale. AWS has released the Builder’s Library. It’s an amazing resource covering AWS’s unique approach to durability, scalability and recoverability. AWS have released it so that you too can think about how you architect distributed systems. After all, it’s not easy!
Between the keynotes there were some additional announcements which we will cover here.
Containers. So 2019
Containers are hugely popular with builders. On top of the announcement of support for Fargate (serverless) in EKS, there has been some other significant improvements.
- Fargate on ECS can now make use of Spot capacity. Spot is a way to get access to compute at up to 70% discount. It uses spare capacity in an AZ to provide compute. Since Spot is dependent on spare capacity, it can disappear at any time (with a short notice period). If you are running applications that can deal with failures, Spot could be for you. It can significantly slash you compute costs;
- AWS has improved the control of ECS clusters by introducing ECS Cluster Auto Scaling. In conjunction with ECS Capacity Providers, it improves your cluster scaling experience by increasing the speed and reliability of cluster scale-out. It also gives you control over the amount of spare capacity maintained in your cluster. Clusters running multiple workloads or workloads that scale-out rapidly are ideal candidates for Auto Scaling;
- ECS containers now supports AD authentication using Windows group Managed Service Account (gMSA). Now, you can keep user account identity configuration separated from your container images. If you are deploying .NET applications on ECS, you can use gMSA to authentication to applications like Microsoft SQL server without having to embed the credentials.
Lambda is 5 years old!
AWS’s serverless implementation, Lambda, grows day by. More and more builders see the benefits. There is no easier server to manage than no server. RedBear’s managed services platform makes heavy use of Lambda.
One of the challenges with Lambda is keeping the underlying container (the execution environment) warm when you know you are going to run the function regularly. Now, there is a solution in Provisioned Concurrency. This new feature keeps functions initialised so that they can be available in double-digit milliseconds. This avoids the cold start time for a new execution environment. With Provisioned Concurrency, you define the number of execution environments to initialise so they are ready for a function invocation. It’s ideal for highly variable web service back ends or for microservices.
Nitro enclaves
Do you have super sensitive data processing application requirements that can’t be met with existing solutions? If so, Nitro Enclaves may be your friend. Enclaves provides an isolated and highly restricted environment for your most security sensitive applications. Enclaves are virtual machines attached to EC2 instances. They have no persistent storage. The virtual machines can only be accessed through a secure local connectivity from the host EC2 instance. It also includes cryptographic fingerprinting for your application components to guarantee that only authorised software is running. They are well suited to healthcare or financial services system of record.
RDS Proxy
Using a gateway or proxy to manage database connections is a common deployment pattern. Now you can do the same with RDS using the new RDS Proxy.
RDS Proxy allows you to create DB connections pools. Once setup, the calling applications can share these connections. In addition, in the event of a failure, RDS Proxy will seamlessly deal with a failover to a standby database. A further benefit is that your applications never have to deal with connection credentials. The access and credentials can be managed through AWS Secrets Manager and IAM.
Athena federated query
Our final announcement is another data insights based one. This follows on from a heap of Redshift updates announced in Andy Jassy’s keynote on Tuesday.
Athena is a fantastic tool that allows you to query S3 objects in place using a SQL like language. You don’t need to ingest data into structured storage such as a database. It’s proven to be very popular for builders of data lakes.
The new federated query allows Athena to run SQL queries across data stored in relational, non-relational, object and other custom data sources. This includes both on-premises and cloud sources. AWS has connectors for a number of AWS services including DynamoDB and CloudWatch. There are also connectors for JDBC compliant DBs such as MySQL. Athena can use these connectors to run SQL queries across these data sources. Of course, you can build you own connectors using Athena Query Federation SDK. It goes without saying that the connectors run on AWS Lambda, so you have no pipelines or infrastructure to manage!
No more moving data from on-premises or between sources to gain insights.
Builders. Go build!
That might be the key announcements from re:Invent, but don’t expect AWS to be sitting back and having a rest. If past years are anything to go by, there will be more good stuff to absorb over the next few weeks.
We will be back soon with a final wrap once we have digested the crazy week that is AWS re:Invent. See you on the other side!
