Infraception with CDK Pipelines
Are you ready to be amazed with some magic?
Well, I am not a magician, per se, but as Arthur Clarke says,
“Any sufficiently advanced technology is indistinguishable from magic”
I totally agree.
One of my favorite movies, The Prestige states that
“Every great magic trick consists of three parts or acts”
let us try incorporating these parts into our story.
The Pledge: (This is the part where I walk through the problem statement: Something ordinary)
We previously had around 50 static websites hosted on EC2+ELB, this was a bit cost intensive, and this architecture could be improved upon with some effort.
That is when I challenged myself to re-architect this setup so that it is secure, reliable, supports automated deployments and saves us some costs as compared to the existing Architecture.
The Turn: (This is the part where I briefly describe my approach and convert the ordinary problem statement into an extraordinary solution)
In this stage we identified the tasks to be worked on for moving from current architecture to the desired state.
We have chosen S3 Static Website connected to CloudFront Distribution for each website/partner added to the Monorepo for this project.
This is achieved using AWS CDK (Cloud Development Kit) to create CloudFormation Stacks such that every new website gets unique resources such as:
S3 Bucket
CloudFront Distribution
ACM Certificate
For automating deployments, i.e., Bucket level deployments to these sites based on changes, we have set up a CodeBuild Project which runs the steps mentioned in buildspec.yml file of the Mono repo
When any push activity/SHA is returned via Webhook to CodeBuild, it triggers the above Build Project, and deploys the changesets to CloudFormation via CDK deploy
301 Redirection support is implemented via a Lambda@Edge Function shared across stacks, to avoid maintenance hassles and multiple copies of same Lambda code.
The Prestige: (If you are still reading, your hypothetical EC2 servers and Load balancers have disappeared, and now you have corresponding websites powered by S3 and CloudFront, secured using ACM SSL Certificates.)
Highlights of this solution:
1. Infrastructure as Code/Infrastructure is Code: As the entire project along with the code necessary to spin up necessary infrastructure resides in the same repository
2. Adding a new site to the repository automatically creates corresponding resources on the Cloud as the CDK Pipeline detects any changes to the repository
3. Stack is CI-CD ready; we can add CI steps easily into the CDK pipeline. Currently it only checks/validates CDK syntax.
4. *Cost Savings compared to EC2 and ELB based hosting.
In less than 500 lines of code, I have provisioned more than 200 distinct cloud resources in two AWS accounts which include but not limited to (1 S3 Bucket, 1 Lambda Edge, 1 CloudFront Distribution, 1 SSL Certificate, 1 S3 Logging Bucket) * 50 Stacks * 2 Environments (Stage & Prod)
As seen above, CDK when combined with CDK Pipelines, can be a beast as it can repeatably deploy/provision thousands of similar stacks with customized Stack Parameters, across multiple environments, and multiple AWS Accounts.
This phenomenon is what I call Infraception (Infrastructure + Inception)
*Pricing comparison:
EC2+ELB estimated costs (Traditional Stack):
~ 200$/mo.
S3+CloudFront (Serverless Stack):
~ 25-30$/mo.
Over 85% cost reduction if moved from EC2+ELB to the proposed solution.
Hope that this piece has shed some light on the optimizations you can use to transform your cloud landscape, by using CDK Pipelines as a CI-CD tool for deploying cloud resources at scale.
Further Reading:
