The AWS Cloud Development Kit (AWS CDK) allows you to describe an infrastructure using the programming languages TypeScript, JavaScript, Python, Java, C#, or Go.
Under the hood, CDK creates a CloudFormation stack with the resources described in your code.
The answer to the question “Our CDK, when is Terraform?” can be found here – 4 ultimate reasons to prefer AWS CDK over Terraform.
But since I haven’t used CDK yet, I won’t say anything about the advantages and disadvantages.
The only thing that you can pay attention to now is that, first of all, there are no state files like in Terraform, which, of course, are useful, but add a little pain to management. The second thing is that CloudFortaion itself has its own shortcomings and “this is not a bug, but a feature”, but we have the opportunity to see all resources in the AWS Console web interface.
In general, I came to AWS CDK because it is already being used on the new project, so before taking Terraform to the project, need to check with what is already there.
UPD: Well, no, I’ll say it anyway. It looks like it’s not bad and interesting, because “Yahho, finally Python!”, but:
- after all, the HCL code looks much more concise and understandable
- there are many more examples and banal Google search results for Terraform, which means that the speed of finishing an IaC task is faster
- well… I got to the point where I had to create an SES domain, and… And nothing. Very unexpectedly, nothing really found in the standard Constructcs, and the only more or less construct from Construct Hub had examples only for TypeScript, even in PyDoc. Such a pleasure, to be honest
Besides the CDK for AWS itself, there are also cdk8s for Kubernetes and CDKTF for Terraform.
Okay, let’s go to see the AWS CDK.
Contents
Key concepts
You can start with the AWS documentation Getting started with the AWS CDK, or watch a good 20-minute video tutorial Getting Started with AWS CDK and Python.
So, the main concepts we will use when working with AWS CDK:
- App: the App is a kind of “container” where we describe our application, and can contain one or more Stacks (which will then be formed in CloudFormation Stacks). See Apps.
- Stack: CloudFormation Stack or change set will be formed from a Stack. In the Stack itself, at the code level, we describe exactly which resources in this stack will be created, and add describe resources using Constructs. See Stacks.
- Construct The basic “building blocks” in the CDK that describe the components that need to be built in AWS. See Construct.
Let’s dwell on Construct in a little more detail, because they are divided into three main groups:
- AWS CloudFormation-only or L1 (“layer 1”): Here, we have resources that are described and supported by CloudFormation itself, and all such resources have names prefixed with
Cfn
, for example, for AWS S3 buckets there is theCfnBucket
. All these resources are in the moduleaws-cdk-lib
. - Curated or L2: These constructs were developed by the AWS CDK team to simplify infrastructure management. Typically, these will include the L1 resources with some default values and security policies. Resources are in the
aws-cdk-lib
and ready for use in production, but if a resource is a separate module, then it is either still under development or experimental. - Patterns or L3: Patterns include several resources that allow you to build the entire architecture for a specific use case. As with L2 resources, production-ready modules are included to the module
aws-cdk-lib
, and those that are under development are in separate modules.
In addition to the AWS Construct Library, there is also the Construct Hub, where you can find modules from AWS partners.
Installing the AWS CDK
To work with the AWS CDK, even if you will write in Python, you need Node.js, since all programming languages on the CDK will work through the Node.js backend.
To work with CDK, we have a CLI through which we can create new Apps, generate CloudFormation templates, perform a diff between our code and existing CloudFormation stacks, and much more.
Install the CDK itself – backend and CLI:
[simterm]
$ npm install -g aws-cdk added 1 package, and audited 2 packages in 1s
[/simterm]
Check the CLI:
[simterm]
$ cdk --help Usage: cdk -a <cdk-app> COMMAND Commands: cdk list [STACKS..] Lists all stacks in the app [aliases: ls] cdk synthesize [STACKS..] Synthesizes and prints the CloudFormation template for this stack [aliases: synth] cdk bootstrap [ENVIRONMENTS..] Deploys the CDK toolkit stack into an AWS environment cdk deploy [STACKS..] Deploys the stack(s) named STACKS into your AWS account ...
[/simterm]
For the sake of curiosity – where does the file cdk
itself leads:
[simterm]
$ which cdk /home/setevoy/.nvm/versions/node/v16.18.0/bin/cdk
[/simterm]
Yup, Node.js.
Authentication with AWS
Documentation – Authentication and access:
- AWS SSO: Using an SSO session via the AWS CLI config (
~/aws/config
), see IAM Identity Center authentication - AWS EC2 Instance IAM Role: if the CDK code will run within the AWS environment, for example from EC2, then IAM roles for Amazon EC2 can be connected to the instance
- AWS IAM User Access/Secret keys: well, the usual access keys for AWS CLI with a profile in files
~/aws/config
and~/.aws/credentials
Creating a project
cdk init
With the cdk init
we can generate a template of files and directories.
To get help on a specific command, for example for init
– add --help
after it, that is cdk init --help
:
Among the interesting options here can be the following:
--verbose
: more detailed output--debug
: even more detailed--role-arn
: use IAM Role--language
: the programming language that will be used when creating a project--list
: get a list of available templates
Let’s try list
:
[simterm]
$ cdk init --list Available templates: * app: Template for a CDK Application └─ cdk init app --language=[csharp|fsharp|go|java|javascript|python|typescript] * lib: Template for a CDK Construct Library └─ cdk init lib --language=typescript * sample-app: Example CDK Application with some constructs └─ cdk init sample-app --language=[csharp|fsharp|go|java|javascript|python|typescript]
[/simterm]
Here, we can create a template for the App, a library for the Construct Library, or create a sample-app
, that is an example App with some Constructcs already included.
Create the directory of our project:
[simterm]
$ mkdir cdk-example && cd cdk-example
[/simterm]
Run init sample-app
:
[simterm]
$ cdk init sample-app --language=python Applying project template sample-app for python ... Initializing a new git repository... hint: Using 'master' as the name for the initial branch. This default branch name hint: is subject to change. To configure the initial branch name to use in all hint: of your new repositories, which will suppress this warning, call: hint: hint: git config --global init.defaultBranch <name> hint: hint: Names commonly chosen instead of 'master' are 'main', 'trunk' and hint: 'development'. The just-created branch can be renamed via this command: hint: hint: git branch -m <name> Please run 'python3 -m venv .venv'! Executing Creating virtualenv... ✅ All done!
[/simterm]
Let’s look at the structure of the files and directories of the project:
[simterm]
$ tree . . |-- README.md |-- app.py |-- cdk.json |-- cdk_example | |-- __init__.py | `-- cdk_example_stack.py |-- requirements-dev.txt |-- requirements.txt |-- source.bat `-- tests |-- __init__.py `-- unit |-- __init__.py `-- test_cdk_example_stack.py 4 directories, 11 files
[/simterm]
Python virtualenv and installing AWS CDK modules
source.bat
– a script for Windows to create a Python virtualenv that should call the file .venv\Scripts\activate.bat
:
[simterm]
$ tail -1 source.bat .venv\Scripts\activate.bat
[/simterm]
But since I’m doing it on Linux, I haven’t the .venv\Scripts
directory at all, instead I have a set of scripts in .venv/bin/
(well, it also looks somehow… AWS CDK seems like a serious project, but such a small thing as scripts were done with some… carelessness? ):
[simterm]
$ ls -1a .venv/bin/ . .. Activate.ps1 activate activate.csh activate.fish pip pip3 pip3.11 python python3 python3.11
[/simterm]
For Linux, we will use the .venv/bin/activate
script, which is a set of shell commands for creating and setting environment variables:
[simterm]
$ . .venv/bin/activate (.venv)
[/simterm]
To check that we are really in a virtualenv, you can check the value of the variable $VIRTUAL_ENV
, which has the path to the directory of the current virtual environment where the installed libraries will be:
[simterm]
$ echo $VIRTUAL_ENV /home/setevoy/Scripts/AWS_CDK/.venv
[/simterm]
Next, install dependencies – modules aws-cdk-lib
and constructs
:
[simterm]
$ pip install -r requirements.txt ... Collecting aws-cdk-lib==2.78.0 Using cached aws_cdk_lib-2.78.0-py3-none-any.whl (41.0 MB) Collecting constructs<11.0.0,>=10.0.0 Using cached constructs-10.2.18-py3-none-any.whl (58 kB) ...
[/simterm]
The app.py
file
Let’s check the content of the app.py
, which is the base of our project, because it is from where CDK will start its work:
#!/usr/bin/env python3 import aws_cdk as cdk from cdk_example.cdk_example_stack import CdkExampleStack app = cdk.App() CdkExampleStack(app, "cdk-example") app.synth()
The first thing we do import aws_cdk as cdk
to import the module aws_cdk dir
, which is in the directory .venv
:
[simterm]
$ find . -name aws_cdk ./.venv/lib/python3.11/site-packages/aws_cdk
[/simterm]
Next, with the from cdk_example.cdk_example_stack import CdkExampleStack
we import the CdkExampleStack(Stack)
class from the cdk_example_stack.py
module:
from constructs import Construct from aws_cdk import ( Duration, Stack, aws_iam as iam, aws_sqs as sqs, aws_sns as sns, aws_sns_subscriptions as subs, ) class CdkExampleStack(Stack): def __init__(self, scope: Construct, construct_id: str, **kwargs) -> None: super().__init__(scope, construct_id, **kwargs) queue = sqs.Queue( self, "CdkExampleQueue", visibility_timeout=Duration.seconds(300), ) topic = sns.Topic( self, "CdkExampleTopic" ) topic.add_subscription(subs.SqsSubscription(queue))
And in the cdk_example_stack.py
we can already see the resources that will be created – SQS, and SNS with a Subscription.
You can view the resources in the PyDoc:
[simterm]
>>> import aws_cdk as cdk >>> help (cdk.App())
[/simterm]
Where the class App
will be described:
[simterm]
Help on App in module aws_cdk object: class App(Stage) | App(*args: Any, **kwargs) -> Any | | A construct which represents an entire CDK app. This construct is normally the root of the construct tree. | | You would normally define an ``App`` instance in your program's entrypoint, | then define constructs where the app is used as the parent scope. ...
[/simterm]
OK, let’s move on.
The cdk list
(or ls
) will return us a list of resources in the current directory/project:
[simterm]
$ cdk ls cdk-example
[/simterm]
Next, we can try with the cdk synth
, which will generate a CloudFormation template, which will be used during deployment:
[simterm]
$ cdk synth Resources: CdkExampleQueue7618E31B: Type: AWS::SQS::Queue Properties: VisibilityTimeout: 300 UpdateReplacePolicy: Delete DeletionPolicy: Delete Metadata: aws:cdk:path: cdk-example/CdkExampleQueue/Resource CdkExampleQueuePolicy839151B5: Type: AWS::SQS::QueuePolicy ...
[/simterm]
Working with the AWS CDK
AWS Account CDK Bootstrap
Before deploying the project, we need to configure our AWS account (or region) for AWS CDK. For this, we use the cdk bootstrap
command, which will create a CloudFormation stack with the resources necessary for the CDK to work – an S3 bucket, roles and policies in IAM, an ECR repository, and records in the AWS Systems Manager Parameter Store. See bootstrapping.
Let’s start:
[simterm]
$ cdk -v bootstrap ... ⏳ Bootstrapping environment aws://264***286/eu-central-1... ... CDKToolkit | 0/12 | 1:43:06 PM | REVIEW_IN_PROGRESS | AWS::CloudFormation::Stack | CDKToolkit User Initiated CDKToolkit | 0/12 | 1:43:11 PM | CREATE_IN_PROGRESS | AWS::CloudFormation::Stack | CDKToolkit User Initiated CDKToolkit | 0/12 | 1:43:16 PM | CREATE_IN_PROGRESS | AWS::IAM::Role | FilePublishingRole CDKToolkit | 0/12 | 1:43:16 PM | CREATE_IN_PROGRESS | AWS::IAM::Role | LookupRole CDKToolkit | 0/12 | 1:43:16 PM | CREATE_IN_PROGRESS | AWS::SSM::Parameter | CdkBootstrapVersion CDKToolkit | 0/12 | 1:43:16 PM | CREATE_IN_PROGRESS | AWS::IAM::Role | CloudFormationExecutionRole CDKToolkit | 0/12 | 1:43:16 PM | CREATE_IN_PROGRESS | AWS::ECR::Repository | ContainerAssetsRepository CDKToolkit | 0/12 | 1:43:16 PM | CREATE_IN_PROGRESS | AWS::IAM::Role | ImagePublishingRole CDKToolkit | 0/12 | 1:43:16 PM | CREATE_IN_PROGRESS | AWS::S3::Bucket | StagingBucket ... CDKToolkit | 11/12 | 1:44:02 PM | CREATE_COMPLETE | AWS::IAM::Role | DeploymentActionRole CDKToolkit | 12/12 | 1:44:04 PM | CREATE_COMPLETE | AWS::CloudFormation::Stack | CDKToolkit [13:44:09] Stack CDKToolkit has completed updating ✅ Environment aws://264***286/eu-central-1 bootstrapped.
[/simterm]
Check the S3 bucket:
[simterm]
$ aws s3 ls 2023-05-10 13:43:42 cdk-hnb659fds-assets-264***286-eu-central-1
[/simterm]
cdk deploy
And now we can execute the cdk deploy
, which will create a CloudFormation Stack with our SQS/SNS/Subscription:
[simterm]
$ cdk deploy ... cdk-example: assets built This deployment will make potentially sensitive changes according to your current security approval level (--require-approval broadening). Please confirm you intend to make the following modifications: IAM Statement Changes ┌───┬────────────────────────┬────────┬─────────────────┬───────────────────────────┬────────────────────────────────────────────────────────┐ │ │ Resource │ Effect │ Action │ Principal │ Condition │ ├───┼────────────────────────┼────────┼─────────────────┼───────────────────────────┼────────────────────────────────────────────────────────┤ │ + │ ${CdkExampleQueue.Arn} │ Allow │ sqs:SendMessage │ Service:sns.amazonaws.com │ "ArnEquals": { │ │ │ │ │ │ │ "aws:SourceArn": "${CdkExampleTopic}" │ │ │ │ │ │ │ } │ └───┴────────────────────────┴────────┴─────────────────┴───────────────────────────┴────────────────────────────────────────────────────────┘ (NOTE: There may be security-related changes not in this list. See https://github.com/aws/aws-cdk/issues/1299) Do you wish to deploy these changes (y/n)? y ...
[/simterm]
Respond Y:
[simterm]
... Do you wish to deploy these changes (y/n)? y cdk-example: deploying... [1/1] [0%] start: Publishing 20e979ce16c7aba5e874330247d9054b841ea313261b523b47a50fc4cd1d6662:current_account-current_region [100%] success: Published 20e979ce16c7aba5e874330247d9054b841ea313261b523b47a50fc4cd1d6662:current_account-current_region cdk-example: creating CloudFormation changeset... [███████████████████▎······································] (2/6) 1:48:50 PM | CREATE_IN_PROGRESS | AWS::CloudFormation::Stack | cdk-example 1:48:54 PM | CREATE_IN_PROGRESS | AWS::SQS::Queue | CdkExampleQueue ...
[/simterm]
CDK will locally generate a template file cdk.out/cdk-example.template.json
for the CloduFormation, and then will load it into the CDK bucket that was created at runtime cdk bootstrap
:
[simterm]
$ aws s3 ls cdk-hnb659fds-assets-264***286-eu-central-1 2023-05-10 13:48:45 5750 20e979ce16c7aba5e874330247d9054b841ea313261b523b47a50fc4cd1d6662.json
[/simterm]
Check the CloduFormation stack:
[simterm]
$ aws cloudformation list-stacks { "StackSummaries": [ { "StackId": "arn:aws:cloudformation:eu-central-1:264***286:stack/cdk-example/3c4e4db0-ef20-11ed-9672-0a9a3483d50e", "StackName": "cdk-example", "CreationTime": "2023-05-10T10:48:45.099000+00:00", ...
[/simterm]
Or from the AWS Console:
Meanwhile, the deployment is complete:
[simterm]
... ✅ cdk-example ✨ Deployment time: 91.52s Stack ARN: arn:aws:cloudformation:eu-central-1:264***286:stack/cdk-example/3c4e4db0-ef20-11ed-9672-0a9a3483d50e ✨ Total time: 97.99s
[/simterm]
cdk diff
Ok, we saw how it all works on everything ready, now let’s try to create something of our own, for example – an S3 basket.
Add `aws_s3 as s3` to imports, remove sns/sqs, I am, and Duration.
Also, remove the SQS and SNS resources from the class CdkExampleStack
and add the creation of the basket, can take an example from the PyPI documentation:
from constructs import Construct from aws_cdk import ( Stack, aws_s3 as s3 ) class CdkExampleStack(Stack): def __init__(self, scope: Construct, construct_id: str, **kwargs) -> None: super().__init__(scope, construct_id, **kwargs) bucket = s3.Bucket(self, "MyEncryptedBucket", encryption=s3.BucketEncryption.KMS )
And let’s see what will be returned to us with cdk diff
:
Red -
is what will be deleted, green +
is what will be created (it reminded me terraform plan
a lot).
Okay, start the deployment:
[simterm]
$ cdk deploy ✨ Synthesis time: 6.38s cdk-example: building assets... ... Do you wish to deploy these changes (y/n)? y cdk-example: deploying... [1/1] [0%] start: Publishing 5bb8c7fc8643769d69d5eb9712af36c955f9b509cc05d26740d035e9d7225a16:current_account-current_region [100%] success: Published 5bb8c7fc8643769d69d5eb9712af36c955f9b509cc05d26740d035e9d7225a16:current_account-current_region cdk-example: creating CloudFormation changeset... [████████████▉·············································] (2/9) 11:29:47 AM | UPDATE_IN_PROGRESS | AWS::CloudFormation::Stack | cdk-example 11:31:54 AM | CREATE_IN_PROGRESS | AWS::S3::Bucket | MyEncryptedBucket ...
[/simterm]
Let’s see how it looks in the UI:
Done.
Let’s clean up after ourselves – delete the stack.
cdk destroy
Check the stack name with the list
:
[simterm]
$ cdk ls cdk-example
[/simterm]
And run cdk destroy
with the name of the stack to completely remove it:
[simterm]
$ cdk destroy cdk-example Are you sure you want to delete: cdk-example (y/n)? y cdk-example: destroying... [1/1] ✅ cdk-example: destroyed
[/simterm]
Check the Console
But the S3 bucket and KMS key were not deleted. Why?
AWS CDK RemovalPolicy
Because aws_cdk.core
has the RemovalPolicy
, which by default has the Retain value.
This policy controls what happens to resources that have been removed from CloudFormation control:
- if a resource is deleted from the template
- a resource needs to be replaced by creating a new one, so CloudFormation creates a new one and removes the old one from its control, but leaves the resource itself
- a CloudFormation stack removed
The last point worked for us.
Okay, let’s repeat the experiment – create a stack again, but now will add the removal_policy
to parameter to the basket to delete it when deleting the stack, and auto_delete_objects=True
to delete all objects in it, because otherwise the basket cannot be deleted.
In addition, the KMS Key for the basket must be created as a separate object and have removal_policy
passed to it, and then this Key must be passed as an argument to the encryption_key
basket parameter:
from constructs import Construct from aws_cdk import ( Stack, RemovalPolicy, aws_s3 as s3, aws_kms as kms ) class CdkExampleStack(Stack): def __init__(self, scope: Construct, construct_id: str, **kwargs) -> None: super().__init__(scope, construct_id, **kwargs) my_key = kms.Key(self, "MyKey", enable_key_rotation=True, removal_policy=RemovalPolicy.DESTROY ) bucket = s3.Bucket(self, "MyEncryptedBucket", encryption=s3.BucketEncryption.KMS, encryption_key=my_key, removal_policy=RemovalPolicy.DESTROY, auto_delete_objects=True )
Run synth
:
[simterm]
$ cdk synth Resources: MyKey6AB29FA6: Type: AWS::KMS::Key ... UpdateReplacePolicy: Delete DeletionPolicy: Delete ... MyEncryptedBucket9A8D2FE1: Type: AWS::S3::Bucket ... UpdateReplacePolicy: Delete DeletionPolicy: Delete ...
[/simterm]
Now it should work – we can see both resources have the DeletionPolicy: Delete
.
Let’s deploy:
[simterm]
$ cdk deploy ... cdk-example: creating CloudFormation changeset... ✅ cdk-example ✨ Deployment time: 180.7s Stack ARN: arn:aws:cloudformation:eu-central-1:264***286:stack/cdk-example/1f5353d0-efe4-11ed-a627-02deb26b4a5c ✨ Total time: 187.53s
[/simterm]
And delete:
[simterm]
$ cdk destroy Are you sure you want to delete: cdk-example (y/n)? y cdk-example: destroying... [1/1] ✅ cdk-example: destroyed
[/simterm]
I didn’t have time to take a screenshot, but the CDK was running an AWS Lambda which deleted objects in the S3 bucket, and maybe the bucket itself.
Well, that’s all for now.
The Workshop has more examples, and some more advanced, so I’d recommend it if you plan to the AWS CDK.