AWS: CloudFormation – Nested Stacks and stacks parameters Import/Export

By | 02/29/2020
0 Comment

Nested Stacks in AWS CloudFormation are stacks, created from another, a “parent”, stack using AWS::CloudFormation::Stack.

The main idea behind the Nested Stacks is to avoid writing superfluous code and to make templates reusable.

Instead, a template is created only once, stored in an S3 bucket, and during stacks creation – you just refer to it.

For example, you can use the same template file to create two Load Balancers with different parameters and/or listeners using Conditions.

Documentation is available here>>>, and a good post is here>>>.

In this post we will:

  1. create a root stack – it will describe our other stacks, it will like a skeleton
  2. will add another stack with a VPC as a child stack to the root stack
  3. and one more child stack with AWS SecurityGroups

Also, our nested stack must be able to share their parameters between them to make it possible to use the same template for various environments – Dev/Stage/Prod.

In the end – we will take a brief overview of AWS CloudFormation parameters Import/Export feature between independent stacks.

Resulted templates are available on the Github.

Pitfalls

  1. do not delete nested stack manually – only via a “root” stack
  2. use AWS S3 Versioning for templates

The Root stack

Let’s start from writing a root stack’s template – root-stack.json:

{
  "AWSTemplateFormatVersion" : "2010-09-09",
  "Description" : "AWS CloudFormation Root stack",

  "Resources" : {
    "VPCStack": {
      "Type": "AWS::CloudFormation::Stack",
      "Properties": {
        "TemplateURL": "network-stack.yml"
      }
    }
  }
}

Here we are creating the only one resource – AWS::CloudFormation::Stack passing a child’s stack template file network-stack.yml via the root’s Properties.

In the TemplateURL will have to set an S3 bucket URL – will update it shortly.

Now, create the network-stack.yml file:

{
  "AWSTemplateFormatVersion" : "2010-09-09",
  "Description" : "AWS CloudFormation Nested Network Stack",

  "Resources" : {
    "VPC" : {
      "Type" : "AWS::EC2::VPC",
      "Properties" : {
        "CidrBlock" : "11.0.0.0/16"
      }
    }
  }
}

Create an S3 bucket to store our templates:

[simterm]

$ aws s3api create-bucket --bucket eks-cloudformation --region eu-west-3 --create-bucket-configuration LocationConstraint=eu-west-3 --profile arseniy --region eu-west-3
{
    "Location": "http://eks-cloudformation.s3.amazonaws.com/"
}

[/simterm]

Remember the «Location»: «http://eks-cloudformation.s3.amazonaws.com/» – will need it now.

For such a bucket will be a really good idea to enable versioning.

Enable it:

[simterm]

$ aws --region eu-west-3 --profile arseniy s3api put-bucket-versioning --bucket bttrm-eks-cloudformation --versioning-configuration Status=Enabled

[/simterm]

Upload your  network-stack.yml to the bucket:

[simterm]

$ aws --profile arseniy --region eu-west-3 s3 cp network-stack.yml s3://bttrm-eks-cloudformationnn
upload: ./network-stack.yml to s3://bttrm-eks-cloudformation/network-stack.yml

[/simterm]

Go back to the root-stack.json, update its TemplateURL – now set it as an URL to the bucket:

{
  "AWSTemplateFormatVersion" : "2010-09-09",
  "Description" : "AWS CloudFormation Root stack",

  "Resources" : {
    "VPCStack": {
      "Type": "AWS::CloudFormation::Stack",
      "Properties": {
        "TemplateURL": "https://bttrm-eks-cloudformation.s3.amazonaws.com/network-stack.yml"
      }
    }
  }
}

Create the root stack:

[simterm]

$ aws cloudformation create-stack --stack-name nested-stacks-root --template-body file://root-stack.json --profile arseniy --region eu-west-3
{
    "StackId": "arn:aws:cloudformation:eu-west-3:534****385:stack/nested-stacks-root/6450c320-57ab-11ea-be30-0a9cc8c39c1c"
}

[/simterm]

Check it:

CloudFormation created a nested-stacks-root stack and its child stack named nested-stacks-root-VPCStack-1FTY8TI2PR2D2 with VPC, as described in the network-stack.yml template:

AWS CloudFormation package && deploy

To avoid uploading templates manually we can use AWS CLI CloudFormation package and deploy options.

package

package will copy specified files or a whole directory in an S3 bucket.

Update your root-stack.json — replace the TemplateURL of the VPCStack Resouce to a local path – full or relative to the root stack’s file:

{
  "AWSTemplateFormatVersion" : "2010-09-09",
  "Description" : "AWS CloudFormation Root stack",

  "Resources" : {
    "VPCStack": {
      "Type": "AWS::CloudFormation::Stack",
      "Properties": {
        "TemplateURL": "network-stack.yml"
      }
    }
  }
}

Pack templates and upload them to the S3:

[simterm]

$ aws cloudformation package --template-file root-stack.json --output-template packed-nested-stacks.json --s3-bucket bttrm-eks-cloudformation --profile arseniy --region eu-west-3 --use-json
Uploading to ce12898553365980827b9aa59a99426d.template  187 / 187.0  (100.00%)
Successfully packaged artifacts and wrote output template to file packed-nested-stacks.json.
Execute the following command to deploy the packaged template
aws cloudformation deploy --template-file /home/setevoy/Work/devops/projects/EKS/roles/cloudformation/files/packed-nested-stacks.json --stack-name <YOUR STACK NAME>

[/simterm]

Here:

  1. CLI uploads all files (artifacts)  found in the root-stack.json to the AWS S3
  2. will update TemplateURL to set the S3 URL instead of the local paths
  3. will return a newly generated template which can be used with the deploy option

The template returned by the package will be saved in the packed-nested-stacks.json (I’m using the --use-json as by default YAML will be used, check the What is: YAML – its overview, basic data types, YAML vs JSON, and PyYAML post).

Check its content:

{
    "AWSTemplateFormatVersion": "2010-09-09",
    "Description": "AWS CloudFormation Root stack",
    "Resources": {
        "VPCStack": {
            "Type": "AWS::CloudFormation::Stack",
            "Properties": {
                "TemplateURL": "https://s3.eu-west-3.amazonaws.com/eks-cloudformation/ce12898553365980827b9aa59a99426d.template"
            }
        }
    }
}

And let’s see the https://s3.eu-west-3.amazonaws.com/eks-cloudformation/ce12898553365980827b9aa59a99426d.template file content:

[simterm]

$ aws --profile arseniy --region eu-west-3 s3 cp --quiet s3://eks-cloudformation/ce12898553365980827b9aa59a99426d.template /dev/stdout
AWSTemplateFormatVersion: '2010-09-09'
Description: AWS CloudFormation Nested Network Stack
Resources:
  VPC:
    Type: AWS::EC2::VPC
    Properties:
      CidrBlock: 11.0.0.0/16

[/simterm]

deploy

After we performed the package command – CLI displayed a tip about the following step:

[simterm]

...
Execute the following command to deploy the packaged template                                                                                                                                                                                 
aws cloudformation deploy --template-file /home/setevoy/Work/devops/projects/EKS/roles/cloudformation/files/packed-nested-stacks.json --stack-name <YOUR STACK NAME>

[/simterm]

Apply it to the stack created a few steps ago:

[simterm]

$ aws --profile arseniy --region eu-west-3 cloudformation deploy --template-file packed-nested-stacks.json --stack-name nested-stacks-root

Waiting for changeset to be created..
Waiting for stack create/update to complete
Successfully created/updated stack - nested-stacks-root

[/simterm]

deploy created a ChangeSet and applied it to the root stack:

Nested stack – passing parameters

At this moment we have no any Parameters in the templates.

For example, in the root stack, we can set some global values.

Let’s add a VPC’s CIDR, which will be used by the network-stack then.

Update theroot-stack.json and add the Parameters block with one parameter VPCCIDRBlock and its default value:

{
  "AWSTemplateFormatVersion" : "2010-09-09",
  "Description" : "AWS CloudFormation Root stack",

  "Parameters": {
    "VPCCIDRBlock": {
      "Description": "VPC CidrBlock",
      "Type": "String",
      "Default": "11.0.0.0/16"
    }
  },  
...

In the Resources block for the VPCStack resource add Parameters section with VPCCIDRBlock parameter where we will pass our VPCCIDRBlock value from the “global” parameters:

{
  "AWSTemplateFormatVersion" : "2010-09-09",
  "Description" : "AWS CloudFormation Root stack",

  "Parameters": {
    "VPCCIDRBlock": {
      "Description": "VPC CidrBlock",
      "Type": "String",
      "Default": "11.0.0.0/16"
    }
  },

  "Resources" : {
    "VPCStack": {
      "Type": "AWS::CloudFormation::Stack",
      "Properties": {
        "TemplateURL": "network-stack.yml",
        "Parameters": {
          "VPCCIDRBlock" : { "Ref": "VPCCIDRBlock" }
        }
      }
    }
  }
}

In the network-stack.yml template add it to the VPC resource:

{
  "AWSTemplateFormatVersion" : "2010-09-09",
  "Description" : "AWS CloudFormation Nested Network Stack",

  "Parameters": {
    "VPCCIDRBlock": {
      "Description": "VPC CidrBlock",
      "Type": "String"
    }
  },

  "Resources" : {
    "VPC" : {
      "Type" : "AWS::EC2::VPC",
      "Properties" : {
        "CidrBlock" : { "Ref": "VPCCIDRBlock" }
      }
    }
  }
}

Pack it to the S3:

[simterm]

$ !518
aws cloudformation package --template-file root-stack.json --output-template packed-nested-stacks.json --s3-bucket bttrm-eks-cloudformation --profile arseniy --region eu-west-3 --use-json

[/simterm]

Deploy:

[simterm]

$ aws --profile arseniy --region eu-west-3 cloudformation deploy --template-file packed-nested-stacks.json --stack-name nested-stacks-root --profile arseniy --region eu-west-3 --use-json

[/simterm]

Check the nested stack’s Parameters:

VPCCIDRBlock was added.

Nested stack Outputs

Also, Nested Stack allows using other stacks Outputs via the Fn::GetAtt function.

Let’s add a third stack named SecurityGroupStack where our SecurityGroup will be described.

We will pass the VPC ID to this SecurotyGroup using Outputs of the network-stack.yml stack.

Remember, that such parameters can be passed only from the “bottom” to “top” and back.

I.e. you can not pass a parameter directly from the VPCStack to the SecurityGroupStack, but you can return a value to the root stack and then use it as a parameter for a child stack.

To do so:

  1. in the VPCStack (network-stack.yml) stack we will add Outputs to return  an ID of the VPC crated
  2. in the root stack root-stack.json we will describe a new stack with the SecurityGroupStack name, which Parameters will accept a vlue from the VPCStack Outputs (network-stack.yml)
  3. will create a new stack SecurityGroupStack, which template will use Parameters > VPCID

network-stack.yml Stack Outputs

Add output for the VPC:

...
  "Resources" : {
    "VPC" : {
      "Type" : "AWS::EC2::VPC",
      "Properties" : {
        "CidrBlock" : { "Ref": "VPCCIDRBlock" }
      }
    }
  },

  "Outputs" : {
    "VPCID" : {
      "Description" : "EKS VPC ID",
      "Value" : { "Ref" : "VPC" }
    }
  }
}

root-stack.json Stack

In the root stack add a new stack SecurityGroupStack and add the  VPCID from the Outputs of the VPCStack stack to  Parameters of the SecurityGroupStack:

...
  "Resources" : {
    "VPCStack": {
      "Type": "AWS::CloudFormation::Stack",
      "Properties": {
        "TemplateURL": "network-stack.yml",
        "Parameters": {
          "VPCCIDRBlock" : { "Ref": "VPCCIDRBlock" }
        }
      }
    },
    "SecurityGroupStack": {
      "Type": "AWS::CloudFormation::Stack",
      "Properties": {
        "TemplateURL": "sg-stack.yml",
        "Parameters": {
          "VPCID" : { "Fn::GetAtt": ["VPCStack", "Outputs.VPCID"] }
        }
      }
    }
  }
...

Create a new template file for the SecurityGroups — sg-stack.yml:

{
  "AWSTemplateFormatVersion" : "2010-09-09",

  "Description" : "AWS CloudFormation SecurityGroups stack",

  "Parameters" : {
    "VPCID": {
      "Description": "Network Stack VPC ID",
      "Type": "String",
    }
  },

  "Resources" : {
    "SecurityGroup": {
      "Type": "AWS::EC2::SecurityGroup",
      "Properties" : {
        "GroupDescription" : "Example SecurityGroup",
        "VpcId"            : { "Ref": "VPCID" },
        "SecurityGroupIngress" : [
          {
            "Description": "Allow HTTP",
            "IpProtocol" : "tcp",
            "FromPort"   : 80,
            "ToPort"     : 80,
            "CidrIp"     : "0.0.0.0/0"
          },
          {
            "Description": "Allow HTTPS",
            "IpProtocol" : "tcp",
            "FromPort"   : 443,
            "ToPort"     : 443,
            "CidrIp"     : "0.0.0.0/0"
          }
        ]
      }
    },
  }
}

Here, in the "VpcId" : { "Ref": "VPCID" }, we are using a VPC ID value to add this SecirtyGroup to the same VPC.

Pack, deploy, check stacks:

A new stack was created.

Check its Parameters:

All good.

Template reuse

As already mentioned at the beginning, the main idea is modularity, when we can use the same template file to create similar resources.

Let’s say, we’d like to have two VPC with a different CIDRs.

We can use CloudFormation Mappins and set two various network blocs there.

In the root template root-stack.json remove the "Parameters": "VPCCIDRBlock" and add Mappings instead:

{
  "AWSTemplateFormatVersion" : "2010-09-09",
  "Description" : "AWS CloudFormation Root stack",

  "Mappings": {
    "VPCCIDRBlock": {
      "vpc1": {
        "cidr": "11.0.0.0/16"
      },
      "vpc2": {
        "cidr": "12.0.0.0/16"
      }
    }
  },
...

In the Resources add another stack with a VPC using the same template file, but now in its Parameters use the Fn::FindInMap function to get a value for the Property VPCCIDRBlock:

...
  "Resources" : {
    "VPCStack1": {
      "Type": "AWS::CloudFormation::Stack",
      "Properties": {
        "TemplateURL": "network-stack.yml",
        "Parameters": {
          "VPCCIDRBlock" : { "Fn::FindInMap" : [ "VPCCIDRBlock", "vpc1", "cidr" ] }
        }
      }
    },
    "VPCStack2": {
      "Type": "AWS::CloudFormation::Stack",
      "Properties": {
        "TemplateURL": "network-stack.yml",
        "Parameters": {
          "VPCCIDRBlock" : { "Fn::FindInMap" : [ "VPCCIDRBlock", "vpc2", "cidr" ] }
        }
      }
    },
...

Do not forget about SecurityGroup – add another one – again, using the same SG’s template sg-stack.yml, and attach this second SecurityGroup to the second VPC:

...
    "SecurityGroupStack1": {
      "Type": "AWS::CloudFormation::Stack",
      "Properties": {
        "TemplateURL": "sg-stack.yml",
        "Parameters": {
          "VPCID" : { "Fn::GetAtt": ["VPCStack1", "Outputs.VPCID"] }
        }
      }
    },
    "SecurityGroupStack2": {
      "Type": "AWS::CloudFormation::Stack",
      "Properties": {
        "TemplateURL": "sg-stack.yml",
        "Parameters": {
          "VPCID" : { "Fn::GetAtt": ["VPCStack2", "Outputs.VPCID"] }
        }
      }
    }
...

Deploy, check:

CloudFormation removed the VPCStack and instead created two new stacks – VPCStack1 и VPCStack2, and in the same way – for the SecurityGroup.

Import/export values vs Nested Stacks

The Outputs in nested stacks is good to share parameters between affined stacks, but it will work only for current stacks “tree” and can not be shared with a not related stack in this AWS account.

Here we can use another AWS CLoudFormatiuon feature called cross-stack reference – in a first stack you’ll create an Export, and in an another – their Import.

Pitfalls

  • exported values are accessible within the same AWS region only
  • you can’t delete a stack if it importing values used by any other stack

Add an Export for the ID of the SecurityGroups crated to make it available to use later in other, independent, stacks.

To do so, update the SecurityGroup sg-stack.yml template and add its ID to the Outputs:

...
            "ToPort"     : 443,
            "CidrIp"     : "0.0.0.0/0"
          }
        ]
      }
    },
  },

  "Outputs" : {
    "SecurityGroupID" : {
      "Description" : "The SecurityGroup ID",
      "Value" :  { "Ref" : "SecurityGroup" }
    }
  }
}

In the root template – update its Outputs and add the Export for both SecurityGroupStack-stacks:

...
  "Outputs" : {
    "SecurityGroup1" : {
      "Description" : "The SecurityGroup-1 ID",
      "Value" :  { "Fn::GetAtt": [ "SecurityGroupStack1", "Outputs.SecurityGroupID" ] },
      "Export": { "Name": { "Fn::Sub": "${AWS::StackName}-SecurityGroupStack1" } }
    },
    "SecurityGroup2" : {
      "Description" : "The SecurityGroup-2 ID",
      "Value" :  { "Fn::GetAtt": [ "SecurityGroupStack2", "Outputs.SecurityGroupID" ] },
      "Export": { "Name": { "Fn::Sub": "${AWS::StackName}-SecurityGroupStack2" } }
    }
  }
}

Here:

  • in the Value we are getting a SecurityGroup ID from its Outputs
  • in the Export: Name with the Fn::Sub we are generating an uniq name.

Deploy, check the SecurityGroup‘s stack Outputs:

And Outputs of the root stack – find the Exported values:

Also, they are available now in the Exports of the whole CloudFormation for this account:

Now, we can use it for other stacks.

To check it, let’s add a stack with the only VPC resource (just because a CloudFormation stack has to have at least one Resource type), and in its Outputs using Fn::ImportValue we will display SecurityGroups IDs from the nested-stacks-root stack:

{
  "AWSTemplateFormatVersion" : "2010-09-09",
  "Description" : "AWS CloudFormation Nested Network stack",

  "Parameters": {
    "VPCCIDRBlock": {
      "Description": "VPC CidrBlock",
      "Type": "String",
      "Default": "13.0.0.0/16"
    }
  },

  "Resources" : {
    "VPC" : {
      "Type" : "AWS::EC2::VPC",
      "Properties" : {
        "CidrBlock" : { "Ref": "VPCCIDRBlock" }
      }
    }
  },

  "Outputs" : {
    "SecurityGroup1ID" : {
      "Description" : "The SecurityGroup ID",
      "Value" :  { "Fn::ImportValue" : "nested-stacks-root-SecurityGroupStack1" }
    },
    "SecurityGroup2ID" : {
      "Description" : "The SecurityGroup ID",
      "Value" :  { "Fn::ImportValue" : "nested-stacks-root-SecurityGroupStack2" }
    }
  }
}

Deploy it (better was to call the new stack like an “independent-” or “external-” instead of the nested-), and check its Outputs:

Done.

Useful links


Also published on Medium.