In a previous post, we created AWS VPCs using the Wizard and the CLI. If you are already using a tool like SaltStack in your infrastructure to handle other tasks, you can also use it to build cloud infrastructure. In the following example, we will build a VPC on our local machine and use AWS API via Salt to create our new network.
The ultimate is really to describe your new VPC and be able to have a tool that can create all the pieces. We are big users of SaltStack, as it supports sending many commands to public clouds to create infrastructure on your behalf. With Salt, you can describe your AWS VPC using YAML and then apply it to the region to create the same network each time it is run.
Salt supports doing this via its states system. The community helps out by providing example "formulas" that facilitate the creation of infrastructure in a cross-platform way. There is a nice AWS SaltStack formula available to do much of the heavy lifting for you. With many of these formulas, all you will really need to do is create a Salt Pillar that describes what you want built. The formula contains the states that read in the Pillar info to build the pieces.
Here is an example of a Salt Pillar that can generate the above VPC for you:
aws: region: us-west-1: profile: region: us-west-1 keyid: [insert your keyid] key: [insert your key] vpc: {%- set vpc_name = 'demo-blog-vpc' %} {{ vpc_name }}: cidr_prefix: '172.20' vpc: name: {{ vpc_name }} cidr_block: 172.20.0.0/16 instance_tenancy: default dns_support: 'true' dns_hostnames: 'true' internet_gateway: name: internet_gateway subnets: 1: name: public_subnet az: b nat_gateway: true
You can test this out locally if you have Salt installed on your desktop. I'm using Homebrew to install Salt locally:
$ brew install saltstack
Note: If you do install via brew, the virtualenv that it creates will not have the boto and boto3 libraries installed. To fix this, you need to use the pip command from inside the SaltStack virtualenv.
$ /usr/local/Cellar/saltstack/2017.7.1/libexec/bin/pip install boto boto3
If you installed via pip into your own virtualenv, you will need to perform the same action as those libs aren't required dependencies of Salt.
Then you can checkout the formula locally:
$ git clone https://github.com/saltstack-formulas/aws-formula
Next you can create your Pillar file and a top.sls in your Pillar directory so it can apply it:
$ cd aws-formula $ mkdir pillar
Inside the pillar directory, drop your YAML into a file called aws.sls that describes your AWS infrastructure. To get it to be used, you will also need a file called top.sls. This file also goes inside the pillar directory.
The contents of your top.sls will be:
base: '*': - aws
This tells salt to apply the aws Pillar to any server that matches. In our case, we will be calling it locally to apply the aws Pillar and build our infrastructure from our desktop.
Next, apply the aws state using the local definition of the states:
$ sudo salt-call state.sls aws --local --retcode-passthrough --file-root=$(pwd) --pillar-root=pillar
The result should be similar to this:
local: ---------- ID: aws_vpc_demo-blog-vpc_create Function: boto_vpc.present Name: demo-blog-vpc Result: True Comment: VPC demo-blog-vpc created. Started: 13:11:36.940121 Duration: 1623.927 ms Changes: ---------- new: ---------- vpc: ---------- cidr_block: 172.20.0.0/16 dhcp_options_id: dopt-431e1421 id: vpc-df84d1bb instance_tenancy: default is_default: False region: us-west-1 state: available tags: ---------- Name: demo-blog-vpc old: ---------- vpc: None ---------- ID: aws_vpc_demo-blog-vpc_create_internet_gateway Function: boto_vpc.internet_gateway_present Name: internet_gateway-demo-blog-vpc Result: True Comment: Internet gateway internet_gateway-demo-blog-vpc created. Started: 13:11:38.564276 Duration: 630.312 ms Changes: ---------- new: ---------- internet_gateway: igw-71ce3715 old: ---------- internet_gateway: None ---------- ID: aws_vpc_demo-blog-vpc_create_subnet_public_subnet Function: boto_vpc.subnet_present Name: public_subnet-demo-blog-vpc Result: True Comment: Subnet public_subnet-demo-blog-vpc created. Started: 13:11:39.194809 Duration: 1089.917 ms Changes: ---------- new: ---------- subnet: ---------- availability_zone: us-west-1b cidr_block: 172.20.1.0/24 id: subnet-abc337cc tags: ---------- Name: public_subnet-demo-blog-vpc vpc_id: vpc-df84d1bb old: ---------- subnet: None ---------- ID: aws_vpc_demo-blog-vpc_create_nat_gateway_public_subnet Function: boto_vpc.nat_gateway_present Result: True Comment: Nat gateway created. Started: 13:11:40.284957 Duration: 829.337 ms Changes: ---------- new: ---------- nat_gateway: nat-0ba184bb33f531f69 old: ---------- nat_gateway: None Summary for local ------------ Succeeded: 4 (changed=4) Failed: 0 ------------ Total states run: 4 Total run time: 4.173 s
Now you can quickly create the exact same infrastructure in any region by changing the region in your Salt Pillar.
When playing with the VPCs and all of their related objects. It can take time for some of these things to be deleted. If you are having issues deleting a subnet, for example, you will need to make sure that the NAT gateway has been deleted. Sometime this can take time and you will get DependancyViolation when attempting to remove the subnet or VPC to quickly.
Have fun building clouds!
Six Feet Up, Inc.
11208 Windermere Blvd.
Fishers, IN 46037 - USA
Phone: +1 (317) 861-5948
Email: info@sixfeetup.com
