Debugging Tips: cfn-init and configsets

Here are some debugging tips that will help when working with cfn-init.

How It Works

The cfn-init script is usually called by the UserData script to kick off a bootstrap process. The cfn-init call is what actually applies the configset.

UserData cfn-init

You can make sure configsets are applied by calling the cfn-init script in the UserData script. The UserData script runs when EC2 instances are launched. Here’s an example UserData script.

#!/bin/bash
yum install -y aws-cfn-bootstrap # install cfn-init
/opt/aws/bin/cfn-init -v --stack ${AWS::StackName} --resource Instance --region ${AWS::Region}

The ${AWS::StackName} and ${AWS::Region} will be substituted for their actual values at CloudFormation runtime. The --resource Instance specifies which resource to add the configsets to.

Note: On AmazonLinux2 cfn-init is already installed.

Run Manually

The UserData script runs at launch time automatically. But you can also invoke the cfn-init script manually. Just log into the instance and call it. This can be especially useful for debugging. Here’s an example of running the script.

/opt/aws/bin/cfn-init -v --stack my-stack --resource Instance --region us-west-2

The command above downloads the Configset from CloudFormation template Instance resource and applies it.

Where is the metadata.json downloaded?

The cfn-init downloads the configset metadata to:

/var/lib/cfn-init/data/metadata.json

Note, this file gets overwritten each time cfn-init runs.

Run With a File

One quick way to debug configsets is to copy the generated /var/lib/cfn-init/data/metadata.json, modify it, and provide it directly to the cfn-init command. Example:

cp /var/lib/cfn-init/data/metadata.json /root/configset.json
/opt/aws/bin/cfn-init -v /root/configset.json

The cfn-init script itself only supports JSON when providing a file.

Here’s a YAML to JSON One Liner to help you convert it.

ruby -ryaml -rjson -e 'puts YAML.dump(JSON.load(ARGF))' < configset.yml > configset.json

And vice-versa:

ruby -ryaml -rjson -e 'puts YAML.dump(JSON.load(ARGF))' < configset.json > configset.yml

Where are the Logs?

Knowing where the cfn-init related logs are can also save a ton of time.

Path Description
/var/log/cfn-init-cmd.log cfn-init and command output with timestamps.
/var/log/cfn-init.log cfn-init and command output. Useful for when cfn-init is manually ran.
/var/log/cloud-init.log cloud-init logs from UserData launch script.
/var/log/cloud-init-output.log Output from the cloud-init UserData commands themselves. Useful for when the instance is launched for the first time.

cfn-hup reloader

The UserData script runs only at launch time. This means if you make changes to the CloudFormation Metadata.AWS::CloudFormation::Init configset, they do not magically get applied. The cfn-hup daemon must be set up. The cfn-hup daemon listens for changes in the metadata and applies them automatically.

It is recommended to set up cfn-hup reloader, so you do not have to log into the instance and kick off the cfn-init script manually.

Here’s the cfn-hup configset that sets up the cfn-hup reloader. The BoltOps Pro cfn-hup configset is available for free.

Pro tip: Use the <- and -> arrow keys to move back and forward.

Edit this page

See a typo or an error? You can improve this page. This website is available on GitHub and contributions are encouraged and welcomed. We love pull requests from you!