How to Use AWS SDK to Retrieve Data for Your AWS CDK App
In this article I discuss the concept of fetching information on AWS resources during deployment of an AWS CDK application using AWS SDK (for JavaScrip v2). If you are not familiar with those tools feel free to check the official documentation for some basics.
Lately I was involved in a AWS CDK project using Typescript to implement infrastructure-as-code (IaC). The goal was to reuse code and deploy infrastructure consistently across different environments. Just by changing some parameters you’re good to deploy as many copies of your infrastructure as you wish.
For instance if you have an IaC artifact in AWS CDK describing a S3 bucket with some custom configuration you can easily deploy this code multiple times. A possible solution to accomplish this to adapt some AWS SDK code in your AWS CDK app and we will further discuss how this can be done as shown in the target architecture diagram below.
Target Architecture AWS CDK App integrating AWS SDK
Simple IaC application example in AWS CDK
The following example showcases how this can be achieved. Once myS3Stack is specified you can deploy multiple instances of your s3 bucket. You only need to change the bucketName in this case.
// Simple stack using s3 construct
import * as s3 from 'aws-cdk/aws-s3'
import {App, Stack} from 'aws-cdk'
// IaC artifact deploying a s3 bucket on creation
class myS3Stack extends Stack {
constructor (app: App, bucketName: string) {
super (app, bucketName);
// This statement actually creates a s3 bucket
const happyBucket = new s3.Bucket(scope, id);
}
}
const myApp = new App();
new myS3Stack (myApp, 'myS3Stack'); // 1st s3 bucket
new myS3Stack (myApp, 'myIdenticalS3Stack'); // 2nd s3 bucket with same config as myS3Stack
new myS3Stack (myApp, '...');
However, this is a pretty static IaC approach which requires you to know each bucketName in forward. But what if you don’t know them until deployment time?
Let’s say you want create s3 buckets for a dynamic number of AWS resources as EC2 instances, AWS Outposts or other. You could look up identifiers each time before you deploy your buckets and insert in your code e.g. in a nice iterable array as resourceIdentifiers:
// Please don't try this at home.
const resourceIdentifiers: string[] = [
"id1",
"id2",
"..."];
This is getting tedious very quick as quantity of resources and frequency of changes of your IaC app increase. If you have multiple stages and environments it`s getting even worse. But no worries because AWS SDK is here to save the day. So let’s check what we can do.
Adding AWS SDK to your Application
Since AWS SDK is not shipped with CDK per default (at least in npm) you must add it to your project. Simply check AWS’s documentation on how to achieve this e.g. by using the package manager for NodeJS npm.
// This will import AWS.Outposts from AWS SDK for JavaScript v2
import * as sdk from 'aws-sdk';
As soon as you have installed SDK you can simply import the library or selected modules and use them in your IaC project. So why don’t we just check on how to use SDK to retrieve a list of AWS.Outposts?
Example: Retrieve a list of AWS Outposts from your account
If you briefly check AWS.Outposts documentation you will learn about the ListOutposts() method. It leverages the AWS Outposts Service API using HTTP POST requests returning lists of AWS Outpost resources in the HTTP response body. But let’s not bother with theory and let us take a look on how this translates to practice:
// Import Outposts API class with its methods
import {Outposts} from 'aws-sdk';
// Calling the listOutposts() function and passing callback function to process data from response
new Outposts.listOutposts({}, function(err, data) {
if (err) console.log("ups"); // error handling
else data.Outposts?.foreach(outpost => {console.log(outpost.Name)}); // print name for each AWS Outpost
});
The above snippet shows how to load the listOutposts() method provided by AWS SDK. It expects a callback function which processes HTTP errors (err). On success it returns a list of Outposts using the data container with some interesting stuff inside.
// content passed through the "data" container
{
Outposts: [
{
OutpostId: 'some id',
OwnerId: 'some account',
OutpostArn: 'some outpost arn',
SiteId: 'some id',
Name: 'happy outpost',
Description: 'some useful info',
LifeCycleStatus: 'some Status',
AvailabilityZone: 'some az',
AvailabilityZoneId: 'some az id',
Tags: { some_tag_key: 'some value',...},
SiteArn: 'some site arn',
SupportedHardwareType: 'some type'
},
{
... //another Outpost
}
]
}
The cool thing about this is that you can now access all of this information quite comfortably and iterate through all resources to do some magic. Isn’t this great? Let’s apply this to our sample app from the beginning!
Adding AWS SDK to the mix
All left to do is to add our AWS SDK snippet into our app to enable it to read available resources by itself.
import * as s3 from 'aws-cdk/aws-s3'
import {App, Stack} from 'aws-cdk'
import {Outposts} from 'aws-sdk'; // hello SDK
class myS3Stack extends Stack {
constructor (app: App, bucketName: string) {
super (app, bucketName);
const happyBucket = new s3.Bucket(scope, id);
}
}
const myApp = new App();
// add S3 bucket for each Outpost
new Outposts.listAssets({}, function(err, data) {
if (err) console.log(err, err.stack);
else data.Outposts?.foreach(outpost => {
new myS3Stack(myApp, outpost.Name?);
});
}
And that’s it! Now we are able to create s3 buckets depending on how many AWS resources we find when deploying our AWS CDK app. Despite we used Outpost this time, the concept works perfectly for most resources accessible by SDK.
Caveat: Looking closer into this in AWS CDK Constructs
The above code snippets use anonymous and arrow functions. This can give you some headaches if you use them in conjunction with this. For instance if you want to create resources withing an anonymous callback function within a Stack class you will quickly learn how it doesn’t work.
class myS3Stack extends Stack {
constructor (app: App, bucketName: string) {
new Outposts.listAssets({}, function(err, data) { // 1st anonymous callback
if (err) console.log(err, err.stack);
else data.Outposts?.foreach(outpost => { // 2nd anonymous callback
// This won't work since anonymous callback functions don't have access to 'this' reference
new s3.Bucket(this, outpost.Name?);
});
}
}
}
In the example above you see two callback functions which wont recognize this. But you can solve this by passing named functions in conjunction with the bind() property. Check TypeScript documentation for further reference.
Conclusion
Integrating AWS SDK into you AWS CDK app is actually straight-forward and gives you a lot of new opportunities. So if you are facing limitations caused by CDK just have in mind that SDK is around the corner and save the day.