Building cloud applications is hard. On the one hand, the number of services available from public cloud providers is astonishing, and these offerings can dramatically increase your team’s velocity. And yet, integrating these services into a product nearly always requires deep knowledge, since documentation is not always as clear or well-organized as you’d like. Integrating them into a real application is tough, all the more when a particular product feature might need to direct requests to 5 or 6 different AWS services, each of which requires piles of JSON to configure correctly.

For my work at the Washington Post, I was recently tasked with creating a proof-of-concept (POC) to test out a particular combination of AWS services (MediaLive+MediaPackage+CloudFront+MediaTailor). MediaLive does live video encoding; MediaPackage performs origination and packaging; CloudFront does content-delivery, and MediaTailor stitches in ads. Over the course of a two week sprint, I managed to put together a fully functional POC in Node/TypeScript that would create the resources needed in AWS for this particular video workflow. Here’s how I did it:

Step 1: Create whatever you need in the AWS Web Console

Amazon’s web console holds your hand. This is a good thing, but should be used with care. I knew from reading the AWS docs what services I’d probably need to use. So I opened up our Sandbox account and got to work. I created a MediaPackage channel and a MediaPackage origin endpoint. The web console helpfully offered to create a CloudFront web distribution to sit in front of this. I said, “sure!” Then I went in and started making the entities in MediaLive that I needed (Inputs, Input Security Groups, and a Channel). The MediaLive channel was by far the trickiest, since it includes transcoding profiles for every different resolution you want to produce (to get a since of the complexity in play, you can look here. It was more than 300 lines of JSON). Once I got the video working (I could stream video to an RTMP endpoint and watch the resulting HLS stream coming out the other side) I added in a MediaTailor to test ad stitching. This threw me for a loop for a while, because MediaPackage was unhelpfully squashing the ad markers. But once I found the right checkbox to flip, this started working too. At this point, it was time to capture what I’d made in code so it would be repeatable.

2: Use the AWS CLI tools to dump the configuration of the stuff you’ve made

Trying to manually figure out how to encode all that I’d done in TypeScript client code would have taken forever. Even with TypeScript SDK docs, or CloudFormation docs to reference, I’d still have needed to map the values in the web UI to the code I was writing. Happily, there was an easier way: the AWS cli will simply dump out the configuration of whatever entity you’re interested in. For instance, this will give you JSON for the CloudFront distributions in your AWS account:

aws cloudfront list-distributions > distributions.json

The exact command varies depending on the service. aws <servicename> help will generally lead you to the right command. Generally you want one that starts with list or describe (e.g.

aws ec2 describe-instances
aws dynamodb describe-table --table-name <TABLE_NAME>

It’s helpful to remember that the aws-cli tools are fundamentally just convenience wrappers to REST APIs, which is why they often aren’t terrible user friendly. REST APIs and traditional *nix tools have different structures and conventions.

3: Craft this into either into CloudFormation or client SDK code

Now that you have a JSON representations of the entities you need, you can write these up as code, either as CloudFormation/CDK/Terraform templates, or with the AWS client SDKs. In my particular case, I wanted to create these resources in response to a REST API call, so I wrote up the creation logic in TypeScript using the AWS -v3 JavaScript SDK. These are awesome to use for a lot of reasons. I was able to just copy and paste in much of the JSON and just tweak the params that were different for each channel. And since it was TypeScript, I got told at compile time if a parameter was named incorrectly. It made juggling hundreds of lines of JSON configuration much less error-prone and frustrating.

Conclusion: The AWS CLI is your friend

I’ve since used the same strategy to put together some ECS blue/green deploy code. Juggling mountains of JSON is not fun. Creating what you need in the web console, by contrast, and then exporting via the CLI will speed up your develop of infrastructure-facing code, especially if you can’t find good examples to follow. Good luck!