AWS Dangling Resources, revisited

This topic, again? Yeah, egg on my face, I believe I ended the last blog with something like “constant pursuit of better.” The truth of the matter is, I deployed the code from the original post and, upon review of the results, it became clear that something was amiss. We were seeing results which we knew were false positives and thus knew that we needed to dig deeper.

The way we originally tackled dangling resources, via Route 53, was by looking for a cloudfront.net entry and then attempting to locate an S3 bucket matching that entry’s alias. While this logic may work in circumstances where a user tells S3 to host the contents of the bucket, it doesn’t necessarily hold true when the CloudFront service is properly utilized. I reached out to the team who handles this type of configuration to get a better understanding of how it’s properly configured and to make sure the change in logic would both eliminate the false positives and also report actual dangling resources.

CloudFront allows you to name entries whatever you’d like, they don’t need to match an S3 bucket name; the “Origin domain” in the entry is the associated S3 bucket. So how do we go about building logic to make sure S3 buckets associated with CloudFront distributions still exist? Much like before, we are still looking for S3 distributions and as we iterate through CloudFront entries we are gathering:

• Origin domain
• CloudFront ID
• CloudFront domain

If the Origin domain contains “s3”, we drop the tail end of the domain (ex: s3.us-east-1.amazonaws.com) and check to see if an S3 bucket exists that matches that domain. If it does exist, awesome. If not, we report that we have a dangling resource which needs to be addressed.

We still check Route 53 for entries pointed at CloudFront domains to validate if a distribution is configured properly. This becomes a bit more straightforward since we already have a dictionary containing all CloudFront defined entries. We then iterate through Route 53 records looking for “cloudfront” in the DNS name and then compare the AliasTarget DNS name to our dictionary of CloudFront distributions. If the AliasTarget DNS name is not in the CloudFront distribution dictionary, we report the useful information as a dangling resource!

“Why do we fall, sir? So we can learn to pick ourselves up.”* It’s easy to get discouraged when things you’ve spent time architecting and building fail but it’s important to learn from such failures. Rather than telling me my results were wrong, the teams I support helped me better understand the deployment architecture and data. Through that learning I was able to accurately identify misconfigurations and detect dangling resources. I hope you too can accept this blog update for what it is and see the positive side of failure.

* Alfred Pennyworth, Batman Begins (2005)