Agility is arguably the most significant gain companies can derive from moving infrastructure to AWS. From a business perspective this means being able to make more meaningful and informed decisions about your business, and affecting change with less effort. AWS cost tracking and optimization in the cloud is among the most overlooked pieces of effectively achieving agility, and goes far beyond “running cheaper servers” or “having a cost tracking tool.”

Tracking AWS costs is complicated, especially when you have several major business units, distributed teams, and dozens or even hundreds of projects.

By default, your AWS bill will tell you how much your business spent overall on specific services (“AWS EC2 Service = $X, AWS S3 = $Y”), but businesses gain more from the cloud by understanding costs on a more granular level. Calculating the specific infrastructure costs on a product or component level allows businesses to logically invest, and avoids a “black hole” of infrastructure costs. Moreover, moving to the cloud often means a shift from CapEx to OpEx models, which we have seen actually increase the visibility of IT spend as a whole.

Unfortunately, there is no simple, push-button way to allocate AWS costs. But this is the process that we have found most successful for most businesses:

1. Determine the type / level of data each department needs.
2. Tag (almost) everything.
3. Automate tagging.
4. Create reports.

1. Determine your cost allocation needs

The first and often most challenging step is to determine what information each department needs. Then you establish the dimensions that would satisfy these needs. If you are just beginning this process, we suggest sticking to just a few dimensions, like Environment, Tier, and Team. You could then potentially break down a resource cost by Production / Dev, Database / Web, or by Critical Project 1 team / Application 5 team, for instance. You can always add more dimensions later. Some teams are small enough where having a different tag for every developer makes sense — it all depends on which metrics matter to you.

>>Watch a replay of our webinar: Cost Efficiency & Agility in the Cloud

A side note on this process. In our experience, companies do not talk about ongoing cost optimization before moving to AWS. They either assume that AWS comes with the kind of built-in reporting that their finance teams need, or that AWS is already “cheap compute” without a lot of cost wiggle-room. In addition, these companies are usually in the midst of some degree of DevOps transformation, meaning that teams are moving faster with greater autonomy — launching and shutting down resources and experimenting with new services. This means more complex cost allocation process that needs to be as agile as your development teams. You need a process that maintains the feedback loop between business and IT — we talk more about other steps necessary for maintaining this feedback loop here.

2. Tag everything

AWS cost tagging is a feature that allows you to associate certain resources with a group, and then create reports across multiple environments that carry this tag. S3, EC2, EBS, RDS, and most common AWS resources can be tagged. Tags are set up as key value pairs (KEY = VALUE).

Setting up these tags is a fairly simple process for existing resources; simply follow AWS’ documentation. Note that these values are case sensitive. Once your team does an initial pass, there will likely still be many untagged resources; you can easily find untagged resources to begin the cleanup process. Even then, some resources will remain untagged, which will show up as “not set” on your AWS Cost Allocation Report.

3. Automate tagging

The third step is to automate resource tagging, meaning that new resources automatically be designated certain tags in their AWS CloudFormation template. Not surprisingly, few engineers remember to tag things appropriately when creating resources, even when tagging is a company policy. Automation puts some “teeth” into your tagging policy, and also ensures your tags are consistent across resources (because “Dev” and “dev” and “development” are all treated as separate tags, for example).

CloudFormation can designate tags, but you need to either manually associate those tags as billing tags or use a configuration management tool like Puppet. Though building a CloudFormation template generally is a complex one, which we discuss here, designating tags in a CloudFormation template is a fairly simple process. If you ever need to rebuild your environment, tagging automation will be crucial, since manually tagging all your resources (again) is especially painful.

4. Create reports

Your finance team, your engineers, and your developers need access to cost data — but at different levels. You first need to link your cost tags to your billing and then create appropriate reports for different teams. You can create IAM policies that give specific users access to the billing console, but when you create custom reports, you probably will have missed smaller data transfer costs and background resources — and you would already have wasted hours of your time and your engineers’ time.

A more accurate, complete, and customizable report can be created using a 3rd party cost reporting tool, like Cloudability or CloudCheckr. It is often very worthwhile to invest in these tools, as they automatically pull data from multiple cost tagging sources and allow you to filter information more effectively. They also allow you to create alerts for when defined thresholds are exceeded.

In the end, there is nothing worse than getting your AWS bill and manually tracking back expenses to associated projects. It is not only inefficient and time-consuming, but it prevents the agile feedback loops between business and IT related to where to invest budgets or refocus priorities. Automated tagging enables your business as a whole to be more agile, and allows businesses to turn cost data into overall cost-efficiency.

by Ryan Freilino
Sr. Solutions Specialist, Logicworks

Logicworks is an enterprise cloud automation and managed services provider with 22+ years of experience transforming enterprise IT. Contact us to learn more about our managed cloud solutions.