Cloud Tagging Best Practices for Better Cost Allocation, Part 2

How to Tag AWS Resources via Terraform (tf)

AWS resources could be tagged by an Infrastructure-as-code (IaC) tool; either Terraform or CloudFormation works. Terraform has gained popularity due to its cloud-agnostic nature of building IaC and better state management and integrations with cloud vendors.

Terraform Example

We must create a block like the below in main.tf where we call the module from Terraform-utils.

module “account_tags” {
source = “github.com/Organization-dev/terraform-utils/modules/tags/account-tags?ref=v0.1″account_name = “devtest”
service_owner = “Ops”
requestor = “ops@Organization.net”
}

We do this by populating default_tags with the module which we defined in main.tf above.

provider “aws” {
region  = “us-west-2”
profile = “Organization-devtest”default_tags {
tags = module.account_tags.tags
}
}

How to Tag Your K8s Resources

 Here’s how to tag/ label your K8s resources via manifest files Label section.

apiVersion: v1
kind: Pod
metadata:
name: front-end-app
labels:
env: dev
app: nginx
service_owner: team-xyz
service_name: nginx

Shared Services Cost Attribution

Shared services range from taxes, support fees, credits, and databases to microservices. This could be a shared S3 bucket, an RDS database consumed by multiple teams, a K8s microservice used to process data, an EMR cluster that processes data, etc.  It’s a pain to build chargeback models with shared services.

Cost attribution for taxes, support fees, and credits

• These charges from the CSP are not split to accounts/projects; they’re usually charged at the payer account level with no granular attribution.
• A fair way to chargeback (allocate these charges back to engineering teams) is to split these charges based on a team’s cloud spending proportionally.
  • Example: If Team A spends 10% of the total bill, 10 % of taxes, support fees, and credits must be allocated to Team A.

Cost attribution for shared services like RDS and K8s (microservices)

Shared services chargeback often gets tricky, and it’s not easy. Every situation can be unique; there is no one-size-fits-all solution. Here are some pointers that can help address this problem.

A meaningful, quantifiable metric is vital in building a shared-service chargeback framework. Examples of such metrics could be:

• Time taken for the query to run: This metric can help build a chargeback model for associating costs to customers. This will help identify which customers are profitable.
•  Amount of data transferred/ processed: This metric helps chargeback costs on a system heavy on data processing.
  • A way to chargeback here is to proportionally distribute costs to users based on the amount of data processed.

For shared microservices, use systems metrics (% CPU utilization, % Memory utilization) together with # of API invocations and/or Network IOPS.

• Use a combination of these metrics to proportionally distribute resource costs such as EC2, S3, RDS, etc.
• You could use a formula that looks like this:
  • shared_service_cost =  cloud_resource_cost * (percent_distribution_of_metric1 * weight_of_metric1 + percent_distribution_of_metric2 * weight_of_metric2 +…)
  • cloud_resource_cost refers to the actual price of cloud resources such as EC2, S3, RDS, etc.
  • weight_of_metric needs consensus from within engineering teams and service owners
  • Use the metrics that make sense for your workloads. The proportional weight of metrics is a prerequisite to building this model.