How to waste money in cloud

Expense optimization is often the main reason for migrating from on-premise to the cloud. The combination of pay-as-you-go and flexible provisioning reduces the problem of overestimated and overprovisioned compute resources. However, in order to actually reduce infrastructure bills, one has to fully understand the cloud pricing model. Otherwise, invoice total may be a huge surprise.

The AWS cost is driven by four metrics:

  • compute power & memory,
  • storage,
  • processing time,
  • data transfer out.

However, those are just general guidelines. Using AWS cost calculator and reading through pricing tables should not be omitted. All three scenarios below are real-world examples that happened when using the cloud without estimating the costs.

S3 endpoint – public or private?

VPC gives one ability to choose between deploying into a public or private subnet. In case of the latter one, VMs get access to the internet via NAT Gateway, which is charged by hour and data transfer.

Having said that, let’s estimate how much would it cost to send 1PB of data from S3 to a private subnet? If one does not use S3 endpoint and go with VM –> NAT Gateway –> public S3 endpoint path, then the NAT GW charge for data transfer will apply: 0.045 x 1024 x 1024 ~= $47k. The alternative, which is using a private endpoint, is free.

Further reading: https://aws.amazon.com/blogs/aws/new-vpc-endpoint-for-amazon-s3/ 

CloudWatch – paid per metric, not point.

Even though CloudWatch is one of the most essential services, it usually does not significantly contribute to the AWS bill. Metrics are charged per number of metrics and data retrieval API calls. Even if one has hundreds of VMs with tens of monitored metrics, the cost will be hardly visible among other expenses.

However, it also means that it may not be suitable for thousands of sparse metrics. Let’s estimate again: 10k time series, each one producing 256 bytes per minute or ~153MB per hour (10 000 x 0.000256 x 60). This would cost $3000 per month (10k x $0.3) for CloudWatch metrics only. For very basic comparison: one m4.large instance with 200G of storage costs $93 per month.

Word of advice

Both of the above situations resulted in a visible spike in the cloud spend. Luckly, in both situations there was budget monitoring and cost reporting already set up. This allows to react quickly, investigate the bill and identify troublesome services.

This example shows that a fully-cloud solution is just a little different than working on an on-premise one. However, estimating expected cost of the whole application, paired with monitoring cloud spent (day-to-day usage) greatly helps to keep expenses reasonable.