News for hackers. Stuff that matters.
Re:Breaking a large AWS spend into understandable pieces by dberg on Thursday May 18, 2017 @12:52PM (#14368822)
We are about to open source our tool that consumes the billing files from S3 (via SQS notification) and then loads them into redshift auto handling the purging/updating.

Dealing with Amazon billing is absolutely infuriating. We are also on their EDP (Enterprise Discount) program and all of it is done MANUALLY. There is literally NO way i can check hundreds if not thousands of instance hours to ensure there was no error in their processing. Its complete madness. We use a dozen services and EBS, Bandwidth, inter-AZ bandwidth is impossible to audit.

That being said we have Looker sitting on top of Redshift, we make sure each "team" has a tag (infra wont spin up without it) and then we can easily set budgets, watch trends in Looker and track spend by team and by product name. Our finance team loves it.

The data is there, but Amazon REALLY makes you do the work. Their default billing reporting is useless.

Will publish on HN when our tool is open sourced.

Re:Breaking a large AWS spend into understandable pieces by fil_a_del_fee_a on Thursday May 18, 2017 @12:58PM (#14368889)
Please do :-) I just want to see cost by EC2 instances type.
Re:Breaking a large AWS spend into understandable pieces by arwineap on Thursday May 18, 2017 @04:09PM (#14370682)
You might be interested in netflix's opensource solution ice ( https://github.com/Netflix/ice )

It's been very helpful to notice and track down extra costs as they come in

Re:Breaking a large AWS spend into understandable pieces by JoshRosen on Thursday May 18, 2017 @10:12PM (#14372572)
Netflix ICE has a few quirks but overall it's a pretty useful free tool.

Unfortunately, the project has been largely abandoned and the current version in the Netflix GitHub repository won't work out of the box for many companies due to lack of support for new AWS regions, instances, reservation types, and services, as well as showstopping bugs (such as https://github.com/Netflix/ice/issues/210). In order to effectively use ICE today most users will need to maintain their own patched fork.

I'm currently campaigning to create a community-maintained version of ICE, with committers from multiple organizations, in order to revive the project: https://github.com/Netflix/ice/issues/240#issuecomment-29960...

Re:Breaking a large AWS spend into understandable pieces by jpgvm on Friday May 19, 2017 @12:46AM (#14373088)
Heres to hoping the ICE team releases their new tool.
Re:Breaking a large AWS spend into understandable pieces by mrtron on Thursday May 18, 2017 @01:19PM (#14369084)
That is available in the cost explorer...
Re:Breaking a large AWS spend into understandable pieces by dberg on Thursday May 18, 2017 @02:58PM (#14370026)
Cost explorer is EXTREMELY limited.

If you want to get good information by Tag, By Product, By detailed Date Range, etc you need it in a DB.

Re:Breaking a large AWS spend into understandable pieces by user5994461 on Thursday May 18, 2017 @03:08PM (#14370130)
The cost explorer gives you information by tag and/or product. You really don't need a DB.