News for hackers. Stuff that matters.
Re:Terraform 0.9 by jbkkd on Saturday March 18, 2017 @05:47AM (#13900626)
I'm guessing there is a finite number machine types, and they are well defined and do not change often?

What you could do is generate a different packer configuration for each of those types (guessing you use packer since you mentioned AMIs), which will create​ a different image for each machine type. You can scale each type individually.

Happy to help if you need specific advice.

Re:Terraform 0.9 by arjie on Saturday March 18, 2017 @06:13AM (#13900670)
Ah, I wasn't terribly clear. Each node is actually the same machine type. It's something like Voldemort with partition-aware routing at the client[0]. Essentially, there's a small amount of configuration to determine which partitions a specific node should serve.

Do people usually just build k AMIs if they have k partition groups that they want to serve?

For a sort of simplistic version of the problem, imagine the data was splittable into 6 partitions and you hashmod the partitions across 3 groups. So instances in group 1 will serve partitions 0,4; instances in group 2 will server partitions 1,5; and so on. Does one build an AMI for group 1, one for group 2, and one for group 3? Does one provide configuration informing which group a specific node should serve through other means? Or does one stick that in a Terraform definition?

0: https://www.project-voldemort.com/voldemort/design.html

Re:Terraform 0.9 by jon-wood on Saturday March 18, 2017 @06:55AM (#13900756)
Personally I'd build a single AMI with a startup script that extracts any configuration for things like which partition group the instance is in from user data (http://docs.aws.amazon.com/AWSEC2/latest/UserGuide/ec2-insta...)
Re:Terraform 0.9 by arjie on Saturday March 18, 2017 @07:44AM (#13900859)
Thank you. It seems to me like you'd need variation in your Terraform file to specify that. How do you do that? The DSL doesn't have control structures.
Re:Terraform 0.9 by arjie on Saturday March 18, 2017 @10:06AM (#13901296)
I should have looked at this more carefully. You can use `element` and `count.index` to walk across a list of things.