https://github.com/cloudfoundry/bosh-lite
https://github.com/CloudCredo/dockpush
You download a BOSH release for your infrastructure, tweak the manifest for your chosen cloud, and push the deployment out. If you make a change, you just push it out.
Cloud Foundry itself (the elastic runtime) is a BOSH application that includes an HAPRoxy load balancer, dynamic router, and allows you to easily push/scale the stateless micro services and declaratively wire them up with other services you've deployed (with CF or Bosh).
It can run on AWS, vSphere, OpenStack, or on your own Vagrant VM in containers (bosh-lite). Or you can use Pivotal Web Services public cloud: http://run.pivotal.io
BOSH requires some wrestling early on but I find it to be a fascinating project.
https://github.com/tim-group/stackbuilder https://devblog.timgroup.com/2012/12/20/towards-a-rich-domai...
The documentation is pretty rubbish, but here's an example config file:
https://github.com/tim-group/stackbuilder-config/blob/master...
It uses libvirt for provisioning, and Puppet for configuration. The coupling is extensive but not tight. It's also coupled to a specific way of deploying Java apps. There's a sort of backdoor that would let you deploy other kinds of apps by writing some Puppet code.
I do think the basic ideas are good, though. Possibly more of an object of study than a tool to actually use!
Locally, you'd use http://www.fig.sh to describe and run your app. You can then import this description into Tutum and run it on e.g. DO or AWS.
Seems like the https://www.hashicorp.com/ guys would have something, perhaps Terraform/Atlas?
I have not done this yet obviously but am in the same boat as you for our company goals this year
There's still a point where it's beneficial to split an application into seperate stacks, in which case the manifest concept breaks down and you're stuck doing a lot of work yourself, but they can take you a fairly long way.
Do you mean like having a different "infrastructure manifest" for each micro-service? In that case, couldn't each micro-service specify which other micro-service it depends upon? For example, I could have a central "API gateway" service which would specify all the other micro-services it depends upon. This reminds me a bit of a Gemfile/Package.json but for processes instead of libraries. On the other hand, I can easily see how this could turn into a dependency hell.
Supplementing this with Hashicorp tools like Terraform and Consul can get you to the last step.
Depending on your concept of "deploy", I can point you to two options, both produced by the company I work for.
If you think in terms of controlling from the VM up, look at BOSH[1]. It's an IaaS deployment / management / update tool which has been used in production for several years.
You create two yaml files: a release manifest (describing the software components of your system) and a deployment manifest (describing the computers and networks of your system and how to map components to them).
This separation means that a single release manifest can be applied to many different actual deployments. At my day job I deploy a large PaaS release into a single virtual machine. My colleagues deploy exactly the same release into AWS with thousands of machines.
Systems defined using BOSH can be deployed to AWS, vSphere, OpenStack or Warden (a containerisation system).
For local development, the Warden backend enables "BOSH lite"[2] -- releasing and deploying into a virtual machine that hosts a local cloud of containers.
If you want to go the next level up to a proper PaaS, you can try Cloud Foundry[3]. It's defined as a BOSH release[4], which is how at my day job my co-workers and I are able to work both locally and at scale on identical software.
In Cloud Foundry we distinguish between stateless (apps) and stateful (services) software, and we provide a simple way to connect them. The easiest way to play with it is to log into Pivotal Web Services. IBM Bluemix is the same software and is also open to the public.
So for example, with Cloud Foundry, if you want ten copies of your app, you do this:
$ cf scale my_app -i 10
And it'll be scaled up to 10 instances.Need more RAM?
$ cf scale my_app -m 2G
Now each node gets 2Gb.And so on.
[1] https://github.com/cloudfoundry/bosh
[2] https://github.com/cloudfoundry/bosh-lite
Edit: Since this question is getting many replies, I'll be a bit more specific in what I am looking for. Is there a tool that would let me describe the infrastructure and deploy it on a given cloud provider but also have the ability to deploy the same infrastructure on my local machine (using VMs/docker) for development purposes.