News for hackers. Stuff that matters.
Re:Ask HN: What weird or hard problems are you trying to solve? by jeremylevy on Sunday July 05, 2020 @05:19PM (#23742097)
Automating cloud architecture creation.

I'm building a "catalog" of architectures that you could use to create a complete cloud architecture on your AWS, GCP or Azure account in less than one minute.

So, for example, you could create a docker-based architecture with CI/CD, auto-scaling, zero downtime deployment, SSL, load-balancing, high availability and MongoDB in less than one minute in your own AWS account.

It's like Terraform with the user-friendliness of Heroku.

It's very hard because every providers have different APIs and concepts so you have to start from scratch for each.

I love working on it because cloud-computing may have so much impact in some organizations like biotech startups or NGOs.

Re:Ask HN: What weird or hard problems are you trying to solve? by busted on Sunday July 05, 2020 @06:31PM (#23742668)
I've thought about this in the past. There's a modern way to build architecture, and cloud providers are in general only providing the building blocks and requiring you to put them together. It's unscalable to expect every company to hire someone smart enough to construct a good architecture (let alone the time) but at the same time, people who have been working on infrastructure long enough know that there's really only a handful of useful architectures that solve 90% of problems. I thought of it as CTO-as-a-service or CTO-in-a-box.

Service meshes like istio.io start to solve a portion of this.

Almost certainly there is endless complexity but I bet you can come up with something useful. Good luck!

Re:Ask HN: What weird or hard problems are you trying to solve? by jeremylevy on Monday July 06, 2020 @06:30AM (#23746384)
Thanks!

> people who have been working on infrastructure long enough know that there's really only a handful of useful architectures that solve 90% of problems

I totally agree with you on that point.

Also, given that architectures are created on your own cloud provider account you could easily add the missing parts.

Re:Ask HN: What weird or hard problems are you trying to solve? by tlrobinson on Sunday July 05, 2020 @05:56PM (#23742423)
> It's like Terraform with the user-friendliness of Heroku.

> It's very hard because every providers have different APIs and concepts so you have to start from scratch for each.

Why not build it as a layer on top of Terraform?

Re:Ask HN: What weird or hard problems are you trying to solve? by jeremylevy on Sunday July 05, 2020 @07:28PM (#23743096)
I do use Terraform. Terraform is very good to create the building blocks of your architecture but not so much for the user-friendliness part.

Let's take an example: You have an architecture with a CI/CD. Great. You add your CodePipeline and CodeBuild ressources to your plan. Perfect.

For the CI part, you want your build to start on every commit on every non-master branch. Bad news: CodePipeline doesn't have support for multi-branch. So you will need to find a way to clone the default pipeline for each new branch.

Repeat this for all the user-friendly features (GitHub check runs, env vars management, deployment monitoring/rollback...) for the three cloud providers for all the different architectures and you start to feel the difficulty (I can testify ;)).

Re:Ask HN: What weird or hard problems are you trying to solve? by tlrobinson on Sunday July 05, 2020 @07:52PM (#23743273)
I’m suggesting your tool could generate Terraform code and run the Terraform command “under the hood” so you don’t have to reimplement everything Terraform already does.

Users wouldn’t need to know about Terraform (unless they wanted to. It might be nice to have a way to export the Terraform code)

Re:Ask HN: What weird or hard problems are you trying to solve? by jeremylevy on Sunday July 05, 2020 @08:19PM (#23743449)
Sorry! My answer was certainly not very clear.

I use Terraform “under the hood” for this project.

What I was trying to explain is that the "hard part" is not the creation of the architecture components but the user-friendly features.

Re:Ask HN: What weird or hard problems are you trying to solve? by threentaway on Sunday July 05, 2020 @06:39PM (#23742729)
This was my first thought. Terraform modules can accomplish all of this.
Re:Ask HN: What weird or hard problems are you trying to solve? by anewlanguage on Sunday July 05, 2020 @08:03PM (#23743344)
I've often found myself wanting something like this, a bootstrap to spin something relatively complex up quickly and then start customizing it. I think it would be nice if there was a way to export to the format native to whatever cloud you're targeting, e.g. CloudFoundry. This is something I've found missing in tools like Serverless Framework.
Re:Ask HN: What weird or hard problems are you trying to solve? by jeremylevy on Monday July 06, 2020 @06:18AM (#23746324)
Thanks!

For the export feature, It's something I've been thinking about for a while.

Given that I use Terraform under the hood, I've been thinking about a feature to merge the base architecture plan with your custom plan.

So you could have a catalog of "base" architectures and another catalog of "custom" architectures that you could spin up in seconds.

What do you think about it?

Re:Ask HN: What weird or hard problems are you trying to solve? by hashamali on Sunday July 05, 2020 @05:29PM (#23742207)
Have you considered something like Pulumi for the glue?

Love this idea, the ease of Heroku + the flexibility of Kubernetes would be amazing.

Re:Ask HN: What weird or hard problems are you trying to solve? by jeremylevy on Monday July 06, 2020 @08:39AM (#23747239)
Thanks!

Yes! I used Terraform because it was the tool I know best but Pulumi seems very elegant.

Re:Ask HN: What weird or hard problems are you trying to solve? by lapnitnelav on Sunday July 05, 2020 @08:33PM (#23743540)
This so much.

One of the main reason the big players dominate the Web (in the marketing space) so much is because SaaS is easier to handle (for most people) than having to deploy and maintain your infra.

if you streamline the process so that your regular SMB can build their digital infrastructure and maintain their private data clouds, it would go a long way to help us all with privacy.

A suite of open-source / on premise license one stop shop.

Re:Ask HN: What weird or hard problems are you trying to solve? by jeremylevy on Monday July 06, 2020 @06:04AM (#23746254)
Thanks! I hope I won't disappoint you.
Re:Ask HN: What weird or hard problems are you trying to solve? by jonwalch on Sunday July 05, 2020 @05:44PM (#23742332)
Any thoughts about maintenance? My personal journey with learning AWS is/was very painful, but now that I understand everything that I need at a high level, it is quite a bit more manageable. I can't imagine having to make minor tweaks to my infra if I had no idea what it was doing.
Re:Ask HN: What weird or hard problems are you trying to solve? by jeremylevy on Monday July 06, 2020 @05:41AM (#23746154)
It's something I've been thinking about for a while.

I think it will depend on the people that will use it.

Some architecture require more maintenance than other.

For example, the first architecture that I've created is a serverless architecture based on Docker. Under the hood it only uses managed services like Fargate, CodePipeline, CodeBuild, Aurora...

Given that you'll have a catalog of different architectures, people that don't have any experience with AWS may need to choose architectures like that.

Also, each architecture contains a detailed Readme that explain the basic operations (updating ELB HealthCheck, increasing the amount of RAM that could be used by a Fargate task and so on...).

It's a complex subject, by the way, so I am entirely open to any comments, ideas or suggestions.

Re:Ask HN: What weird or hard problems are you trying to solve? by mattjoyce on Sunday July 05, 2020 @06:11PM (#23742545)
That does sound challenging. APIs and service change too. I can't imagine how complex that can get if something stops working.

Still when you succeed perhaps the next goal is to simply specify what sort of compute function, the scale and risk tolerances, and let the patterns be assembled accordingly.

Re:Ask HN: What weird or hard problems are you trying to solve? by jeremylevy on Monday July 06, 2020 @06:02AM (#23746242)
Yes, you're right.

Even if you can lock the API version of the SDKs, you will still need to update all the architectures with the new features and changes.

It's very challenging but I love working on it!

Re:Ask HN: What weird or hard problems are you trying to solve? by derision on Sunday July 05, 2020 @07:54PM (#23743283)
Check out gruntwork.io
Re:Ask HN: What weird or hard problems are you trying to solve? by jeremylevy on Monday July 06, 2020 @05:15AM (#23746032)
Gruntwork is very good to generate your infrastructure as code, but in my opinion it misses the "user-friendly" part.

What do you think about it?

Re:Ask HN: What weird or hard problems are you trying to solve? by Addono on Monday July 06, 2020 @03:17AM (#23745479)
Reminds me a bit of https://juju.is/. Haven't looked at it much, but it seems super powerfull and tries to solve at least some of the issues you mentioned.
Re:Ask HN: What weird or hard problems are you trying to solve? by jeremylevy on Monday July 06, 2020 @06:33AM (#23746401)
Thanks!

I didn't know Juju, I'll try it out!

Re:Ask HN: What weird or hard problems are you trying to solve? by ahnberg on Sunday July 05, 2020 @05:23PM (#23742139)
Do you have something published yet? Code, articles or anything? Sounds very interesting!
Re:Ask HN: What weird or hard problems are you trying to solve? by jeremylevy on Monday July 06, 2020 @05:09AM (#23746008)
Thanks! I've released an alpha version three days ago.

I'm not very comfortable using a comment for promotional purpose so if you would like to try it out, drop me an email at [email protected], I will send you the link.

I will try to make a Show HN post in the following weeks.