I have a different question, if you don't mind.
Under what conditions or use cases, do you think it may make sense to migrate from ec2 to ecs? What are the advantages or disadvantages that one needs to take into consideration?
Thanks
I guess what I am asking is that what points should I take care of or give thought to if I decide to migrate from ec2 instances to ec2 instances inside of ecs?
Does it clarify now?
-----
I have just a very very shallow knowledge about these two services, so I'm not going to comment on them. Also, to make sure there are no missunderstandings, clusters of EC2 Instances and Fargate clusters are the things you put stuff in, like barrels, ecs is the tool you use to get water out of the river and put it in the barrels.
At this point, I don't really see any benefit on running 3rd party apps, like I assume logstash and the other one are for you, directly on the ec2 instance.
As a note, I would not run my own db server/cluster or redis or what have you, if I can avoid it. I would pay for it (and do atm, we are using RDS dbs).
The question is if you can benefit from using ECS to start docker containers or not.
You don't need ECS to run a docker container on an ec2 instance. All you need to do is ssh on the instance, install docker, docker pull and docker run. There, assuming you are building your own docker images with all the configs required, this is all it takes to start a new instance of your app. You can even automate some of this with a Launch Template. Two big advantages of doing this: fairly simple setup and might get cheap - if you prepay for your instance, say for 1 year or more, you can get huge discounts.
There is a drawback to this method, but it depends on your use case: you can't react quickly on spikes. The way to increase capacity, for this setup, is to simply replace the ec2 instance with a bigger one. Then put back the smaller one. Since you have to manually do this, I think is fairly obvious why this is a drawback.
So far, the assumption is that your app can only scale vertically, eg to get moar power you need a bigger server. There is the case when you can scale horizontally, eg if you need more power, you just spin up another instace of the same app and share the load.
If you can do this, eg scale horizontally - instead of replacing a t2.medium with a t2.2xlarge, you simply start 3 more t2.medium and all 4 can cope with the load, then shut the extra 3 off when the spike is over (I have no idea if 4 x t2.mediums can do the job of a t2.2xlarge, just giving an example). You can manually do this or automate it via autoscale/CloudFormation or even your own scripts that read crap from CloudWatch and use the api to do things.
You noticed that I didn't say when to use ECS yet and this is because ECS is the tool you use to automate most of the above. The truth is you can live a happy ops guy life without ever touching ECS, or EKS, or GKE, or K8, or chef or ansible or what have you. But knowing how these tools work, this is going to make your life soo much easier.
EKS and GKE are the "managed" k8 solutions offered by Amazon and Google. Not sure how much of the management they do here, never researched the subject, the advantage being that your automation that uses the k8 "language", can work with both. I repeat, I don't know the low level details, eg how do you get some resources in such a cluster.
ECS is the pure AWS alternative for K8. Eg, is a software solution to manage docker containers. Is not compatible "language" wise with K8 tools afaik. But, surprise surprise, is very well integrated with a host of AWS services.
ECS can work with two different types of resources, EC2 clusters and Fargate clusters.
EC2 Clusters are sets of EC2 instances, you start them in whatever way you want, make them as big or small as you want, you just need to run the ECS Agent on them. If you use the Amazon AMI optimised for ECS, this is trivial. The drawback is that you need to manage these instances, software update, you need to start new ones if you need more capacity, stop them when you are wasting resource etc. Obviously, AWS has a bunch of other services you can use to make this easier (CloudFormation etc). The advantage is that you can customise the AMI if you need to and you can leave data behind, after a container terminated. For example, all my tasks in this particular ECS Cluster need access to the same S3 bucket of data. So I'm precaching it on the machines and then simply mount a volume in each docker container that needs that.
A Fargate cluster takes away all of this complexity, you just need to specify a name. Then you tell ECS to start tasks/services (services are just tasks that need to run in perpetuity) on the Fargate cluster and the thing will scale up and down, and scale your price up and down, as you need it. Two drawbacks: you can't customise the underlying host and you can't have persistent data left on the host after a container exits. Therefore, the project I'm working on atm would end up costing more to run, since I can't cache the s3 data and every container that needs that data will end up re-reading it from s3.
Best I could do :)
/edit: GKS -> GKE