We keep coming up with these layers of abstractions around stuff that at the bottom is essentially already stateful but just not in a useful way. Doing it right from the ground up might help. Of course, left to the usual suspects, this would just turn into another Frankenstein blackhole of devops time. The usual suspects are billion dollar corporations that thrive on layers of complexity.
This is how we do our initial terraform for new projects. We build up what is missing by hand, then export the terraform to save time and effort in building our own internal templates. We throw away most of it, but it is very useful for not having to look up how every aspect is named and what options are needed.
0: https://docs.public.oneportal.content.oci.oraclecloud.com/en...
Edit: and actually, without a state there is no concept of a remote state. All codebases talk to the same "service" so circling a bunch of infrastructure and saving it to a specific git project would be easier than connecting to a bunch of remote states and reading specific resource days from them. Even if you're code repo doesn't declare a resource, the omniscient IAC service "knows all"
I believe, Terraform needs a mode that removes unmanaged resources, but that's quite the big ask and would result in yet another API / SDK change that providers would need to implement.
What if terraform was able to perform a complete audit of your environment for each provider and say "this is what you've got, mate", then give you options for each resource:
1. Accept (turns it into terraform code)
2. Reject (removes the resource)
Taken further, this idea effectively means that manually creating something like EKS could be a way to auto-compose Terraform code and store it in git.
Pros:
* You have the options of writing code, using the console or using a script or binary to perform a complex setup
* You have 100% code coverage once Terraform compared code with reality and forced you to choose
Cons:
* Auto-generated code is shitty.
* Slooooooow without caching (which is only one of the functions of state)
To make these concepts work without a remote state you pretty much have to run your infrastructure tool as a service, and implement things like:
1. Polling of providers on an interval
2. Alerting when something changes
3. Eventually consistent cache that can be forcibly updated in the UI or during sensitive operations
4. "Infrastructure Composer" UI, like a vastly simplified IaaS console that only shows deployed infrastructure and lets you group infrastructure by service, and let you define code-level modules
5. Show discovered dependencies and ordering, and allow you to explicitly set them (these should be stored as tags on the actual infrastructure components where possible)
6. Use some hot AI to group code-level resources, compare their configurations and generate config data to apply to resources/modules using for_each loops, while respecting service boundaries and offering advice on when to refactor.