In the end I believe, the question is about lock-in and control. In a true "dumb pipe" offering, you could just take your software elsewhere and run essentially the same code without changes. Meanwhile, if you use various proprietary APIs, this is not the case. I'd argue, even if the APIs have open source implementations that you could theoretically use elsewhere, it's not clear how viable this would be in practice.
I think AWS does not become a "dumb pipe" only because you could theoretically use it like that - at least not if doing so would require you to go "against the grain" of the platform the whole time and would force you to only use a fraction of what you paid for.
If someone truly wants to not be dependent, you need to identify to what degree you want that, and then end up paying a lot more to get there, usually ending up multi-could. Even three 'dumb pipes' aren't similar enough to create a 'write once' IaC and application definition. The closest you can get is not using FaaS-type offerings and sticking to OCI containers. You'll still have to work with the IAM and network primitives each vendor requires, and even if you don't run active-active you'll still have to write every deployment to every provider you use to ensure your systems are always ready to deliver.
It's not impossible, but impractical and expensive. For most companies, the ROI just isn't there. Offloading more day-to-day and non-company-specific work is the way, and dumb pipes can't do that.
You can avoid all lock-in by ignoring all sorts of proprietary features of any vendor, and leave ENDLESS benefits and optimizations on the table for the hypothetical likelyhood that you might one day have to "free yourself".
Conversely, if you embrace those things wholeheartedly, and at some point find yourself wanting to break the relationship and do a full egress migration, even if your costs end up monumental, I would bet anything that over the time you were "locked in" you will have gained more from that cloud than whatever "one-time migration costs" you have to pay.
If your core business proposition is literally something like storage (e.g. Dropbox) where with time you will want to vertically integrate your whole technology stack rather than relying on a 3rd party, yeah avoid lock-in.
If you have a frivolous C-suite that will want to change cloud providers every 24 months depending on what discount they can negotiate on the golf course, yeah avoid lock-in.
Everyone else? Throw that word out from your vocabulary, and embrace the idiomatic capabilities of your cloud vendor.
If you have access to all the details of your systems you can always 'go elsewhere' but it becomes a matter of cost, knowledge and time. The balance is how much you want to spend on those, and what it gives you in return.
As an anecdote: I and the teams I manage avoid CDKs like the plague, they end up a version of IaC self-lock-in that makes it nearly impossible to integrate between vendors. Using tools like Terraform or perhaps Ansible or SaltStack (or Idem) still means you write your abstractions against a specific provider, but the code and data formats are standard enough that you can retain and re-use your knowledge and skills and depending on how much layering you apply you can reasonably easily migrate wherever you want to (from an infra perspective). Contents (unique data, that is) are always a different issue, even with full access you'll be thinking about egress cost. All of those issues are of course mostly annoying at scale, but that's also one of the reasons you'd go to AWS: for scaling options.
You can run your stuff on AWS without ever touching compute, storage and networking yourself, and you can also run your stuff on AWS by only touching compute, storage and networking and nothing else. You can guess your risk wrong and do everything as a root user in a single account, and you can create an account vending machine in a well-architected organisation with policies in place to prevent the huge amount of footguns lying around everywhere from going off.
There are so many ways you could be using it that trying to put a single label on it seems pointless to me. Just like the only constant you can count on is 'change', the only 'proper' usage or terminology in terms of AWS is what you know, what you need, and what you ended up doing. This includes shooting yourself in the foot with CloudFormation because you accidentally thought CDKs are your friend and JavaScript was the best tasting language for your goal, or thinking that clicking around in the console would be a good way to get your department of developers up and running.