Same experience here, moved from cypress to testcafe after getting frustrated by cypress API.

It was some time ago so I may not remember correctly but I felt like cypress was not for people already fluent in javascript, a bit like jquery but pushed even further (you need to do everything the cypress way now, you can't even use variables or promises...).

Doesn't it force you to use some form of page objects? That stopped me from using TestCafe didn't like that

No, page objects (AKA page models) are a recommended but optional pattern in TestCafe.

We use them on my team because they make the tests more readable and keep us from having copies of the same selectors in multiple places, but you can easily use TestCafe without them.