It helps to be upfront about it. For most of the reasonably popular stuff I put up, I clearly say something like “if you have a small change, it’s probably best to talk to me instead of submitting a PR because I’ll just do it faster without the tiresome back-and-forth review dance; if you have a big change or a major feature, definitely run it though me first, and unfortunately I’ll likely say no in most cases” in CONTRIBUTING.md or the developer section of the website.
Of course, lots of people don’t read... But at least I feel a lot better closing big PRs I’ll never get to.
If I'm using an open source project and I need something changed, I'll usually just make the change and put the new code online, because I think that's how open source is supposed to work. Others can now benefit from my change.
I'll also make a PR to the original project. But I don't mind if it's rejected. I made the changes for myself. If you can use the changes, fine. If can't, also fine.
That said, while I generally do not mind fielding PRs or even trying to resolve bugs, the actual worst thing for me is having to close a PR because I simply don’t want to merge it. I know it’s the right choice, but it’s not fun.