Seems like there should still be enough context. A user risks lost workflow if they've done nontrivial form entry. Exactly where you set that bar may be debatable or measurable, but certainly there is a difference between a partial search query and a full address form, for example.

Alternatively, the browser could just play it safe. I recall at one point the old Opera would prompt if unsaved form edits were present, on any navigation attempts (clicking away or browser back button/backspace). This saved me on many occasions and I never remember being annoyed by over-notification.