There's definitely better ways of doing this. Like the idea is good, but it is very confusing when it happens automatically. A simple UX fix might be to show the relevant results, and then have a spacer with message like "we didn't find a lot of good matches, but maybe these will be useful".

Immediately a lot less frustrating and confusing.