Google Spreadsheets is where it often fails to work perfectly - hard to get it to take me to the spreadsheets homepage if I don't remember exactly which sheet I want to open.
Nevertheless, they don't let themselves be caught up by this minor obstacle and use whatever random reward function they can implement with the data they have.
The resulting system won't actually learn to solve the original problem - but it will learn something, so, hey, it's self-improving!
See also: Probably every single recommender system in use. (At least that's my subjective impression)
For me (and relevant to TFA) this is what is often missing from such learning engines; an option to manually intervene when necessary. It's hubris to assume that the heuristics will always get it right, and I would have to imagine there's valuable data in seeing what manual corrections a User makes, so I'm not sure why it's not included more. A .conf file, the "add/remove" weight as mentioned before, just some way for the user to correct the heuristics when they get it wrong. And they will get it wrong, and frequently do.
This is the part where Apple apologists get the whole “Siri/Spotlight is bad because they do everything locally and they respect your privacy” thing wrong. Nothing about this problem needs to be done “in the cloud” - their local algos need work.
But "fi<enter>" never launches Firefox anymore because of that one time you accidentally launched Finder instead. And since you keep accidentally launching Finder that way, it never learns that you want Firefox.
So you try "f<enter>" but you accidentally launch Flux. And because you keep forgetting it thinks "f" = Flux, it always launches Flux.
So you have to use "fir<enter>" to launch Firefox. But "fire<enter>" somehow launches Firewatch, a game you forgot you had installed on your laptop.
It's the most obnoxious system ever where each additional keystroke reorders the entire fuzzy result list in the name of "learning." It really is the opposite of predictable.