>Is it possible that it doesn’t serve YTs interests to have a recommendation engine that works best for the end user?

100% absolutely it is similar to Goog's search

>I honestly think they just experiment with it a lot. The variety of videos that are recommended for me don’t seem to follow a pattern.

Possibly, but I'd posit that it's not a good system if the experiment is causing a lack of consistency. I'm sure there's a lot of people that would opt-in for a exeperimental reco system if just asked.

> Maybe they know my brain better than I do.

Now you're just being silly.