I turned off YouTube watch history entirely partially for this reason. I'd recommend it. The algorithm still tries to guess based on your subscriptions and likes, but it has much less data to work with.