Reinforcement learning is not strictly what is considered supervised learning in ML, but it's very much in the same vein. And a supervised learning algorithm doesn't have any "knowledge" of the domain it's learning about either - it just adjusts its parameters based on training example and class/output pairs. RL attempts to find the best actions to take to ultimately maximize a measure cumulative reward, i.e. a signal which provides an objective measure of performance (much like the class or target output of a training example used in supervised learning).

RL is definitely not unsupervised learning, which in contrast, attempts to find some structure in unlabeled data.

Knowledge that the score is the score, is knowledge of the game. Deep reinforcement learning uses the score as an objective measure for how well it's doing. That's is the definition supervised learning.