Technically fudge is just a crystalline sugar candy with a certain water percentage. Milk is optional (and frequently omitted). Reese's peanut butter cups are fudge, for example.

Forgive me for asking but... do you have a source for that other than an LLM? Every search I've tried just confirms what I already thought.

This explains better what I mean. https://en.wikipedia.org/wiki/Fudge#Texture - milk is added only as a stabilizer, and many candies that resemble fudge in texture incorporate milk or other stabilizers to prevent too rapid a crystallization. Fondant is just fudge without stabilizers.