(1) Where the head is. (2) If you program it right, where the end of the tool is. (3) If you program it right, what the width of the cutting edge of the tool is.
That's it. You could maybe prevent it from diving the tool into the table. But you won't prevent it from trying to take a 2" cut into D2 steel. Or going down into a pocket and crashing the tool holder into a section of the workpiece. Or going down into a pocket and rapid moving left, slamming the tool into the workpiece (which if you're lucky will just break the tool).
... And, if it's using steppers, it only knows that much if it hasn't slipped/missed/lost a step.
On stupider machines that are only 10s of thousands of dollars/euros/etc, you also have to be certain that what the machine expects for tool geometry matches reality (meaning that the correct tool was in the correct tool location, that the tool is mounted in the holder at just the right length, that the tool itself hasn't deformed, etc.
That said, I wouldn't bet against AI stuff potentially being good at generating tool paths in the future.
Sometimes I think that existing CNC stuff might be too dumb. Where is the lathe equivalent of auto-probing a 3d printer bed or using machine vision to monitor the print? Where is the mill equivalent of visual examining each part like a pick and place machine does? Why can't I put a piece of metal in a vise and just tell the mill to square of the ends? It could be that what I'm picturing is only stuff that casual users would be interested in, and thus there isn't sufficient money to develop it.
Also let say that the use case are a bit different. a 3d printer is a far far far more controlled environment
A CNC starts with an unknown block on the table, held by an unknown workholding fixture, made of an unknown material. There just is not enough information to not crash into something unless programmed around it. Or just run way too fast and destroy tools.
In theory you could design sets of rules, but now you're having to add so much specificity to a design it's a big time waste. There's not really any meaningful entry level CNC machines, so if you can pay for the big machine... you kind of can pay for the expertise to not destroy the machine.
It also often works better because doing small parts can overheat the plastic. Moving between copies lets things cool down.
But anyway it was just an example of a scenario where the slicer might not have enough information to know how to not crash the print head.
That 'full conviction' part is the kicker. I don't mind if an AI says "gee, this might work", or "maybe this?" But the confident idiocy is going to kill someone. I asked one what the best breed of dog was for a family with a child who had allergy problems and was sensitive to dander. It said a golden retriever was best, since they are hypoallergenic and don't ever shed, plus they're friendly and eager to please.
(I have no idea how feasible that is, I'm just curious.)