As a child, my grandmother once accidentally used a poison oak stem to roast a hotdog over a campfire. She hadn't cooked it hot enough to denature the oil, and she reacted to it internally, which required a hospital visit.

Granted, consuming it is the worst-case scenario, but exposure to those oils can be life threatening.