In looking at rat studies a couple weeks ago, my napkin math said that it would take a human between 1 and 10 KILOgrams of LSD to cause death. Tartaric salt may be more toxic that the LSD bound to it. Most people would struggle consuming this much before entering a catatonic psychedelic state, and few have access to these quantities: street value would be ~$10,000,000.

There are certainly risks of engaging in dangerous behaviors while intoxicated, and possibly risks of precipitating latent psychotic predispositions, but its really amazing how non-toxic it is in the physical sense.