Sure, everything you ingest affects you. But I think most people wouldn't expect drinking some clean water to cause them to hallucinate. Ingesting a chemical like LSD that is known to affect the brain in certain ways is different.

Put another way: all things are not equal when you ingest them.