I noticed that before on this subject: I agree with you and said that before on the subject of AV on Linux and got downvoted. Not claiming there are not viruses on Linux, just saying that, in roughly 20 years of using it, I have never seen one on my servers or workstations. So why would I sacrifice speed?
For workstations I think Linux people are more careful than others (I have no citation just personal experience). I do not need warez and if I have to browse something fishy I do that from a VM or another account; not 100% safe but a lot safer than the entire mess people seem to have on Windows boxes (incidentally it has been around 20 years since I last worked with Windows seriously too).
On the flip side, I have 4 infected mac's being flagged by webroot right now because the users downloaded and ran things like fake flash installers. So "every day" users do dumb things on macs and windows. For those user, the AV performance hit is worth it (to me) :)
I'm vehemently against running AV software on Linux at the enterprise level, too. The best I've been able to do there, though, is to ensure that on-access scanning (aka "slow the system down to 10 percent of its usual speed for no reason at all") is completely disabled.
Edit: any explanation for the downvotes?