No it won't if it's in a properly managed landfill. This is the misconception that anti-plastic fanatics have. Plastic doesn't kill fish when it's trapped underground forever.

Straws in the ocean were dumped there by careless people. Since you care about the environment, there's no way you're using a paper straw so you can continue dropping it into the stormwater drain without the problems that you used to cause when you dropped plastic straws into the drain. The very people who choose paper straws are the ones that can't make any difference because they weren't the problem in the first place.