Stuff like BPA is not acutely toxic. There are some concerns about long-term ("subchronic") exposures spanning a decade or more, and even there, there is basically no clear evidence of adverse effects on humans.
Besides, BPA and its ilk are a concern chiefly with a variety of fancier, transparent plastics. Food-grade HDPE and polypropylene jugs are of relatively little concern. They are just not particularly pretty, so they don't sell.
Steel and glass are two other options, although many steel bottles are lined with epoxy or other coatings. Plus, in a car accident, I'd rather have a soft HDPE jug flying around...
Probably still OK in an emergency though.
Plastic also has the advantage that you can use it to sterilize water. Glass blocks too much UV, but a couple of days in the sun will disinfect water in a clear plastic container. You can have safe water indefinitely just by rotating containers (at least here in the Pacific Northwet, where availability of some kind of fresh water is more or less assured).