'Audio files that you download from the internet' could be absolutely anything - a song, an opera, a news broadcast, a test signal, a play... tells you nothing. 'Podcast' gives you a general idea of the medium and intent. That's useful.

Otherwise we might as well call everything in the universe 'stuff' if you really don't like nouns that may have a slight overlap.

One is a subtype of the other. Would you insist on using "radio broadcast" instead of "morning show"? Aside from that, a podcast is still a podcast regardless of whether you downloaded it, streamed it, or distributed it over media like flash drives and compact discs. That's why it's useful to have a new noun.