Simply put, you need to have some assurance that the binary that you're downloading is the one that the developers compiled, and the easiest way to do that is to have crypto checksums. If you don't, all bets are off.

And it's not just a "possible threat" but a likely one, and it's happened multiple times in the past. Sites get hacked all the time, and trojans are standard practice these days (unlike compromised-binary-generating C compilers):

https://threatpost.com/en_us/blogs/savannah-gnu-site-comprom...

http://www.esecurityplanet.com/trends/article.php/2248811/GN...

https://threatpost.com/en_us/blogs/apache-site-hacked-throug...