The Second Amemdent's in there because without it, the states wouldn't have been willing to ratify a Constitution that allowed the federal government to take their guns away. After all, they had just won a war against a monarch an ocean away; they weren't about to give their power of defense to an executive half a continent away. Remember, there was no guarantee a Constitution was going to happen at all; quite a few compromises and curveballs in that document are there because every single state had the option of rejecting the whole thing and being their own country. Many Founding Fathers envisioned defense of the US would look like individual states generally handling their own affairs, and a national army only being mobilized in the event of a national threat (however, practically speaking, that didn't come to pass... No sooner was the Constitution ratified than Washington led the Continental Army to force the citizens of western Pennsylvania to pay federal taxes).

Gun regulation was properly recognized as a state authority for most of the country's history prior to the Civil War and varied wildly from state to state, with many being significantly more restrictive than modern law. It's only with the passage of the Fourteenth Amendment (and the Due Process clause, "No State shall make or enforce any law which shall abridge the privileges or immunities of citizens of the United States; nor shall any State deprive any person of life, liberty, or property, without due process of law") that the Second Amendment ties the hands of state governments in the same way it tied the hands of the federal government. This restriction has been re-interpreted over the decades, but broadly speaking that's how we got from a 1700s Philadelphia that banned gun ownership for some immigrants to the modern interpretation where even tracking who owns a firearm might be considered an abridgment of the right of ownership.