Yes they should. Either they are arresting you and the gun gets processed like everything else, or they aren't arresting you and thus need a warrant to seize anything from you. If they don't have enough evidence to arrest you for the crime then they don't have enough evidence to seize the gun without a warrant.
As for whether they should ever be able to seize a gun without arresting you, I'll defer an opinion--I feel like my knowledge of the variety of cases isn't up to snuff.
"There have been 61,998 cash seizures made on highways and elsewhere since 9/11 without search warrants or indictments through the Equitable Sharing Program, totaling more than $2.5 billion. State and local authorities kept more than $1.7 billion of that while Justice, Homeland Security and other federal agencies received $800 million. Half of the seizures were below $8,800."
http://www.washingtonpost.com/sf/investigative/2014/09/06/st...
Temporarily.
If they're not going to charge you, they should return the gun.
The alternative is to declare "gun free" zones near the location of all gun crimes. Not sure who did it? No problem! Take everyones guns.
The problem is that the courts have allowed cops to twist that "good reason" to the point where it is a blanket right to search anyone, any time, and steal their property. That's the abuse of what ought to be a relatively narrow exception.
[1] https://www.law.cornell.edu/wex/exigent_circumstances
[2] Note that this point isn't about gun control (whether you like or hate gun control). It's about someone who carried or used a gun while committing a crime, which is an entirely different beast!