Small government can also provide border controls and customs. Small, in this case might refer to the scope of the government's powers rather than the size of its budget or number of employees.

Reasonable limitations in this case might include a requirement that property seized temporarily be the subject of a criminal investigation, with a short, strict time limit for either filing charges or seeking an extension from a judge, who should be expected not to grant it without exceptional circumstances. Forfeiture of property could require a felony conviction, and if the felon isn't the owner of the property, additional proof that the owner knew it was being used illegally.

I have no illusions here: these restrictions would sometimes prevent the government from taking the assets of real criminals. That seems like a reasonable trade. The sort of criminal behaviors that civil asset forfeiture might be effective against are not such a significant threat to the stability of the USA as to justify taking the property of innocents with less legal protection than they would have in the criminal justice system.