If you give the attacker the salts, difficulty scales linearly. For two identical plaintexts, all that differs is the salt, but it is given to him.
If you store the hashes and salts separately (the technical details of which I know nothing about), then you augment the keyspace exponentially. For N byte salts, using P characters, you augment it by N^P. Furthermore, who forces you to append a salt? You could prepend. Or n-pend.
I'm just throwing ideas around.
Edit: Or one could move just the salts into the device and store an index into them in the password file. Hashing would be carried out by the device but an attacker would only gain access to the indices. Without stealing the device itself it'd be impossible to properly salt the passwords for hashing.
If a cracker gets access to the web app, they'll be able to monitor the exchange, but they'll need actual privilege escalation to be able to read/dump the whole hash database.
That one server/device is now a single point of failure and a bottleneck in processing logins.
You're also still relying on the security of a computer, just a special-purpose one.
And yes, it would not be possible to dump the contents of the devices.
It's also theoretically impure in any case, as you've done nothing but add an additional peripheral to the computer. You're seeking obfuscation, not real cryptographic integrity.
The only issue is cost of HSMs; they're about $20k/ea right now, since there are only two significant vendors, and they're not widely used.
If someone wanted to do "HSM for general purpose web login, to eliminate the DoS potential of scrypt, and the brute force hash db problem of anything else, and the idiocy of plaintext", the price could probably drop down to $500 or less.
It's not obfuscation. The peripheral has a far smaller attack surface than a server. This is real security, even if it comes at a cost of reliability (though one can envision ways of fixing this, too).
Stop trying to patch a hole that isn't there. Salt is not secret data. If you want to protect the hash with secret data, take A1kmm's advice and use the smart card to encrypt it. But don't call that a salt, because it fundamentally is not one.
What's making it hard for you to analyze this is that you're using the wrong terms. The secret stuff is password - the non-secret stuff is salt. If you want two secret bits, you want two passwords, or to break the password into two pieces for separate storage.
But after all that, you still want a salt, because you're not using it for secrecy but for ambiguation of identical plaintexts.
But splitting the password hash is a bad, or at best neutral, idea because passwords are likely to be semi-human readable and thus a guessable password that matches hash1 is likely the actual password, and will match hash2. This wouldn't be true if we used random passwords, but we don't. So splitting the hash is mostly totally ineffectual, as is having two separate hashes of the same string - the attacker usually doesn't need to examine both.
If you've read the article, you have a pretty good idea of how it works without salting. You have a list of N password hashes, and you come up with candidate passwords. You run each candidate password through the hash function, and compare the resulting hash to your list. If it matches any of them, you've got the plaintext for those user accounts. The key thing here is that you only have to hash each candidate password once, and you can compare that hash to as many accounts as you'd like.
Now, let's add in a salt. It's stored next to the password, so after stealing the password db you've got a list of password hashes, each with its own salt. In order to check a candidate password against a particular user account, you have to append that user account's salt to the candidate, then hash it and compare against the stored hash. That part isn't any slower. The problem is when you try to take that hash and compare it to other accounts. Even if two users have the same plaintext password, their password hashes will be different because they have different salts. The end result is that you have to hash each candidate password fresh for every user.