The challenge is then you lose accountability, and the ability for people to work on what they are weak at.

There's a good argument that we shouldn't attempt to work on our weaknesses but that we should work on our strengths instead. I don't have a link but I've seen reference to this in research about performance reviews.

I've seen similar. I kind of view it as "A couple As and a couple Cs are better than all Bs" The one caveat is that you don't want any Fs. For example, to be a great programmer, you don't need to be a great English writer, but if you can't communicate at all, your work will suffer. There's a minimum level after which it's better to focus on programming. If you're a great accountant, you don't need to be a master of spreadsheets, but you need to be good enough to get the job done.

How do you know if the soldiers you have assigned to a project are any good without testing them?

Test the project.