Do you really think that OP's man management skills could be so infallible that he's already getting the absolute best from this guy? If he's not a cultural fit, then it's distinctly possible that the things that motivate the others don't motivate him. Surely it's better to make sure that the employee is as motivated as he can be, and is given an opportunity to improve before just throwing him out of the door.
I would never again work in the types of places that follows the maxim you describe. I've worked in those types of places before, they're horrible, hollow experiences where everyone chants about how awesome they all are and pulls all-nighters instead of being with their families, just to hide the deep seated job insecurities underneath.
I've worked in the converse - hire-quickly, fire-slowly - and I found it a horrible, hollow experience. Nobody, from the top down, gave a shit about getting anything done or learning. This is a place that refused to allow generics in Java code because people might not understand them. There were enough C players across the ranks that had moved up the ranks just by being there forever that the culture revolved around ignoring problems and not making waves, and it nearly killed the company. I bet you've heard of them. The company was AIG.
You're absolutely correct that hiring and firing, like all management, is highly dependent on culture.
In North America, the overwhelming majority of research has shown that organizations that "hire slow, fire fast" have more positive work environments, with happier employees and lower job insecurity. You could literally fill a small library with the books and papers that have been written on this topic.
I would love to know more about your culture and the "hollow experiences" you're referencing.
By contrast, British companies tend to focus more on getting the job done, going home and spending time with the family. Sometimes you pull late ones, but you're not expected to be in the office at 9am the next day. In some places there's beer after work, but you're not expected to go (at the risk of being denounced as a 'non-team player').
To provide a comparison, working in France is completely different. In France the work-life balance is shifted further towards the life end, in the Middle East (if you're a local) even more so.
[1] - http://www.youtube.com/watch?v=wvsboPUjrGc - ok, that's an extreme example, but tone it down about 97%, crank the alcohol levels up by 40% and you're roughly where ISS was in 2001.
I've followed advice 2 and 3 before, but failed at advice 4 and 5. Will do that next time, especially 5.
Here's why:
You make it harder for other people to do their best at work and love their company when they have to work with mediocre people.
I say this as someone who has had to work with some heart-achingly frustrating situations.
Good luck with the hiring changes and the company.
Cheers.
"Hire slowly, fire quickly" is an oft-repeated maxim for a good reason.
As for the most humane way to fire someone in a startup, I have a few thoughts (borne from several misfires earlier in life):
1. Sit down with your employee and give them a formal warning. "You're not performing up to our expectations, and if things don't improve within 14 days, we're going to have to replace you." Give them a list of specific items to work on.
2. On the day of the firing, always have two people from your company present in the room. This mitigates a surprising number of "he-said/he-said" type disputes, and also (surprisingly) leads to a less contentious conversation. If it's just the two of you, the odds of someone saying something they'll regret increase substantially.
3. Be generous with severance. I've never regretted giving an employee 4 full weeks of severance, no matter how junior.
4. Be direct. Resist the urge to treat it like a high-school breakup (i.e. "It's not you, it's me.") I have always been advised by HR professionals to never mention "cultural fit", as it's a gateway to wrongful termination suits. However, I have been ignoring this advice for about a decade. I believe an honest firing trumps a legally "safe" firing.
5. Most importantly: don't plan on doing anything else for the rest of the day. Take your team out and order drinks. If your other team members want to go home after lunch, by all means let them. In a small team, terminations can be very emotional for everyone, and nobody feels good about working right after their coworker got canned.