If you are disciplined, it's ok to build on master, but once you realize you need a branch (3 or 4 commits in), it's really easy to deal with. You branch from master, re-checkout master, and git reset --hard back to the public psuedobranch. And it is as if you had a branch all along.
I play the same game with git svn sometimes, except the psuedobranch is the "stuff that has not yet been committed to SVN" and it's a lot more clear where the cutoff is. I won't pretend this is perfect, but it does work.
(The problem I have with branching off for absolutely everything is that I find it easy to end up with too many branches to remember what's what. Granted, I'm the team lead and I may be more prone to that than a team member with more defined tasks.)
You develop on master. if you need s 'stabilisation' phase, when you only do bugfixes you can do that on a release branch. Or if you usually deploy the latest you can just tag the version that you deploy. If you want to just fix something from the currently deployed version you can create a 'hotfix' branch out of the tagged release version, do the fix, deploy, merge the hotfix branch into master and delete it:
git branch hotfix v1.2.3
git co hotfix
# do the fix
git tag v1.2.4
# deploy
git co master
git merge hotfix
git branch -d hotfixCommits are what's fixed. Branches are basically just mutable pointers to commits.
You can at any time go back to it as long as you know the SHA-1 identifier for the head commit you want. Because of this, there's little reason to keep temporary refs cluttering the ref namespace.
If you really want to keep references to everything, a "git attic" tool exists which allows you to move branch refs out from the main ref namespace into a normally invisible (ie. doesn't show up in git branch etc.) namespace.
If my edits grow to the point that I need to write tests or push to staging, I just checkout a feature branch before committing.
If you tag each release, does git allow you to checkout by that tag and then start making commits to it / make a new branch from it? I could be wrong but I thought tags in git were basically just read-only and informational - can you start work from them?