I have experimented with those kneeling things that have no back, and to my surprise they seem to work ok. I find it best to swap back and forth between on those and a normal chair.

In general, I find hard straight chairs better than the more common padded office style. I think the periodic shifting and squirming you do keeps your back from setting up in a bad way. Wooden kitchen chairs are cheap to come by if you don't care about matching them, but one disadvantage is that they don't slide around as well.

I am currently using a wooden rocking chair for all my office chair needs. It's easier to slide around (I glued felt strips to the rockers). If I were setting up a new office now, I would get only rocking chairs.