I found myself unable to focus on thinking in the office, when there are other people present. I learned to compensate for that by doing my thinking in writing - I just start dumping my thoughts and talking to myself through my text editor. It feels a bit slower, but it does wonders for concentration and for short-term memory.
I like a text editor because I can type faster than I can write, but something about writing on a paper works better for me.
That said, I've recently started to go back to killing trees for things like daily checklists, as a physical reminder of stuff - and more "diagram" style doodling - all packaged into a three ring binder. At some point I plan to grab myself a document scanner so I can bulk archive my loose notes for convenience...
Easier said than done, but some unsolicited suggestions:
1) Acquire the self confidence that your results will speak for themselves, appearances be damned. 2) Realize your coworkers also value thinking, and recognize it properly. 3) Realize your coworkers are probably too busy worrying about their own appearances. 4) Any busybodies not too busy worrying about their own appearances can see through your unproductive busywork anyways ;) 5) Consider journaling / take stream-of-concious notes / mind-mapping. Might slow your thinking down some, but might compensate for memory some too.