you can do
SetOptions[EvaluationNotebook[], DockedCells :> {}]
and there are options to remove the window frame, etc. but i think if you are using certain licenses the toolbar may not disappear. and the main toolbar never disappears, to my dismayI think it was a default. Or, at least, with one formatting toolbar which I simply disabled. I never used palettes of special symbols: it was easier to remember some common shortcuts via "escape" key (e.g. Euler constant is <escape> ee <escape>)
How did you set that up?