Well, bureaucracy is half French and half Greek. Bureau being the French word for "desk", imported into English to refer to the metaphorical desk. The "-cracy" part is from -κρατίᾱ, a derivative of κράτος (power) which ends up denoting a form of rule (see also autocracy, plutocracy, democracy; all of these purely Greek).

The key to good English spelling is the meticulous study of etymology; I learned this when I was about five years of age. Once you get used to thinking about words in terms of their root languages, then things start to make sense most of the time, except for all the exceptions.