You do need to guess things like activation functions, number of attention heads, order of attention layers, etc. Often the parameter names reveal something about these.