From what I understand, the Go language team wanted to write robust servers, thus they wanted to "optimize for the case where the error can be handled."
But were programmers to consult the history of their compiler's output (representative summary: "ERROR ... ERROR ... FAILED ... WARNING ... ERROR"), it would be obvious that in fact errors are highly likely, even if you know what you're doing. So what makes the error case different from all the other cases?
You're quite right that exceptions promote giving up control to a higher level. And that's the right thing to do for exceptional situations. The ultimate higher level is the programmer who's going to have to fix the bug, or the admin who needs to fix the configuration problem - because they are the two main classes of errors for which exceptions are great. (The third class is a logical abort, and depending on the language and how good its idioms support exceptions, this may or may not be good practice.)
You get rich information about the error, including a call stack, the ongoing operation gets cancelled as soon as it occurs rather than muddling onwards and causing more damage. If you can foresee the error such that you can write an error handler, using the method that throws exceptions is probably not the right one - unless it's unavoidable owing to race conditions, or impoverished libraries.
Exceptions are ideal for things that require aborting and unwinding to a point - usually a loop, like a server request handler or UI event dispatcher - high up on the stack. If the general behaviour is not to abort and unwind, it's not a good fit for an exception.
It's also why exception handlers should be very rare. There shouldn't be any more than a half-dozen in most programs. Error codes that need checking and aborting by hand at every function call boundary are a poor fit for replacing exceptions, because they optimize for the case where the error can be handled, rather than the usual case for exceptions, where the error cannot be handled.