1. It creates a coherent syntax for making names visible that works both for member variables and package variables; you don't ever have to wonder how to expose something, because you always do it just by making the name uppercase.
2. It allows readers to see at a glance without thinking whether a value is public --- it's public if it's a proper name.
In practice, Go struct definitions are extremely succinct, almost (at first) to a fault, because of decisions like this that eliminate syntax from the language. Compare a Go package to an equivalently functional Java package to see what I mean.
#1 does not seem to be a real issue. Most similar languages have the common, popularly understood, and easily-remembered keyword "public". Whereas capitalization is an exotic special rule that must be learned. If "you don't ever have to wonder how to expose something" is an important measure, I think Go is at a disadvantage here - especially if you want your code to be read by neophytes.
#2 is the argument for hungarian notation. Why is it important to visually distinguish public/private as opposed to, say, local vs member variable? I point this out because IDEs have a long history of visually distinguishing local vs member variables and some programmers even use naming prefixes like '_' or 'm_'. Yet I've never seen an IDE highlight members based on visibility. Given this backdrop, it's certainly not obvious that visibility requires such an important visual reference. And why shouldn't the IDE be responsible for this decision? Counterarguments that programming languages should be designed for 80-col amber VT100s sound pretty wacky in 2013.
Regarding terseness of the language, I'm not convinced. Without exceptions, Go adds an ton of "if error then return error" boilerplate to practically every function call. This is a lot more typing than adding 'public' to struct members.
I'm sure you've had the experience of working with libraries in other languages where the designer couldn't or didn't make up their mind about whether to expose capabilities as free functions or classes that needed instantiation, or, worse, exposed factories and singletons to get around their discomfort with simple functions.
Go sidesteps this problem. I found it helpful to think of it not as removing the "public" keyword from struct defs, but as creating a mechanism to "promote" functions and variables in packages --- something that would feel very shady in C++, but doesn't in Go. As a result, a lot of packages do The Simplest Thing That Could Possibly Work and just expose simple functions. The result is usually refreshingly clear.
I agree with you about error handling; Go's error handling makes its source code (though not its library interfaces) fussy and hard to read.
Second, if Go didn't use capitalization and wanted to keep package level visibility (which I will strongly argue is the Right Thing), there would be choice of having both private and public as keywords and requiring one of the pair to be put before before all constants, package level variables, type declarations, struct members, functions, and methods -- creating an immense amount of noise -- or to have one of the two and make the other one the default. Probably it would make more sense to have everything private by default and public only when prefixed with public… But it would be really ugly and inconsistent.
Now it might be possible to get around this with some other convention. Maybe it could be like Python, where everything is public by default but private if it starts with _. Or it could use a different, shorter keyword like 'p' so there's less noise overall…
But I think that once you make the decision to have package level visibility, case is the most efficient way of expressing it.
In denser structures, particularly dense all-lowercase structures, this is easy to miss.
Javascript - always capitalize classes
Python - underscore before private properties
C# - underscore in front of class level private properties
Python upgraded the whitespace convention to meaningful whitespace, why shouldn't Go upgrade the naming convention?
C# - I don't really know what's common practice in C#, but without a really good reason placing underscore in front of private properties would be really, really stupid.
On Python again - I used Python everyday for the last 3 years and I hate the whitespace convention. It's not because I have to use significant whitespace, but this syntax is the number one reason for why the language never got anonymous code-blocks. And I love the lightweight syntax, but looking at other lightweight languages, working with significant whitespace is just not worth it if it disallows such a useful and common feature.
More philosophically, I have to wonder what the benefit of this capitalization scheme is. Just to avoid typing 'public'? Hardly seems worthwhile. To make it obvious when reading code? This is the same argument for hungarian notation, which even Redmond has abandoned. IDEs for static languages like Go are perfectly capable of presenting identifiers appropriately - if indeed public/private is really what you want to call attention to.
I admit that I've not done more than dabble in Go, but I'm somewhat skeptical of this last point. Of all the IDEs I have used for various static languages, I have yet to see any that visually distinguish public vs private identifiers. If calling attention to this distinction was so important that it should be built into the language, I would expect it to be common practice in IDE formatting already.