(Almost exactly what you describe is my current dream project, pieces designed, pieces roughed out, no part even remotely finished. The one I will get back to working on as soon as my startup exits and I have money and time. Or when it fails and I have time. You are not alone in wanting this.)
To add to your specifications: a "programer's web browser" should communicate with the user in a way which assumes that the user is knowledgeable. It should not use ridiculous metaphors like "this site could harm the computer! back to safety" or noob-oriented advice like "check the address for typing errors", but tell me things like "this URL is registered as a phishing site in such and such blacklist", "access to this site would violate an STS policy", "no DNS record for this domain, here is the result of the lookup", "no reply from IP A.B.C.D, here is a traceroute", etc.
My current choice of a keyboard-oriented browser is Firefox with the pentadactyl extension. It's not perfect, but better than anything else I've tried (including uzbl, which doesn't seem mature or well-documented enough).
Why not have a packet-based interface for browser extentions AND automation? (like a database)
I mean a full blown browser of current generation abilities, but with the option of completely keyboard driven interaction. Concise input of small commands (navigation, data extraction, exploring and massaging of extracted data, data uploading etc.) that can be composed to form more complex interactions.
It should have full interoperability with the CLI, don't go re-inventing grep, sed etc. but instead stand on their shoulders by interoperating fluently with them. In practice i expect this means the "programer's browser" functions as a full blown terminal emulator too.
If it were a text editor (and it should be a competent one of those, but the web is more than a textual medium so some implementations should certainly offer richer functionality than plain text editing) then the net and its many protocols would be its filesystem.
It should be extensible, but only by 1 language. Competing programer's browsers could offer alternative extension languages as their differentiator.
It should be entirely open for extension, by that i mean more than plugins, i should be able to rip out core parts of it to be replaced with alternative implementations (A's needs won't match B's needs and so on).
It shouldn't go too far though, it shouldn't be a full blown operating system: why re-invent the wheel, reuse the thousands of man years effort in existing OS's. It should be multi platform.
We almost have the requisite component parts available now, who's going to get the ball rolling with the new wave of "programer's browsers"?