News for hackers. Stuff that matters.
Re:Golang custom transports and timeouts by tptacek on Thursday April 09, 2015 @04:49PM (#9350933)
I had a similar (though unrelated) problem early on with Golang writing a port scanner, which forced me to hoist my own event loop (build around an explicit select call) into my program: timeout or not, I needed lots and lots of OS file descriptors, and by relying on Golang's own event loop to close stale ones on timeouts, I'd invariably run out.
Re:Golang custom transports and timeouts by sebcat on Thursday April 09, 2015 @08:57PM (#9352003)
> I needed lots and lots of OS file descriptors

Why?

I ask because I wrote a port scanner professionally once, and I got away with using one fd for network I/O. I can only think of one case where you end up using a lot of fds and that is if you're using TCP sockets and connect(2) where you'd end up with one fd/tested port, and that's not good.

Re:Golang custom transports and timeouts by tptacek on Thursday April 09, 2015 @10:50PM (#9352335)
It had to work unprivileged in userland.
Re:Golang custom transports and timeouts by FiloSottile on Thursday April 09, 2015 @06:07PM (#9351363)
Similar unrelated problem here, too, on the sole machine now running the Heartbleed test.

After recompiling with Go 1.4 (from Go 1.2 - apparently the network poller was rewritten?) I'd start running out of file descriptors really fast, all leaking deep in net/http.ReadRequest.

Turns out http.DefaultServer doesn't have any default ReadTimeout, so when a client goes away before finishing a request I'd lose the goroutine and the file descriptor with it.

So yeah, you probably also want to explicitly set http.Server.ReadTimeout if you are not doing weird streaming/"comet" stuff.

Re:Golang custom transports and timeouts by tptacek on Thursday April 09, 2015 @07:48PM (#9351767)
That's related to my problem, but not the same; what I needed was control over when an fd/goroutine would be destroyed. Explicit timeouts (at least in 1.0, but I think still) couldn't give me that. I needed an obscene number of fds, though.
Re:Golang custom transports and timeouts by jbox on Thursday April 09, 2015 @08:45PM (#9351972)
If you're going across the network, a slow timeout is often trickier than an explicit failure.

I wrote about handling this in Python: http://www.mobify.com/blog/http-requests-are-hard/