News for hackers. Stuff that matters.
Re:Network Programming with Go by asto on Sunday June 10, 2012 @07:45AM (#4090799)
I find it funny how he skips an introduction to Go saying there are many such introductions already available but explains networks and the internet in great detail!
Re:Network Programming with Go by stewie2 on Sunday June 10, 2012 @09:48AM (#4091030)
I really like this book, but what I found most difficult when implementing a website is the authentication part.

How password should be encrypted? How session id should be generated and passed to the user? How session information should be saved? (cookie? file? database? memory?)

Also, I'd be interested in https, from theory to implementation.

Re:Network Programming with Go by dchest on Sunday June 10, 2012 @11:23AM (#4091276)
I understand that your comment is about this book, but I'll go ahead and answer your questions with links to packages (plus I get a chance to plug a few of my own creation :). If you want to learn how they work, reading docs and code is the best exercise. It's not a substitute for the proper book, though, but may turn out to be useful.

How password should be encrypted?

Choose your package:

    go get github.com/dchest/scrypt
    go get code.google.com/p/go.crypto/bcrypt
    go get code.google.com/p/go.crypto/pbkdf2
How session id should be generated and passed to the user?

If you mean how you can authenticate logged in users, then you can use this package:

    go get github.com/dchest/authcookie
How session information should be saved? (cookie? file? database? memory?)

Up to you, but generally it's better to avoid cookies for storing anything other than IDs. There are a few packages for "sessions" like in PHP, but I don't remember their names.

One more important question to add:

How do I protect against XSRF attacks?

    go get code.google.com/p/xsrftoken
You can read documentation for these packages here: http://go.pkgdoc.org/

As for HTTPS, it's a complicated thing. Pick any good book on it, then apply your knowledge to the standard package crypto/tls (http://golang.org/pkg/crypto/tls/).

Re:Network Programming with Go by stewie2 on Monday June 11, 2012 @03:18PM (#4096327)
Wow, Thank you very much!
Re:Network Programming with Go by swdunlop on Sunday June 10, 2012 @12:13PM (#4091412)
https is quite straightforwards in Go; the most direct means [1] essentially adds two parameters to http.ListenAndServe to supply paths to your cert and key file. The more flexible indirect ones involve either providing a TLS config [2] or using the crypto/tls package to create a TLS listener [3]. It comes down to which abstraction level works for your configuration.

Due to Go's pervasive use of io.Writer / io.Reader / net.Conn interfaces, once you have the problem of loading keys sorted, everything else is the same as writing a plain old TCP/IP listener. Configuring TLS will always be a pain -- decades, committees and broken implementations have seen to that.

As for actually verifying passwords and session integrity, Go's crypto packages are also pretty straightforwards and composable. The crypto/hmac [4] package is easily combined with various implementations of hash.Hash. In exchange for this simplicity, though, Go gives you plenty of opportunity to emulate LinkedIn and blow your own foot off, forgetting salt and common sense. There are plenty of higher level packages that offer to do it for you. (Blow your foot off, or hash your passwords -- I recommend reading implementations to decide which. :))

Also of interest when dealing with writing webapps in Go, the database/sql [5] package is an important abstraction. Since it is a more recent package, it often doesn't show up in google searches for sqlite, postgres, etc. This can give an impression that Go's database support is fragmented.

1 -- http://golang.org/pkg/net/http/#ListenAndServeTLS

2 -- http://golang.org/pkg/net/http/#Transport

3 -- http://golang.org/pkg/crypto/tls/#Listener

4 -- http://golang.org/pkg/crypto/hmac

5 -- http://golang.org/pkg/database/sql

Re:Network Programming with Go by stewie2 on Monday June 11, 2012 @03:19PM (#4096333)
Thank you very much for the answers!
Re:Network Programming with Go by stewie2 on Sunday June 10, 2012 @09:59AM (#4091055)
And also caching, that is an important topic.
Re:Network Programming with Go by plf on Sunday June 10, 2012 @07:08AM (#4090759)
Aside from being interesting and a new thing to learn, is Go used extensively in the professional world?

For network programming I've often used Python + Twisted + ZMQ, what are the most commonly used alternatives?

Re:Network Programming with Go by jbarham on Sunday June 10, 2012 @08:27AM (#4090864)
I'm developing a DNS hosting service (http://slickdns.com) and am using Django to do the bulk of the development, but I've also written a little utility web server in Go as it needs to be high performance but also lightweight and Go fits the bill perfectly. Updates are also a breeze as it's just a single static binary, compared to Django/Python where I have to make sure that any 3rd party dependency packages are installed first.

Last week I also developed a little random password generating web server with a web service API in Go. It's live at http://random-password-please.com/, and the source is at https://github.com/jbarham/random-password-please. IMHO it's a good illustration of how easy it is to write network servers in Go as it makes use of goroutines and channels and it all fits together very easily and naturally.

Re:Network Programming with Go by swdunlop on Sunday June 10, 2012 @12:29PM (#4091457)
I use Go very heavily for "duct tape" in production, combining processes like machine vision, VNC clients, and video transcoding into a uniform internal API. It has been conceptually a lot easier to work with than my native Python since it offers better (for me) abstractions and a very manageable concurrency model.

Using Go in a professional setting really isn't a technical challenge. The biggest concern is if you have to grow your team, you're either going to have to teach Go (which isn't that hard -- I've seen two different people come from dynamic languages and pick the basics up in a week) or hire people who know Go (often already happily employed).

To further mangle Charles V: "I speak Spanish to God, German to my horse, and Go to connections that might make me wait."

Re:Network Programming with Go by bleusink on Sunday June 10, 2012 @07:20AM (#4090771)
Here's a list of a few organizations that are using Go: http://go-lang.cat-v.org/organizations-using-go
Re:Network Programming with Go by jesstaa on Sunday June 10, 2012 @08:36AM (#4090879)
You might find that the blocking style of Go networking code is far less mind bending than using twisted.
Re:Network Programming with Go by james4k on Sunday June 10, 2012 @03:34PM (#4091941)
Yes, and the beautiful thing about it, is that the blocking calls you make are actually implemented as evented IO. This is great when you have a large number of goroutines; it keeps it very fast!
Re:Network Programming with Go by chc on Monday June 11, 2012 @01:03PM (#4095606)
Wait, what? Do you have any references for that? My impression was that Go creates new threads for new Goroutines when the current ones are blocked. Is that not how it works?
Re:Network Programming with Go by jesstaa on Tuesday June 12, 2012 @03:52AM (#4099335)
The Go runtime creates new threads to run goroutines when a thread is blocked making a syscall. The net package avoids having too many threads blocking on syscalls by using epoll/kqueue etc. for socket I/O.
Re:Network Programming with Go by baruch on Monday June 11, 2012 @03:42PM (#4096484)
The goroutine is a user-thread, Go can create as many threads as there are cores and schedules user-threads inside these threads without the programmer managing that, you just create goroutines. When a goroutine "blocks" it schedules out from the thread and another can replace it.

http://golang.org/doc/effective_go.html#goroutines

Re:Network Programming with Go by jesstaa on Monday June 11, 2012 @05:56AM (#4093868)
All I/O is evented I/O. It's just that with modern I/O requirements it's too expensive to use a whole thread to wait for the event.
Re:Network Programming with Go by truncate on Sunday June 10, 2012 @07:18AM (#4090769)
Go is very new language, so I doubt its currently being used much outside Google.

Answering your second question, almost every good programming language has some good networking libraries. Choice of language usually depends on what you are building and what are your requirements.

Re:Network Programming with Go by noselasd on Monday June 11, 2012 @05:23AM (#4093805)
Is there any way you can control the number of native threads in go ? I wrote a small go example that used net/smtp to bombard myself with emails. Just a loop that fired off a goroutine, which each sent an email.

It was quite disturbing that firing off several thousand goroutines resulted in several thousand native threads - resulting the system trashing. If I inserted a very small pause between creating the go routines, this did not occur and just 1 or 2 native threads were created. I tried the runtime.GOMAXPROCS, to no awail. Is there any way of controlling this kind of behavior ?

Re:Network Programming with Go by jesstaa on Monday June 11, 2012 @05:51AM (#4093859)
It's likely this is related to DNS lookups. Doing a lookup on a modern operating system requires calling in to some C library. The call in to the C code means that the thread can no longer be used to schedule goroutines, the runtime doesn't know how long the C call will take so it creates a new thread to keep running goroutines. For calls in to C that are very quick this can lead to a lot of thread creation.

You can do one of:

1. limit thread creation by using your operating systems limiting features. eg ulimit

2. rebuild the standard lib with DISABLE_CGO=1 to avoid the calls in to C if that's possible in your environment.

3. batch all your dns lookups in to a single goroutine.

4. make changes to the operating system so provide async dns lookups.

Re:Network Programming with Go by noselasd on Monday June 11, 2012 @08:42AM (#4094277)
Thanks, I'll do some testing with these points later on. Though it does sound like go should batch up gethostbyname/getaddrinfo C calls on one or a few threads. Having DNS lookups be the cause of thousands of threads to be created, up till the point where go routines fails is rather ridiculous for a language that's supposed to be good at concurrency imho.
Re:Network Programming with Go by jesstaa on Monday June 11, 2012 @10:10AM (#4094646)
Batching the C calls means that you all lookups are delayed by other lookups. This isn't very good for concurrency especially since it's reliant on network traffic. If you had a few lookups in the queue, you could end up having goroutines waiting a few seconds or for lots of lookups this could balloon out to minutes for a lookup. This is the trade off. In general most programs aren't making thousands of concurrent DNS lookups.
Re:Network Programming with Go by dillonmacc on Monday June 11, 2012 @11:04PM (#4098484)
i just use http://www.unlocktheinbox.com/dnslookup/mx/ or some other online dns lookup tools.
Re:Network Programming with Go by pjmlp on Sunday June 10, 2012 @04:00PM (#4092022)
Ever notice how all Go tutorials are just about Web servers somehow?
Re:Network Programming with Go by vasco on Sunday June 10, 2012 @07:59AM (#4090820)
It seems like when trading C for Go I'll be trading a few more lines of code for a bunch of black magic and a new syntax. Not sure if want.
Re:Network Programming with Go by jesstaa on Sunday June 10, 2012 @08:35AM (#4090876)
It's easy to forget that the black magic that runs C is commonly confused with just being an operating system.
Re:Network Programming with Go by baruch on Sunday June 10, 2012 @02:51PM (#4091817)
I'm not much of a Go programmer and do 99% of my work in C but I've played with it a bit and there are places where the abstractions and facilities provided in it are so nice they make it worth it.

The whole goroutines and channels mechanism is fascinating. I've seen and used coroutines in C that in a way achieve what goroutines do but the simplicity and the combination with channels is really great.

Comparing a co-routine based network server in C and Go would be the way to go to see the real strength of Go, IMNHO.