News for hackers. Stuff that matters.
Re:Self-Contained Pure-Go Web Server with Lua, MD, HTTP/2, Q... by est31 on Thursday April 04, 2019 @08:30PM (#19578500)
This is quite impressive, but this claim is a bit wrong:

> All in one small self-contained executable.

Size of algernon executable: 24.4 MiB

Size of nginx-full executable: 1.1 MiB

Size of apache2 executable: 648K

Re:Self-Contained Pure-Go Web Server with Lua, MD, HTTP/2, Q... by sagichmal on Thursday April 04, 2019 @09:45PM (#19578909)
For self-contained architecture-specific server binaries, there is no practical difference between 240KB, or 2.4MB, or 24.4MB, or even, at a stretch, 244MB. It's not worth mentioning or optimizing for, except as novelty. I wish people would stop golfing with these numbers.
Re:Self-Contained Pure-Go Web Server with Lua, MD, HTTP/2, Q... by takeda on Thursday April 04, 2019 @10:42PM (#19579193)
You have it backwards, apache or nginx size is not for the novelty. Go is just a pig and its size grows every new release, and the size isn't really for being statically linked or debugging symbols, because it's huge even when those options are disabled.

Right now a "hello world" application in Go has comparable size to an OS with full GUI.

Re:Self-Contained Pure-Go Web Server with Lua, MD, HTTP/2, Q... by chx on Thursday April 04, 2019 @11:03PM (#19579302)
You would need find an application where the binary image size is a problem. In an age where 1TB SSDs are 300 USD this will be ... challenging. You are able to have about a thousand gigabyte sized images on that one SSD and I suspect you will hit CPU and I/O limits well before you have a thousand different binaries running for real.

Media storage would go on spinning rust disks anyways separate from the SSD(s).

Re:Self-Contained Pure-Go Web Server with Lua, MD, HTTP/2, Q... by derefr on Friday April 05, 2019 @02:13AM (#19580135)
The consideration isn’t binary size, but hot-loop cache coherency. Big binaries cause code-segment cache-line eviction.
Re:Self-Contained Pure-Go Web Server with Lua, MD, HTTP/2, Q... by nolok on Friday April 05, 2019 @06:22AM (#19581202)
Then that's a typical case of optimizing on the wrong metric.

Find a case where it's actually too slow, ok, but saying "A does X, and X can lead to Y, therefore A does Y" is wrong.

If you say the size of the binary is an issue, then give the issue, not how it could (or not) be an issue.

Re:Self-Contained Pure-Go Web Server with Lua, MD, HTTP/2, Q... by takeda on Thursday April 04, 2019 @11:38PM (#19579478)
We observed noticeable difference in deployment time and auto scaling based on container size.
Re:Self-Contained Pure-Go Web Server with Lua, MD, HTTP/2, Q... by sagichmal on Thursday April 04, 2019 @10:49PM (#19579231)
Who cares? Literally, what problem does it cause, or what does it make worse? It is totally immaterial.
Re:Self-Contained Pure-Go Web Server with Lua, MD, HTTP/2, Q... by takeda on Thursday April 04, 2019 @10:59PM (#19579271)
- increases amount of time to fetch and run container (it actually is quite noticeable when you have app that scales out and you updating it)

- it increases amount of storage to store multiple versions of containers (when you have an internal app and do frequent releases it adds quickly)

- it increases amount of data transferred on every deployment

- increases amount of memory used (the whole point of containers, was to efficiently use hardware (BORG), although a lot of people today miss that reason and run containers on VMs)

Re:Self-Contained Pure-Go Web Server with Lua, MD, HTTP/2, Q... by yetanotherme on Friday April 05, 2019 @12:47AM (#19579783)
True. Luckily, most Go binaries can be upx'd ( https://upx.github.io/ ) for a fraction of their original size. Just put it into your Dockerfile as a part of the build process.
Re:Self-Contained Pure-Go Web Server with Lua, MD, HTTP/2, Q... by penagwin on Friday April 05, 2019 @09:55AM (#19582483)
This works and helps with storage/transit. Word of warning though that many AV like to flag UPX'd executables (only important if it's not an internal tool), and it'll take even longer for it to start, and will use more memory. My understanding is it essentially does standard compression on the executable and appends the decompression part on the front of it.
Re:Self-Contained Pure-Go Web Server with Lua, MD, HTTP/2, Q... by xyproto on Sunday April 07, 2019 @09:37AM (#19596683)
With UPX the entire docker image with Algernon takes only 9MB.
Re:Self-Contained Pure-Go Web Server with Lua, MD, HTTP/2, Q... by TickleSteve on Friday April 05, 2019 @01:07AM (#19579875)
You have to ask yourself what is that space used for??

They're all compiled languages doing (roughly) the same operations. There is an order of magnitude difference in the number of instructions in one compared to the other.

Apache is more likely to be entirely cached whereas the others aren't.

Size matters for performance.... if you're not CPU bound, fine, but to say its immaterial is naive.

Re:Self-Contained Pure-Go Web Server with Lua, MD, HTTP/2, Q... by xg15 on Friday April 05, 2019 @08:24AM (#19581786)
Even if we don't know which particular problem it might cause right now, being wasteful when you can avoid it is never a good idea.

Image size is currently not the most important metric, but - guessing how your average node.js package already looks today - demanding people ignore it completely will probably set you on the road of multi TB images that also contain the developer's favorite desktop environment in the medium future.

Re:Self-Contained Pure-Go Web Server with Lua, MD, HTTP/2, Q... by Propolice on Thursday April 04, 2019 @10:38PM (#19579175)
Algernon does a bit more than plain Nginx or Apache. 24.4 MiB also includes bloat. See: https://github.com/golang/go/issues/27266 https://github.com/golang/go/issues/2559

Hopefully we can get smaller binaries by Go 1.13.

Re:Self-Contained Pure-Go Web Server with Lua, MD, HTTP/2, Q... by takeda on Thursday April 04, 2019 @11:34PM (#19579456)
My Apache installation with modules takes 4.3MB, and I'm quite sure Apache with modules can do more than Algernon.
Re:Self-Contained Pure-Go Web Server with Lua, MD, HTTP/2, Q... by mholt on Thursday April 04, 2019 @08:36PM (#19578538)
If that's the main criticism you have of the project, I'd say that's pretty good.

In fact, the readme of this project is really thorough!

Re:Self-Contained Pure-Go Web Server with Lua, MD, HTTP/2, Q... by interfixus on Friday April 05, 2019 @12:23AM (#19579677)
Very civilized comment from mr. Caddy himself. Heartening, not least considering the kind of vendettas some projects - Caddy among them - have to put up with from competing developers.
Re:Self-Contained Pure-Go Web Server with Lua, MD, HTTP/2, Q... by est31 on Thursday April 04, 2019 @09:17PM (#19578750)
Well yeah it's definitely an interesting approach. I guess it's much more lean than running a whole Ubuntu container VM that has all of these things installed, or running a gigantic bloated Javascript toolchain just to convert the scss or md files to css or html.
Re:Self-Contained Pure-Go Web Server with Lua, MD, HTTP/2, Q... by lawl on Thursday April 04, 2019 @08:58PM (#19578656)
A big part is almost certainly the difference between dynamic and static linking.

Can you run ldd on all of these and then report the combined size for each binary+libraries?

Re:Self-Contained Pure-Go Web Server with Lua, MD, HTTP/2, Q... by marcus_holmes on Friday April 05, 2019 @12:11AM (#19579630)
As someone else commented, the huge size of Go executables is down to a design decision to include a map of functions for panic reporting. There was a whole discussion on this recently on HN.

I don't know why the grandparent was downvoted. Go binaries are not small and the claim that this is a "small" single executable is untrue.

Hopefully the Go team will give us a flag to decide for ourselves whether to optimise for executable size or initialisation time. I know I'm fed up of uploading 50Mb files over dodgy wifi+vpn connections to update my server.

(edit fix repetition of design)

Re:Self-Contained Pure-Go Web Server with Lua, MD, HTTP/2, Q... by baldomo on Friday April 05, 2019 @06:34AM (#19581245)
You can actually pass LDFLAGS to any go tool (e.g. go build[1]). The flags are specified in the linker documentation[2]

[1] https://golang.org/cmd/go/#hdr-Compile_packages_and_dependen... [2] https://golang.org/cmd/link/

Re:Self-Contained Pure-Go Web Server with Lua, MD, HTTP/2, Q... by marcus_holmes on Friday April 05, 2019 @07:25AM (#19581479)
which is useful, except as far as I'm aware it doesn't cover this case.

The original is at: https://science.raphael.poss.name/go-executable-size-visuali...

Re:Self-Contained Pure-Go Web Server with Lua, MD, HTTP/2, Q... by throwaway5d097 on Friday April 05, 2019 @12:48AM (#19579790)
That's one thing that might motivate me to set up CI/CD on a server.
Re:Self-Contained Pure-Go Web Server with Lua, MD, HTTP/2, Q... by est31 on Thursday April 04, 2019 @09:14PM (#19578737)
I don't have nginx installed (got the number from a web download of the .deb package), but running this for apache:

$ echo $((`ldd /usr/sbin/apache2 | cut -d">" -f 2 | sed "s/(.*$//;s/ //" | xargs du -L | cut -f 1 | sed "s/$/+/" | xargs echo` 0))

3200

So 3.2 MB of shared library dependencies. 1.8 MB just being the libc which is almost guaranteed to be used by a different program already.

Re:Self-Contained Pure-Go Web Server with Lua, MD, HTTP/2, Q... by koffiezet on Thursday April 11, 2019 @11:53AM (#19635897)
> 1.8 MB just being the libc which is almost guaranteed to be used by a different program already

Not if we're talking about containers :)

Re:Self-Contained Pure-Go Web Server with Lua, MD, HTTP/2, Q... by chvid on Friday April 05, 2019 @06:53AM (#19581337)
I don't think it is fair that you are being downvoted.

Size does matter and not just in sense that it is using resources. The largest part of the 24 MB probably never gets executed but it adds unnecessary complexity that may hide bugs and security flaws.

Re:Self-Contained Pure-Go Web Server with Lua, MD, HTTP/2, Q... by xyproto on Friday April 05, 2019 @06:25AM (#19581209)
Algernon would be a lot smaller if it was not statically compiled and compiled with gccgo instead.

Sadly, the Go package that provides support for QUIC does not compile with gccgo, yet.

Re:Self-Contained Pure-Go Web Server with Lua, MD, HTTP/2, Q... by justinsaccount on Thursday April 04, 2019 @09:20PM (#19578778)
apache2 is mostly modules that are loaded at runtime.
Re:Self-Contained Pure-Go Web Server with Lua, MD, HTTP/2, Q... by takeda on Thursday April 04, 2019 @10:24PM (#19579098)
On my FreeBSD machine, all files in the entire apache package (including modules, manpages, headers, default pages, graphic for displaying directories in gif and png formats, tools are) take 4.3MB
Re:Self-Contained Pure-Go Web Server with Lua, MD, HTTP/2, Q... by xyproto on Friday April 05, 2019 @07:27AM (#19581492)