News for hackers. Stuff that matters.
Re:Go performance from version 1.2 to 1.18 by DyslexicAtheist on Friday February 04, 2022 @02:35AM (#30203872)
> Tiny output binaries are also very quick to transfer and spin up in serverless as well - often reducing costs further.

Your point is specifically about cloud so maybe I shouldn't use go on Edge? In fact I'm struggling with the binary size right now (and have only just started with go after decades of C - because reasons beyond my control).

with go 1.17 the resulting executable (running on ARM) is 11MB (stripped ldflags -w -s). I can run upx --brute and reduce it to 3MB. But that is still heavy when another lua based application (a complete NodeRed clone - with massive complexity) and 100x as many lines of code (before importing external packages) is still just short of half a MB. (apples & oranges yes ...)

Maybe I shouldn't use go on an edge devices with such poor hardware?

I heard about tiny-go but looking at their website could only find instructions on how to x-compile for specific boards but not for specific HW (like ARM-5 softfloat).

Though I totally love the free stuff I get out of the box with go while also it feeling very familiar coming from C. The only thing I dislike is that it's not C :P

Re:Go performance from version 1.2 to 1.18 by marcus_cemes on Friday February 04, 2022 @04:14AM (#30204533)
If you're coming from C, I understand that 11MB seems like a lot to you. I've done quite a bit of serverless Node.js, a relatively slim docker image is always a few hundred MB and can easily stretch to a GB. However, I don't consider using C a viable alternative for that purpose.

What's cool about Cloudflare workers, for example, is that they provide the V8 runtime, you just provide the code, which like with Lua will be a very small footprint.

I'm curious, as a C developer, would you say that you prefer Go over Rust? I've used both and in my opinion Rust feels closer to C/C++ in terms of control and flexibility that the language gives you. To me (from a web service point of view, not systems programming), Go is just a faster (and better designed) pre-compiled version of JS/Node, it may look like C with it's simplicity (you end up having to write a lot of repetitive code), but still packs a garbage collector and (albeit very fast!) runtime under the hood.

Re:Go performance from version 1.2 to 1.18 by feffe on Friday February 04, 2022 @05:49AM (#30205186)
I've written a lot of C code and to me Go is like a high level C with some sharp corners removed and a GC. Maybe I don't see the limitations in Go as a problem because of this. Have done quite a bit of C++ as well. I prefer simpler languages that I can keep in my head and like C and Go for that reason.

Do I prefer it to rust? I've not bothered to learn rust yet. I think I would like it better than C++ but it's a bit large for my taste. I like to see the language level as an abstraction level that I want to be able to keep entirely in my head. I fear that rust (just as C++) may have too many features for that.

Re:Go performance from version 1.2 to 1.18 by usrbinbash on Friday February 04, 2022 @09:36AM (#30207227)
> I'm curious, as a C developer, would you say that you prefer Go over Rust?

Not the guy you asked, but coming from a similar background: Yes, I absolutely would.

Reason: Rust is an amazing language: Amazingly safe, amazingly fast, amazingly complicated. And the last isn't an advantage but a huge showstopper. What do I get from Rust over Go? A bit more performance. Okay, but I also get that from C, so the real question is, what do I get from Rust over C? Memory safety guaranteed by the compiler. Cool. But in exchange, I have to learn a language easily as complex as C++, but completely different from C.

No thank you.

Bottom Line: For me, the advantages of Rust over C do not justify the added complexity, not even close.

Re:Go performance from version 1.2 to 1.18 by DyslexicAtheist on Friday February 04, 2022 @12:43PM (#30210016)
> If you're coming from C, I understand that 11MB seems like a lot to you.

My issue is that the environment this thing is supposed to run on only has 30MB disk space.

> would you say that you prefer Go over Rust?

my answer is similar to my siblings. what stops me from using Rust the way it's intended is 100% learning curve. Unless I get paid for the time I invest (on the job) I would spend my evenings or weekends on gaining experience in yet another system/language or whatever. Pretty much did that during the first 20 years in this profession. I get that type-safety is a massive advantage so hopefully I'll get thrown on a Rust project eventually and then I'll meddle through :).

I'm not a fan of C++ especially for embedded. Nobody forces one to use exceptions but reality is, if I start an embedded project in C++, then the minute I don't pay attention somebody will end up refactoring and go overboard with templates or exceptions and then my flamegraphs look like the fires of hell.

2 things that I would love to get good at are Zig and Nim (in addition to Go where I'm far from an expert yet).

Re:Go performance from version 1.2 to 1.18 by KronisLV on Friday February 04, 2022 @12:57PM (#30210202)
> If you're coming from C, I understand that 11MB seems like a lot to you. I've done quite a bit of serverless Node.js, a relatively slim docker image is always a few hundred MB and can easily stretch to a GB. However, I don't consider using C a viable alternative for that purpose.

As someone who's worked in a number of stacks with higher abstraction levels, i can more or less confirm this. When you add containers in the mix, it gets much much worse, file size wise, even if i think that the increased reproducibility is definitely worth it in many types of deployments - mostly because file sizes have never been a primary concern for me, but misconfiguration and inconsistent environments have.

Now, nowadays i manage to keep all of my apps under 500 MB when shipping the full sized containers, typically they're in the 200 - 300 MB range: apps in Java (which need JDK), apps in Ruby (typically with the default interpreter), some .NET ones (at least Core, or .NET 6), some with Python (CPython) and Node.js etc. Of course, using Debian or Alpine for container base images is also a pretty good choice in regards to file sizes!

Of course, the beauty of all of it is that you can take advantage of layers - if you base everything on the same old container image between any two releases (or maybe have an intermediate layer for updates which may change) and only your ~50 MB .jar app changes for the most part, you'll be able to just ship those layers instead of shipping the base image all over, at least in most Docker registry implementations.

I guess that just goes to show how much variety and difference in opinions/concerns there is out there! I think GraalVM and other technologies (project Jigsaw in Java comes to mind IIRC) have the promise of even smaller deployments with stripped down runtimes even for these higher abstraction level stacks, but that's hard to do due to how dynamic they can be (e.g. class loading, reflection etc.).

From where i stand, Go is amazing in that regard, as are lower abstraction level languages.

Re:Go performance from version 1.2 to 1.18 by Cthulhu_ on Friday February 04, 2022 @07:44AM (#30206088)
> But that is still heavy when another lua based application [...] and 100x as many lines of code (before importing external packages) is still just short of half a MB. (apples & oranges yes ...)

Apples and oranges indeed; a Go binary is a fully self-contained, external-runtime-less application. That said, I can picture (in my head, don't expect me to build it) an edge / lambda environment that only takes your codebase and will run / feel like a script moreso than a complete binary.

I'm willing to put money that if you add the libraries + runtime to the lua application, you'd end up with more than 11MB.

Anyway, I'm trying to keep neutral; if your binary size is that important then staying with C may be the way to go. It's a compromise between binary size, language features, 'safety' (e.g. memory management is easier in Go thanks to auto zeroed variables and garbage collection), concurrency (not as much an issue in lambdas), etc.

Re:Go performance from version 1.2 to 1.18 by tgtweak on Friday February 04, 2022 @12:57PM (#30210207)
compiled LUA interpreter and runtime is pretty slim - but we're glossing over the reality that LUA is a drastically less functional language than golang - much in the way LUA is less functional than C.

I think to each their own - it is possible to have a lua app compile down, including the interpreter, to sub-megabyte size, but a 10k LOC lua app could probably be a 200 LOC golang app with some libraries pulled in.

I shudder thinking about the development cost of building and maintaining a lua codebase but if you had 100 million client devices running your code (say a wifi lightbulb) then it might make sense to save on client device and put that money into development of something very bespoke to optimize it.

Re:Go performance from version 1.2 to 1.18 by Abishek_Muthian on Friday February 04, 2022 @09:22AM (#30207045)
To me Go feels like a natural progression of C, And could see C often when I write it.

That said my experience with Go on ARM with low memory and CPU power hasn't been as same as my cloud services. I had to resort to ugly GC hacks and even preemptively killing the application on memory threshold.

But it's just my experience, I wouldn't be surprised if there are highly optimized Go apps running successfully on low-mem devices on edge. I don't think Go can replace C for embedded programming, Perhaps newer systems programming language like Rust, Zig is the way to Go.

I used to have Netdata on my edge devices, It would trip on its own Go plugin usage(mem) while the plugins written on NodeJS, Python ran fine; Likely because of unabated use of Goroutines.

Go's strength is its Network stack, It's been years since I've touched Apache/ngnix as deploying Go binary and setting up systemd rule is all that's required. For non-network application on edge, Especially on low memory devices I think Go is not ready yet.

Re:Go performance from version 1.2 to 1.18 by ghusbands on Friday February 04, 2022 @06:28AM (#30205461)
It's almost never worth using binary packers - every saving in on-disk size is lost on extra memory usage and extra cpu usage on starting up; it also interferes with memory management features (a binary typically takes up no writeable swappable memory - a packed one does).
Re:Go performance from version 1.2 to 1.18 by tgtweak on Friday February 04, 2022 @10:44AM (#30208246)
You can make notable compression gains by using a simple 7z self-extracting PE file on your already-static golang binary. In some cases this can yield another 10-50% savings in binary size. For environments where fresh startup time are important, then I would skip that as it does add some unnecessary overhead before your app comes to life. If you need something smaller than that, you're looking at writing c and building for your target arch.
Re:Go performance from version 1.2 to 1.18 by dagmx on Friday February 04, 2022 @03:34AM (#30204276)
Is your Lua code standalone? i.e is the interpreter part of the application size?
Re:Go performance from version 1.2 to 1.18 by edflsafoiewq on Friday February 04, 2022 @10:22AM (#30207876)
Even if it isn't, LuaJIT is half a meg, and regular Lua is about half that.
Re:Go performance from version 1.2 to 1.18 by tgtweak on Friday February 04, 2022 @10:47AM (#30208293)
If you're thinking about an edge where you have less than 3MB of storage space, and likely very limited in-memory space, you'd probably still be better to go with C. These kinds of environments are becoming very small % of use case over time. Usually such resource-constrained devices & systems have their own build system or crutch libraries built in to help reduce binary sizes to this scale. It is against golang's design principle to make a language that requires external frameworks/libraries to run. As it is, you can build your golang app, send it over an ssh tunnel and have it running on the remote system without much thought at all.
Re:Go performance from version 1.2 to 1.18 by stjohnswarts on Friday February 04, 2022 @02:11PM (#30211463)
Comparing size of executables is basically useless. You have to compare memory footprint and execution speed to be honest in a comparison between applications, and the applications shoudl serve roughly the same complexity of the problem being solved.
Re:Go performance from version 1.2 to 1.18 by blub on Friday February 04, 2022 @05:10AM (#30204912)
Why is the size a concern, start-up or limited disk space? If the latter, this seems quite resource constrained and Go might not be a good fit.

If you know C, you could try a minimal C++: C + vector,string,unique_ptr. And depending on your tastes, classes and shared_ptr.

Re:Go performance from version 1.2 to 1.18 by pjmlp on Friday February 04, 2022 @09:46AM (#30207358)
In addition to that, look at the compiler flags how to enable bounds checking even in release builds.