News for hackers. Stuff that matters.
Re:The long goodbye to C by throwaway2016a on Friday November 10, 2017 @11:58AM (#15671173)
I think it is easy to overestimate the importance of a managed language. Having to manage memory yourself is not that big a burden in many applications. In many cases it is a help.

For instance, I had a ETL program I was writing to process 100+ GB files. I pre-allocated all the memory I needed for exactly one row of data and immediately printed out the data when it finished that row (stream style).

The entire program processed 100 GB with lightning speed in less than 1K of memory. Maxing out the performance of the disk.

Best of all, I only had to free my memory at the end. Completely safe. No danger of double freeing and no extra memory allocation at all. Literally a dozen mallocs at the start and a dozen frees at the end.

Could I do that in Go? Sure. Would it have as optimized memory access? Maybe... but probably not. And C my entire program was 200 lines of code and absurdly fast.

Don't get me wrong, I love Go. I write programs in Go often. But for simple apps where the code can fit in a single file nothing can beat C. No bloat, no extra memory operations, and a code optimizer that has been being optimized for 30+ years. Go's code optimizer is not even close yet from what I can tell.

Re:The long goodbye to C by Franciscouzo on Friday November 10, 2017 @12:29PM (#15671415)
> But for simple apps where the code can fit in a single file nothing can beat C

I don't know how you concluded that nothing can beat C because Go is most of the times slower.

You can see it has pretty similar performance for a problem [0] pretty similar to yours.

[0] http://benchmarksgame.alioth.debian.org/u64q/revcomp.html

Re:The long goodbye to C by throwaway2016a on Friday November 10, 2017 @01:01PM (#15671667)
I may have been using hyperbole.

Looking at that benchmark that tiny difference in performance would have have been very magnified in my use case.

Though I will concede that it is possible that while slower it might still max out disk I/O and once the bottleneck isn't RAM or CPU it doesn't much matter what language you use.

And based on that, Rust does beat C.

Re:The long goodbye to C by kardianos on Friday November 10, 2017 @12:38PM (#15671478)
Yes, you can pre-alloc buffers in Go. And if you are reading and writing to disk, SSDs or HDs, the Go runtime isn't going to be your bottleneck. Yes, you can optimize memory access. That's an explicit design goal.
Re:The long goodbye to C by throwaway2016a on Friday November 10, 2017 @01:05PM (#15671698)
I am happy to be proven wrong.

That is something I have not done in Go and is good to know. But I could be wrong on this one, I have no idea how Go buffers are performance optimized but reading a character from a C array is one CPU instruction... it feels intuitive that in Go just the fact the extra layer of abstraction is added on top means it will be more instructions on the CPU.

But that is just a gut, I have not tested it.

Which in my use case would make a big difference over time.

But my intuition also tells me that it would still max out disk access in Go with modern CPUs.

Edit:

I should mention that my use case was to processes as many gigs of files on as cheap an AWS server as possible while maintaining a very strict SLA on how fast processing should be (the data became obsolete in hours and when we tried higher level languages the data was obsolete before a single file could finish). So I was trying to squeeze out as many CPU cycles as possible.

I should also mention that Go didn't even exist yet in a stable state when I did this project.

Re:The long goodbye to C by stmw on Friday November 10, 2017 @01:25PM (#15671818)
The real issue is the garbage collector, something that neither C nor Rust have. Automatic memory management (or thoughtful manual memory management) is good. Garbage collection is a problem.
Re:The long goodbye to C by throwaway2016a on Friday November 10, 2017 @01:58PM (#15672051)
Presumably if the buffer is pre-allocated and there are no other variables in play, the Go garbage collector wouldn't get executed often if at all.
Re:The long goodbye to C by stmw on Friday November 10, 2017 @10:51PM (#15674906)
Unfortunately, "presumably" is not good enough for systems programming - you have no idea when it might run, and you might end up with a lot of environment variables attempting to tame its behavior - like the JDK has.
Re:The long goodbye to C by isaiahg on Friday November 10, 2017 @12:25PM (#15671381)
Just out of curiosity from someone who started with languages like JavaScript and other languages. But what makes C special for this case? For instance couldn't I achieve the same thing with typed arrays in JavaScript?

Sorry if it's a naive question. My experience with C is limited.

Re:The long goodbye to C by vvanders on Friday November 10, 2017 @12:39PM (#15671491)
Nope, the main advantage of C/Rust/D/etc is explicit control of memory allocation and placement.

90% of your program performance is going to be dominated by cache usage(unless you're doing something really dumb). With C you can place things next to each other explicitly so that your prefetcher/etc is automatically pulling in the data you need to the L1 cache.

Any ref'd/GC'd language is going to make that really hard(not impossible, but you have to jump through a million hoops like sun.misc.unsafe in Java).

Look up Data Oriented Design if you're interested in this stuff, Mike Acton does a great job covering it.

Re:The long goodbye to C by pjmlp on Friday November 10, 2017 @04:45PM (#15673318)
> Any ref'd/GC'd language is going to make that really hard(

Not true at all.

Oberon, Active Oberon, Component Pascal, Modula-2+, Modula-3, D, System C#, Nim all offer ways to manually control memory outside GC control, just like C and C++.

Glad to provide Modula-3 examples to any C tricks you might want to do, or in D if you prefer something more modern.

Re:The long goodbye to C by vvanders on Friday November 10, 2017 @05:05PM (#15673446)
Yet not a single one of those examples is a mainstream language. So when you finish the project and move on to bigger things I've got a codebase that no one can maintain.

Note that I didn't say that it was impossible, just that the ergonomics of GC'd languages don't tend to focus on performance in the ways that languages with explicit memory control do.

Re:The long goodbye to C by pjmlp on Friday November 10, 2017 @05:33PM (#15673632)
Most of those languages died due to political reasons and company acquisitions, not because of lack of technical features.

GC systems programming languages offer exactly the same explicit control as C does, in unsafe memory blocks.

Do you want to shot yourself like C in Modula-3? Declare a variable as UNTRACED REF, 100% like a C pointer.

Do you want a C union? Declare a tagged RECORD type.

Do you want a packed record to fill you cache line? Compiler alignment pragmas.

Allocation of value types on the stack or global memory section? Great, it is the default allocation type.

Bit fiddling to prepare a statically allocated buffer for a DMA transfer? UNSAFE code section.

Re:The long goodbye to C by vvanders on Friday November 10, 2017 @06:27PM (#15673977)
> Most of those languages died due to political reasons and company acquisitions, not because of lack of technical features.

Yet you just talked past my point to give some sort of soliloquy about Modula-3.

It doesn't matter how those languages died, it just matters that they did. The ecosystem and number of people who know how to write a language is almost, if not more important than the language when it comes to shipping software.

Re:The long goodbye to C by pjmlp on Saturday November 11, 2017 @03:01AM (#15675436)
No, my point was that these languages have 100% the same ergonomics as C in what concerns explicit memory management, with the additional productivity of having a GC around when those requirements aren't a must.

We should not think of Java when comparing system programming languages with GC against C.

Re:The long goodbye to C by vvanders on Saturday November 11, 2017 @11:31AM (#15676952)
Yet you still keep talking past my point(while also nicely moving the goalpost).

Yes, the languages you mention meet all the technical requirements(congrats, here's your 10 internet points) it doesn't matter if I can't find people who know how to write it.

Since you seem to want to avoid talking about anything I raise I don't think it's productive to continue this conversation.

Re:The long goodbye to C by pjmlp on Saturday November 11, 2017 @01:38PM (#15677479)
Just because these languages died, doesn't mean we cannot take those lessons into GC languages where you can find people who know how to write it.

For example, all the work with System C# done in Midori has been being applied to C#, with .NET Native, ref returns, local ref variables, value tuples, regions with disabled GC, memory slices, with a few more down the roadmap, like effectively manual memory management presented at OOPSLA 2017.

Or the ongoing @nogc and BetterC improvements in D, Wallmart, EBay, Netflix, Remedy Games among many others, seem to be able to find people that know how to write it.

Re:The long goodbye to C by zeveb on Sunday November 12, 2017 @11:54PM (#15684373)
Do you want a reliable project which is difficult to hire for, or an unreliable project it is easy to hire for?
Re:The long goodbye to C by justin66 on Friday November 10, 2017 @02:10PM (#15672139)
Creating a program that is bound by disk performance and indifferent to l1 cache is not necessarily "really dumb." Premature optimization is a real thing.
Re:The long goodbye to C by vvanders on Friday November 10, 2017 @02:25PM (#15672254)
Generally a cache miss to DRAM is going to be somewhere around 3 orders of magnitude slower than accessing something in L1.

All it takes is 4-5 of these and now you're incurring performance hit of ~1ms which gets you into SSD territory[1].

As always, profile before making these optimizations but the OP was asking why something like this isn't possible in managed languages.

[1] https://arstechnica.com/information-technology/2012/06/insid...

Re:The long goodbye to C by dom96 on Friday November 10, 2017 @06:53PM (#15674100)
Wait. Isn't D considered a GC'd language? You seem to be implying it's not.
Re:The long goodbye to C by btrask on Friday November 10, 2017 @12:37PM (#15671467)
If you used Node, you would have IIRC around 40MB (edit: 24MB on my system) of memory overhead before your code even started running. If you are CPU-bound, multiply that by the number of cores you want to use.