News for hackers. Stuff that matters.
Re:Nim Apocrypha, Vol. I by jstimpfle on Wednesday December 23, 2020 @07:15AM (#25516759)
Citation required. How can Wirth languages (e.g. Pascal, Modula) have "the same capabilities for low level programming", but without the traps, of C derived languages (e.g. C, C++)? The idea suggests itself that you can have one or the other, but not both.

Personally from trying it out, I like the speed resulting from the stricter module system. And there might be a little additional protection against bad builds resulting from it. But I assume it comes at a price in flexibility as well - I can't really tell since I quit after working 6 months in it.

Re:Nim Apocrypha, Vol. I by pjmlp on Wednesday December 23, 2020 @08:33AM (#25517457)
Has answered multiple times, including to your previous comments.

- bounds checking enabled by default for arrays and strings

- real enumerations that aren't implicit converted into numeric types (fixed in C++11, if one bothers to use enum classes)

- enumerations can be used as indexes (C++ can work around this with enum classes and some boilerplate templates)

- most data conversions must be done explicitly

- ability to actually define subtypes that are enforced by the type system (C++ offers this, provided one does the required boilerplate class/template code)

- memory allocation by default doesn't require doing math with data sizes (C++ as well, yet there is too much malloc()/free() pollution in C++ code bases)

- no need to deal with pointers for out parameters, thanks reference parameters, which even in C++ there isn't any guarantee they aren't null, even though that would actually trigger UB)

- in some of the dialects, namely Modula-2 and Oberon and the languages derived from them, unsafe code requires explicit import of SYSTEM package

- if one wants to do crazy C like code, all the necessary gear is available, pragmas to turn off bounds checking, pointer arithmetic, unchecked casts, unions, mapping variables to explicit memory addresses, pointers to callbacks, it is everything there in the package. The original bare bones ISO Pascal wasn't no longer relevant in the mid-80's, unless the teacher didn't knew any better.

In fact, IBM i, z/OS and Unisys ClearPath have done pretty well with PL/S, PL.8 and NEWP, only adding support for C and C++ after the market started to care about POSIX support on mainframes.

Re:Nim Apocrypha, Vol. I by jstimpfle on Wednesday December 23, 2020 @09:46AM (#25518059)
> bounds checking enabled by default for arrays and strings

You can make your own explicit bounds checks, or you can use valgrind (which, just like any type system, can understand only simple situations). You can use C++ which actually does allow you to do bounds checking by default, if you're into such things.

> - real enumerations that aren't implicit converted into numeric types (fixed in C++11, if one bothers to use enum classes)

> - enumerations can be used as indexes (C++ can work around this with enum classes and some boilerplate templates)

> - ability to actually define subtypes that are enforced by the type system (C++ offers this, provided one does the required boilerplate class/template code)

These are total non-issues in practice, since treating them as integers is usually what you want (you almost always want to support some light arithmetic on them, you want to store sentinel values sometimes...). Furthermore (as you say) C++ allows you to treat them as separate types.

> most data conversions must be done explicitly

This is the same with C, except for integral types where though you need to turn on warnings to avoid implicit downcasts of integral types with some (most?) compilers. For upcasts, implicit is what you want.

> memory allocation by default doesn't require doing math with data sizes (C++ as well, yet there is too much malloc()/free() pollution in C++ code bases)

That's FUD, if you do memory allocation - of course you need to compute the number of bytes you need.

If you do "new Foo[count]" you're likely doing it wrong anyway, just like if you're calling malloc directly. To equate memory allocation with malloc (or syntax sugar in fancy object languages) is WRONG. malloc() is a stupid default allocator, and you don't know what allocator you're getting if you don't provide your own. Good system performance often requires writing (simple) custom allocators instead of using malloc which has to deal with random allocation sizes, allocation patterns, and multiple threads.

Whether using your own allocator or not, it shouldn't be too much to expect you to write a single line like:

    #define ALLOCATE(type, count) ((type *) do_alloc(sizeof (type), (count)))
Or even

    #define ALLOCATE(ptr, count) (ptr = do_alloc(sizeof *(ptr), (count))
Now tell me, why exactly is "new Foo[count]" safer than "ALLOCATE(Foo, count)"? Or just do this minimal boilerplate by hand, this is not a practical issue unless your code is very badly factored (good code requires only few allocations).

> no need to deal with pointers for out parameters, thanks reference parameters, which even in C++ there isn't any guarantee they aren't null, even though that would actually trigger UB)

This is not a problem at all in practice, and anyway, if you need a lot of that you're doing something wrong. I very much like that C keeps it simple and you always see what happens. This is unlike Pascal and its Objects extensions, where it's impossible to see which data you're actually mutating because there are like 10 different "adressing modes". For example, dynamic arrays in Pascal can be assigned to other variables (or passed through a function parameter), and they will share memory, but only until one calls SetLength() on one "reference" at which point a Copy-on-Write is happening and they split lifes. This is not only not useful but extremely confusing. And the whole object extensions to Pascal (I can speak for Delphi) are built in that way - trying to not let the user see the complexity of what they're doing - which works until it doesn't, which is when you're dealing with extremely difficult to debug problems. Frankly Delphi ecosystem is a mess because it's a set of layers where each time they tried to be clever and add another philosophy-driven set of classes with implicit behaviours on top that was supposed to fix the previous layer's mistakes.

> - if one wants to do crazy C like code, all the necessary gear is available, pragmas to turn off bounds checking, pointer arithmetic, unchecked casts, unions, mapping variables to explicit memory addresses, pointers to callbacks, it is everything there in the package. The original bare bones ISO Pascal wasn't no longer relevant in the mid-80's, unless the teacher didn't knew any better.

This is why I took issue - you can't have both low-level bit twidding and "memory safe" programming. At most you can have a language that lets you choose between one or the other in a rather integrated system, but then you're still dealing with this abstraction gap, and having to traverse it constantly is not pleasant.

Re:Nim Apocrypha, Vol. I by pjmlp on Wednesday December 23, 2020 @09:57AM (#25518186)
> You can make your own explicit bounds checks, or you can use valgrind (which, just like any type system, can understand only simple situations). You can use C++ which actually does allow you to do bounds checking by default, if you're into such things.

Yeah, apparently Google, Apple, Microsoft think otherwise.

> Now tell me, why exactly is "new Foo[count]" safer than "ALLOCATE(Foo, count)"? Or just do this minimal boilerplate by hand, this is not a practical issue unless your code is very badly factored (good code requires only few allocations).

1 - Everyone needs to create their ALLOCATE macro, and not everyone does it right

2 - Actually it fails with certain kinds of type given as parameter

> I very much like that C keeps it simple and you always see what happens.

That is what people that never read ISO C and the myriad of UB cases think they do.

That is why I get such a kick of C Pub Quizzes, everyone is so full of themselves in regards to their C knowledge.

> This is why I took issue - you can't have both low-level bit twidding and "memory safe" programming. At most you can have a language that lets you choose between one or the other in a rather integrated system, but then you're still dealing with this abstraction gap, and having to traverse it constantly is not pleasant.

That is the whole point, dangerous code that can trigger memory corruption code should only be written when there is no way around to do it in a safer way.

And most of the time, it is an optimiser bug that is even the case to start with, unless one is writing drivers or kernel code.

Re:Nim Apocrypha, Vol. I by jstimpfle on Wednesday December 23, 2020 @10:54AM (#25518871)
> 1- Everyone needs to create their ALLOCATE macro

This is not an issue at all. It's one (!!) line. And the (huge) advantage is: Everybody can create their own allocation macros. Not all allocations are born equal: Some need specific contexts or arenas/pools where to allocate from, some want a specific alignment, some want zeroed memory...

And again, you don't even need this macro, it's just cosmetic. Better invest in some structure that reduces the number of required allocations to a small constant (per type or subsystem).

> and not everyone does it right

This is like saying, don't write code because not everyone is doing it right. This is almost the simplest type of code you can imagine.

> 2 - Actually it fails with certain kinds of type given as parameter

Which ones, then?

--- For the rest, trying to de-flame this and not to reply, and maybe be actually productive for an hour this afternoon :-)

Re:Nim Apocrypha, Vol. I by pjmlp on Wednesday December 23, 2020 @11:11AM (#25519056)
I wear the scars from comp.lang.c with pride, old songs never go out of tune apparently.
Re:Nim Apocrypha, Vol. I by wffurr on Wednesday December 23, 2020 @08:14AM (#25517273)
Undefined behavior and null terminated strings with the attendant buffer overflows are not fundamental constants of low level programming; they’re implementation details of C. Rusty (and Nim, Modula, Pascal, Ada, etc.) show that it’s perfectly possible to write low-level code with zero-cost abstractions without “unsafe by default” programming languages like C.
Re:Nim Apocrypha, Vol. I by jstimpfle on Wednesday December 23, 2020 @10:18AM (#25518413)
> Undefined behavior

UB in C has some unfortunate historic problems, and is _probably_ way too complicated. But UB per se is needed for a number of really important optimizations - for example, for array indexing using 32-bit integers you need to be able to assume that index-scaling operations (implemented as multiplications/shifts) don't overflow.

> null terminated strings

null terminated strings are like 10-50% space savers compared to string + length tuples for realistic strings (which are small). They're also easy to pass around and require no bikeshedding what's the best string type.

You can easily make your own type if you want (struct String { const char *buffer; int length; }) or you can just use C++ and one of the more terrible full featured string types with all sorts of badly matching built-in memory management and what not.

(By the way, the issue with SetLength() that I described in my other comment applies to Pascal Strings, too)

(Oh, and Pascal/Delphi also wants you to choose between PChar, String, AnsiString, WideString and what not, and for every little thing you're doing you need to locate and use the proper API for whatever string type you've chosen. Or you need to do conversions before the calls, with all the necessary implicit memory allocations. Enjoy!)

(Oh, this is basically how I achieved a 100x-1000x speedup in a module of a business application in a short Delphi stint. I converted the string data to chunk-allocated bytes with manually computed indexing - instead of allocating thousands of little strings and throwing them back and forth. That reduced the total application startup time (including a lot of code that I never touched) from minutes (in some cases) to something bearable (1-5 secs).

> with the attendant buffer overflows are not fundamental constants of low level programming

Of course they are. If you're not controlling bytes directly, and deciding about data layout and tradeoffs, I wouldn't call that low-level programming. YMMV. But - while my code is never fuzzed - it's been years since I've been bitten by a string bug. The importance of this stuff is way overblown, and knowledge and experience how to program with simple data structures is only found in more obscure corners of the internet. Also, very little in systems programming is about strings.

Re:Nim Apocrypha, Vol. I by wffurr on Wednesday December 23, 2020 @10:23AM (#25518476)
Optimization war stories aside (which is cool, thank you!), this reads like the classic apologetics "C is fine if you just use it correctly". Whereas in practice large C and C++ projects and libraries regularly suffer from severe security vulnerabilities exploited in the wild specifically from buffer overflows and use-after-free bugs.
Re:Nim Apocrypha, Vol. I by jstimpfle on Wednesday December 23, 2020 @11:18AM (#25519142)
There is also "Rust is fine once you know how to not get burnt by the type system, and if you carefully choose your dependencies it will maybe still be able to start a build of your project a month from now, and if you're lucky it will build in less than a lifetime".

Or "Java is fine if your fingers aren't sore yet and if you don't mind the latency spikes for your interactive application or if you're basically programming like C with training wheels and less performance".

;-)

Re:Nim Apocrypha, Vol. I by earwaxauditor on Wednesday December 23, 2020 @12:03PM (#25519607)
At least “this code doesn’t compile” isn’t a potential security vulnerability.
Re:Nim Apocrypha, Vol. I by wffurr on Wednesday December 23, 2020 @03:38PM (#25521922)
That's exactly the point I was trying to make.
Re:Nim Apocrypha, Vol. I by jstimpfle on Wednesday December 23, 2020 @02:56PM (#25521476)
Sure, and projects that never ship aren't either.
Re:Nim Apocrypha, Vol. I by earwaxauditor on Wednesday December 23, 2020 @03:31PM (#25521837)
I think you should lighten up ;)
Re:Nim Apocrypha, Vol. I by kaba0 on Wednesday December 23, 2020 @01:54PM (#25520767)
As for the latter: Or you know, most applications don’t need manual memory management because more often than not, programmers WILL get it wrong. Unless you really need that level of control over the hardware, you ARE better off with a good runtime env
Re:Nim Apocrypha, Vol. I by jstimpfle on Wednesday December 23, 2020 @02:55PM (#25521466)
That may be, but personally I am interested in those applications - GUI work, signal processing, low-latency networking.
Re:Nim Apocrypha, Vol. I by elcritch on Wednesday December 23, 2020 @04:33PM (#25522472)
Nim, with the new ARC garbage collector works well for those situations now (well within a few percent usually). Especially with the `move` and `sink` semantics.

I'm using it in embedded devices and needed a few tight loops a week or two back. With a bit of work and a few odd contortions, I got it to compile to almost the exact type of C code I'd write by hand to run a fast inner loop using preallocated arrays, etc. It turns out that code wasn't noticeably faster than more idiomatic Nim code with a few tweaks to ensure move's were able to be used. So I ended up going back to the more readable/idiomatic version while still roughly doubling the overall speed.

The new `move` semantics with Nim's ARC system really do help make it easier to write fast code. Now I understand why the C++ ecosystem is so bullish on `move` in the C++ stdlib now. It really can take code which looks like straightforward C code but performs closer to painstakingly optimized C code. Mainly since it result in less copying of objects/structs and reducing the number of malloc's. Overall it means less effort to design a fast system.

Re:Nim Apocrypha, Vol. I by atombender on Wednesday December 23, 2020 @06:01PM (#25523296)
I'm not sure why any "citation" is needed here. C is widely accepted as being deeply unsafe language, and C++ has inherited those traits, and is still working on plugging the weaknesses some 35 years later.

Of course you can do things like bounds checking manually in C, and run with all sorts of warnings enabled, and you can use static code analyzers that uncover entie classes of bugs, but the whole point is to never get that far in the first place.

I worked with Borland's Pascal dialect for 10+ years, and it offered a great deal of protections while still allowing you to use unsafe constructs when you needed to get close to the metal (e.g. pointer arithmetic and even embedded assembly code). But "safe by default" solves a lot of problems.

Re:Nim Apocrypha, Vol. I by jstimpfle on Wednesday December 23, 2020 @06:15PM (#25523393)
Pascal's low-level semantics are comparable to C's. That's why I dispute that it doesn't have the same traps.

I agree that if you program C by emulating higher-level object systems, it becomes unmanageable and a lot of bugs around object lifetimes appear, and they're very hard to fix.

I personally don't have a great interest in such object systems, but fail to see how C++ should be any worse than Pascal in terms of perceived safety resulting from the higher level of abstraction.

Re:Nim Apocrypha, Vol. I by blackrock on Thursday December 24, 2020 @07:58AM (#25527058)
The problem that I see with Nim, is that it transpiles down to C.

So, does it mean, that Nim can inherit these unsafe behaviors of C? Or did the Nim language creators safely avoid the pitfalls of C?

Whereas a language like Zig and Rust, compiles its code down to LLVM.

Re:Nim Apocrypha, Vol. I by nimmer on Thursday December 24, 2020 @02:58PM (#25530888)
> Nim can inherit these unsafe behaviors of C?

Not at all. You could apply the same reasoning for assembly: just because you can write insecure assembly it does not mean that all assembly generated by compiler is using vulnerable coding patterns.

Re:Nim Apocrypha, Vol. I by pjmlp on Thursday December 24, 2020 @08:41AM (#25527342)
That is an implementation detail, nothing prevents a future release of Nim compiler to directly generate machine code.
Re:Nim Apocrypha, Vol. I by blackrock on Thursday December 24, 2020 @08:48AM (#25527394)
Fascinating. When will they do it? I would prefer it generate direct machine code instead.
Re:Nim Apocrypha, Vol. I by nimmer on Thursday December 24, 2020 @03:00PM (#25530901)
> I would prefer it generate direct machine code instead.

Why? C has been the best choice for Nim so far.

Compiling with Nim+GCC is much faster than Rust, it generates applications with comparable performance, and it supports way more architectures.