News for hackers. Stuff that matters.
Re:Zig: a system language which prioritizes optimality, safe... by ocschwar on Sunday August 28, 2016 @08:15PM (#12379221)
Once you've made the decision to leave C for a systems programming project, why would one not just go over to Rust?
Re:Zig: a system language which prioritizes optimality, safe... by quacker on Sunday August 28, 2016 @08:25PM (#12379252)
Maybe easier integration with existing C libraries?

They give an example of a tetris clone in Zig. This is how they include GLFW with Zig: https://github.com/andrewrk/tetris/blob/master/src/c.zig and all other locations in the project just import this c.zig file.

In contrast in Rust, I think you have to actually declare all the functions you want to use from a C library. For example: https://github.com/PistonDevelopers/glfw-rs/blob/master/src/....

Re:Zig: a system language which prioritizes optimality, safe... by tomjakubowski on Sunday August 28, 2016 @08:29PM (#12379260)
There are C library binding generators for Rust which do the boring work for you. It's not perfect, but rust-bindgen was pretty good last time I tried it (~15 months ago): https://github.com/Yamakaky/rust-bindgen
Re:Zig: a system language which prioritizes optimality, safe... by skissane on Sunday August 28, 2016 @08:38PM (#12379283)
When a language has built-in support for easy binding to C, that's a positive. When other languages such as Rust offer that feature through external tools, that's not as good, since I need to evaluate which of those external tools to use (when there is more than one), and it is something else to download/install/learn and another step to add to the build process. So it looks to me like Zig handles this better than Rust does (of course, Rust has clear advantages over Zig in other areas: much more mature / widely used, and more advanced memory management.)
Re:Zig: a system language which prioritizes optimality, safe... by ycombinatorMan on Sunday August 28, 2016 @08:57PM (#12379340)
not nearly as attractive as inline C or direct C imports
Re:Zig: a system language which prioritizes optimality, safe... by Ygg2 on Monday August 29, 2016 @01:39AM (#12380211)
Pfft, pretty sure you could inline C in Rust. No one was brave enough to develop said compiler plugin.
Re:Zig: a system language which prioritizes optimality, safe... by Gankro on Monday August 29, 2016 @03:11AM (#12380497)
Re:Zig: a system language which prioritizes optimality, safe... by Ygg2 on Monday August 29, 2016 @06:54AM (#12381107)
Eh, C++ isn't really C :P

I kid, it's really impressive work.

Re:Zig: a system language which prioritizes optimality, safe... by kbenson on Sunday August 28, 2016 @08:29PM (#12379263)
I think there exists, or work has been done on, a tool to auto generate that data for you. So not quite as simple, but possibly more fine grained control.
Re:Zig: a system language which prioritizes optimality, safe... by c3534l on Sunday August 28, 2016 @08:47PM (#12379315)
Yeah, I really hope Rust becomes the default C replacement. It's such a well-designed language, it's just very early in its development right now.
Re:Zig: a system language which prioritizes optimality, safe... by aninteger on Monday August 29, 2016 @09:02AM (#12381545)
Rust seems very complex and C is more about simplicity (in the sense that K&R is a short read). Other languages with the same spirit of simplicity would be Tcl and perhaps Go. I just wish Go had more supported machine architectures.
Re:Zig: a system language which prioritizes optimality, safe... by ocschwar on Monday August 29, 2016 @11:24AM (#12382462)
My feeling is that one should use Python or Go to prototype a program, and then move to Rust to make it systems-ready.
Re:Zig: a system language which prioritizes optimality, safe... by andrewchambers on Sunday August 28, 2016 @09:01PM (#12379360)
Yes. One flavour of icecream is all we need. (I hope people can see the sarcasm)
Re:Zig: a system language which prioritizes optimality, safe... by nickpsecurity on Sunday August 28, 2016 @09:32PM (#12379453)
"My qbe C frontend is actually written in myrddin which is a lot like rust/ocaml. To be honest, C is not well suited to places where there is adversarial input such as servers, but when it comes to logical correctness of code I do not see amazing benefits from languages like ocaml."

Did you mean two? Or did you come to new conclusions after that experience? Or did I misunderstand that entirely? ;)

Re:Zig: a system language which prioritizes optimality, safe... by andrewchambers on Sunday August 28, 2016 @10:08PM (#12379550)
There ARE benefits to correctness from better languages, just not as amazing as people think.

I'm just saying a vulnerability in C is just going to be an out of bounds exception in another language. both cases the code is not correct, one is just safer than the other when it comes to exploits.

Keep making more languages, we can't know for sure if we don't try.

Re:Zig: a system language which prioritizes optimality, safe... by nickpsecurity on Sunday August 28, 2016 @11:02PM (#12379734)
I can see where you're going but I don't think it's correct. There are clearly cases where an error will mess you up whether it goes as far as in C or not. Other times, the language or tooling prevents the error before runtime to ensure correctness. Are you aware of the benefits of Design-by-Contract (Ada/Eiffel), static proving (SPARK), or dependent types (ATS), though? The correctness criteria you can encode in them can straight-up prevent errors at interface or algorithmic expression levels. Three of these have been used for low-level code with two often in real-time and one for an 8-bitter. Depending on automation or interactive use involved, the errors caught at compile-time can increase pretty far past what a basic, low-level, type system can do.

So, we already know we can knock out extra classes of errors with such languages. It was proven in theory and in the field. Using or improving them is just good engineering. We can also make more languages in trial-and-error discovery process to see if we find more benefits. Exceeding C's benefits, though, is already empirically proven to be worthwhile whether it's a Myrddin, a SPARK, or an ATS.

Re:Zig: a system language which prioritizes optimality, safe... by wolf550e on Sunday August 28, 2016 @11:26PM (#12379815)
C code will continue to dereference pointers after object deallocation, access arrays out of bounds, and not complain about integer overflow. Even if input is not an RCE, you will silently get an incorrect result of computation. A safer language helps in cases other than security.
Re:Zig: a system language which prioritizes optimality, safe... by majewsky on Monday August 29, 2016 @04:23AM (#12380693)
And there are newer languages that have this problem. I was shocked to see that I could produce a SIGSEGV in Nim within the first 15 lines of code that I wrote.
Re:Zig: a system language which prioritizes optimality, safe... by dom96 on Monday August 29, 2016 @04:46AM (#12380766)
Why were you shocked? Is it because you're used to languages that prevent null values by default or do you have concerns about the safety of Nim because of this error?

Whatever the case, as far as I am aware, sooner or later Nim will prevent nil values by default.

Re:Zig: a system language which prioritizes optimality, safe... by majewsky on Tuesday August 30, 2016 @04:27AM (#12388342)
If that happens, I'll gladly look at it again. My point is that we have enough languages with unchecked access to nil values by now.
Re:Zig: a system language which prioritizes optimality, safe... by jwatte on Tuesday August 30, 2016 @10:38PM (#12395601)
But how is a nil pointer segv different from a irrefutable pattern mismatch in Haskell, or "undefined is not a function" in JavaScript?
Re:Zig: a system language which prioritizes optimality, safe... by SamReidHughes on Tuesday August 30, 2016 @02:40PM (#12392717)
A segfault on a null pointer isn't a memory safety problem, so that's not the same thing at all.
Re:Zig: a system language which prioritizes optimality, safe... by dbaupp on Tuesday August 30, 2016 @04:30PM (#12393603)
A segfault on null is fine if the language defines it to be OK (i.e. dereferencing null is reliably a crash), but most environments that allow interacting with null (C, C++, LLVM IR) have dereferencing it be undefined behaviour, i.e. the compiler may optimise in ways that mean code that naively results in a null dereference/segfault do not do that and instead result in random memory corruption.

Note that Nim uses C as an intermediate language for compilation, so unchecked dereferences of possibly-null pointers are not memory safe.

Re:Zig: a system language which prioritizes optimality, safe... by dom96 on Tuesday August 30, 2016 @05:52PM (#12394245)
Modifying Nim to perform a check before each dereference would be trivial. The reason this isn't done yet is because in practice unchecked dereferences do not cause random memory corruption, I have been using Nim for a while now and have never seen an instance where this was a problem.
Re:Zig: a system language which prioritizes optimality, safe... by nickpsecurity on Tuesday August 30, 2016 @03:19PM (#12393046)
Sure it is. It means your language and tooling allowed an unauthorized access that resulted in a segfault. Many don't. There's even system-level languages like Clay and Rust that can catch those things. Also, tools like Softbound+CETS neutralize them in C code. The category is called "temporal, memory safety."

Definitely a memory safety issue. Definitely worth preventing if possible. If not preventable, definitely worth handling better than mere segfaults.

Re:Zig: a system language which prioritizes optimality, safe... by SamReidHughes on Tuesday August 30, 2016 @03:34PM (#12393174)
No, it's the same as a .unwrap() in Rust, or malloc exiting when it fails. It means the language doesn't force you to handle all cases, not that it's "unsafe."
Re:Zig: a system language which prioritizes optimality, safe... by steveklabnik on Tuesday August 30, 2016 @05:08PM (#12393908)
unwrap won't cause a segfault, it will either unwind the stack, or abort.
Re:Zig: a system language which prioritizes optimality, safe... by SamReidHughes on Tuesday August 30, 2016 @07:10PM (#12394702)
The point about it being like .unwrap is that the bug is a completeness bug.

My theory is that people have converted "memory unsafety can cause segfaults" into "segfaults are memory unsafe." Segfaults are actually the desired outcome, and initializing a pointer with null, instead of leaving it uninitialized is how that outcome is achieved.

Re:Zig: a system language which prioritizes optimality, safe... by nickpsecurity on Tuesday August 30, 2016 @08:49PM (#12395124)
The desired outcome is a program that either doesn't have the error or crashes safely with a report of exactly how it happened. Not segfaults or undefined behavior in general. The latter are simply one of the consequences of C's design and programming style than anything inevitable or ideal.

It's tradition that failures of a language's safety system to handle memory are "memory, safety issues." However, that they can be used for code injections in some scenarios is even more reason to think of them that way and make our languages prevent them where possible. Quick example:

https://tk-blog.blogspot.com/2009/01/exploitable-userland-nu...

Re:Zig: a system language which prioritizes optimality, safe... by SamReidHughes on Tuesday August 30, 2016 @11:19PM (#12395775)
Yeah, in C you could have a problem if there's a large object, but in that language you already have undefined behavior. Nim is not C, and bounds checking on arrays prevents that. (I am only skimming the docs, so tell me if that is incorrect.)
Re:Zig: a system language which prioritizes optimality, safe... by steveklabnik on Tuesday August 30, 2016 @08:43PM (#12395112)
Segfaults can often be turned into an actual exploit; unwinding the stack cannot. It would be better to explicitly abort than to purposefully cause a segfault.
Re:Zig: a system language which prioritizes optimality, safe... by SamReidHughes on Tuesday August 30, 2016 @11:35PM (#12395837)
You are committing the exact generalization of segfaults=bad that I mentioned in the parent comment. Memory safety violations which cause segfaults can often be turned into an actual exploit. That does not mean the same thing as, "If A is a segfault, A can be turned into an exploit."

And it's not better to "explicitly" abort, because you get zero benefit from that. And because testing for null would be slow. (This is why JVM implementations often let it segfault.)

Re:Zig: a system language which prioritizes optimality, safe... by steveklabnik on Wednesday August 31, 2016 @10:25AM (#12398508)
I didn't say that all segfaults are always security problems, just that the often are.

I think we'll have to agree to disagree here, though. Segfaults are memory safety problems, and unwrap is significantly different because it's not a memory safety problem.

Re:Zig: a system language which prioritizes optimality, safe... by SamReidHughes on Friday September 02, 2016 @08:48PM (#12417171)
You can "agree to disagree" about a definition, but we'll also have to agree to disagree on what properties the definition should have.

I'd say that language L is memory-safe, then a language M which is like L, except that some programs terminate earlier, should also be memory-safe. We'll have to "agree to disagree" about that.

I'd say a compiler's choice of behavior-maintaining implementation technique should not determine whether a language is memory-safe. We'll have to "agree to disagree" about that.

I'd say a definition of memory safety should correspond to some mutually relevant notions of being safe (with consequences around security and ease of debuggability), and that it shouldn't include some kinds of unrelated notions about termination safety while omitting other kinds of termination safety. We'll have to "agree to disagree" about that.

> I didn't say that all segfaults are always security problems, just that the often are.

My mistake, I assumed you were trying to make a point that had some bearing on the discussion.

Re:Zig: a system language which prioritizes optimality, safe... by daurnimator on Sunday August 28, 2016 @09:18PM (#12379409)
One of my biggest reasons is the lack of ability to handle OOM when using the standard library. see e.g. https://internals.rust-lang.org/t/could-we-support-unwinding...
Re:Zig: a system language which prioritizes optimality, safe... by whyever on Wednesday August 31, 2016 @04:29AM (#12396812)
How can you handle OOM if your process gets killed by the kernel for using to much memory?
Re:Zig: a system language which prioritizes optimality, safe... by daurnimator on Wednesday August 31, 2016 @10:15AM (#12398439)
By never going that far. Turn off overcommit; I advise to never run with it in production.
Re:Zig: a system language which prioritizes optimality, safe... by Koshkin on Sunday August 28, 2016 @10:39PM (#12379640)
Rust? Maybe. But then, why not Go or Swift or Lisp? Personally, if I were to be expelled from C/C++ paradise, I would try Oberon - the only language I know that can compete with C in its simplicity and, despite normally being perceived as garbage-collected, the ability to serve as a truly systems programming language.
Re:Zig: a system language which prioritizes optimality, safe... by mordocai on Sunday August 28, 2016 @10:48PM (#12379682)
As far as vs Go or Swift or Lisp, Rust is much "closer to the hardware"(as discussed in other threads nothing is really close to the hardware anymore) than the other languages, go and most lisps use GC which causes problems in many systems language situations, and last I heard swift still has lackluster cross platform support.

I don't really know anything about Oberon so can't speak to that.

P.S. Personally, all my hobby projects are in lisp. I write applications though, not system stuff.

Re:Zig: a system language which prioritizes optimality, safe... by pjmlp on Monday August 29, 2016 @10:51AM (#12382197)
Oberon was used for writing a complete graphical workstation OS used during the mid-90 at the ETHZ in Switzerland.

It was a language and OS, following the same spirit of Xerox PARC OSes.

Re:Zig: a system language which prioritizes optimality, safe... by nickpsecurity on Monday August 29, 2016 @01:09PM (#12383322)
The A2 Bluebottle OS is latest incarnation of Oberon OS with GUI and very responsive. Languages like Oberon can do unsafe stuff for low-level work like OS's. You just use manual, memory management and UNSAFE keyword. Rest that can be GC'd, bounds-checked, etc is by default.
Re:Zig: a system language which prioritizes optimality, safe... by ziotom78 on Monday August 29, 2016 @07:39AM (#12381224)
A few months ago, I considered Oberon as a viable alternative to C for my pet projects. Horace, after some time spent playing with it, I was scared by a fewserious limitations: (1) there is no support for UTF-8 strings, native strings are just sequences of ASCII characters, (2) lack of libraries and available bindings (e.g., SQLite, GSL, FFTW...), (3) scarcity of tools like package managers and Doxygen-like tools, (4) very few resources available on the web (Wirth's book is exceptional, but it does not cover use cases that are common in 2016.)
Re:Zig: a system language which prioritizes optimality, safe... by jakub_h on Monday August 29, 2016 @08:32AM (#12381403)
I suspect that Oberon wants you to roll out your own text library. It's actually how C probably wants you to roll out your own string library, too, because "C strings" are no better than Oberon's text facilities. The problem is that nobody knows how you want to work with your strings. There doesn't seem to be "the one correct way" to work with texts. Gauche Scheme, for example, has an interesting take on strings (http://practical-scheme.net/docs/ILC2003.html): backing arrays are shared and substrings are O(1), semantics for Scheme strings is observed - mutability is still possible but expensive, thus also gently discouraging the programmer from using it.
Re:Zig: a system language which prioritizes optimality, safe... by pjmlp on Monday August 29, 2016 @10:53AM (#12382210)
Oberon was great in the mid-90, but it is stuck in time.

I loved the complete OS experience, but nowadays Swift, .NET Native,Rust among others would be better options.

Re:Zig: a system language which prioritizes optimality, safe... by amyjess on Monday August 29, 2016 @09:36AM (#12381719)
Honestly, my first thought upon seeing GP's post was "why not Ada?". Designed by the DoD for safe systems programming, it wouldn't really be a bad choice.
Re:Zig: a system language which prioritizes optimality, safe... by ocschwar on Monday August 29, 2016 @10:25AM (#12382014)
That would be my thought too, but at this point Rust covers compile time checks against a larger class of errors, and is gentler about doing it.
Re:Zig: a system language which prioritizes optimality, safe... by teacup50 on Sunday August 28, 2016 @09:32PM (#12379454)
Among other things, systems programming projects generally require stability of language syntax, API, and ABI.

In terms of ABI stability, Rust has none.

In terms of API stability, they seem willing to drop features within a major release cycle or two.

In terms of language stability, they seem to only guarantee stability within one major release (e.g. 1.x).

Rewriting for the sake of language updates is a huge resource drain when you're talking about code bases with lifetimes that are likely to be longer than their authors'.

For reference, just about every major shipping OS contains significant LoC dating back to the 70s or 80s.

Re:Zig: a system language which prioritizes optimality, safe... by Gankro on Sunday August 28, 2016 @09:44PM (#12379486)
> In terms of API stability, they seem willing to drop features within a major release cycle or two.

I don't think any feature/api that was at one point marked as stable post 1.0 (when they started making any guarantees) has ever been removed.

> In terms of language stability, they seem to only guarantee stability within one major release (e.g. 1.x).

While it's true that 2.x has the rights to break everything, this is incredibly dishonest to state in a vacuum, because it's a well established point that 2.x should never exist. The devs want all code to work forever, and only a catastrophe would push them to pull the lever. They have of course reserved the right to require fixes in special cases -- e.g. fixing a bug in the type checker might break some code, and that's acceptable. Although even then they'll try to phase in the fix over a few releases. Every release is tested against the entire crates.io ecosystem to try to catch these regressions.

You are absolutely correct on ABI stability, though.

Re:Zig: a system language which prioritizes optimality, safe... by teacup50 on Monday August 29, 2016 @04:57AM (#12380791)
Given the litany of deprecation markers applied to existing APIs, one must presume the intention of removing them, no?
Re:Zig: a system language which prioritizes optimality, safe... by pcwalton on Monday August 29, 2016 @08:54AM (#12381495)
> Given the litany of deprecation markers applied to existing APIs, one must presume the intention of removing them, no?

No! That would be a basic violation of the compatibility promise that we've upheld since 1.0.

Are you going to criticize C for marking gets() as deprecated?

Re:Zig: a system language which prioritizes optimality, safe... by fanf2 on Tuesday August 30, 2016 @06:49AM (#12388871)
C11 does not have gets() anymore!
Re:Zig: a system language which prioritizes optimality, safe... by SamReidHughes on Tuesday August 30, 2016 @02:51PM (#12392806)
Sorry, C will never live it down.
Re:Zig: a system language which prioritizes optimality, safe... by Manishearth on Monday August 29, 2016 @08:03AM (#12381294)
Not necessarily. I suspect even in the case of 2.0, most if not all of those apis will survive. Deprecation markers exist to encourage people to use newer, better APIs.

I suspect that even if 2.0 happened, the only apis that would be removed would be ones that we're reasonably sure nobody uses (via crater), and even then there would be debate.

And like Gankro said, 2.0 is a thing that's not supposed to happen in the first place.

Re:Zig: a system language which prioritizes optimality, safe... by steveklabnik on Sunday August 28, 2016 @10:08PM (#12379552)
To re-iterate what gankro said, we care a _lot_ about making sure that it's trivial to update your Rust. Your ABI compatibility comment is correct, but the rest of them aren't. We only break backwards compatibility in very few cases: if there's a soundness hole, or if we have to fix something in certain underspecified areas of the language. For those latter ones, if it's not a trivial fix, then it's a no-go, and even if it is, we do warnings until we don't see them in the ecosystem any longer.

But really, it's not about rules-lawyering over technicalities about what our policies are: the underlying attitude we share towards new Rust versions is that it should be trivial to upgrade. And we collect data to make sure that we're on target there: part of it is the train-based release strategy, part of it is stuff like testing everything on crates.io, part of it is things like the community survey, where most people said their code has never broken, and if it did, it was extremely easy to fix.

Re:Zig: a system language which prioritizes optimality, safe... by teacup50 on Monday August 29, 2016 @04:48AM (#12380771)
"Trivial to upgrade" is not the same as stable. It means code bitrots in-place.
Re:Zig: a system language which prioritizes optimality, safe... by Manishearth on Monday August 29, 2016 @08:00AM (#12381286)
True stability is impossible for almost any language out there, especially one with a nontrivial type system.

An example of a thing considered "trivial to upgrade" over is when the stdlib adds a new method to a type. Libraries may already have implemented a method of the same name on that type via a trait. In this case, folks upgrading would have to explicitly specify which method they're talking about.

It's these kinds of things that are not counted as breaking changes, because the alternative is freezing the stdlib forever. Other languages have the similar problems.

New releases are tested on the entire ecosystem to ensure that this rarely happens, if ever.

Re:Zig: a system language which prioritizes optimality, safe... by pcwalton on Monday August 29, 2016 @01:40AM (#12380212)
> In terms of API stability, they seem willing to drop features within a major release cycle or two.

This is completely false. Do you have an example?

Besides, this is completely off topic in a Zig thread. From the FAQ: "Zig is not afraid to roll the major version number of the language if it improves simplicity, fixes poor design decisions, or adds a new feature which compromises backward compatibility."

Re:Zig: a system language which prioritizes optimality, safe... by teacup50 on Monday August 29, 2016 @04:55AM (#12380784)
> This is completely false. Do you have an example?

Review the "Breaking Changes" of release notes, and the number of discussions regarding removing API after a short (in the view of someone who works on said "systems code") period of deprecation.

> Besides, this is completely off topic in a Zig thread. From the FAQ: "Zig is not afraid to roll the major version number of the language if it improves simplicity, fixes poor design decisions, or adds a new feature which compromises backward compatibility."

And thus, it won't be used as a systems language.

Re:Zig: a system language which prioritizes optimality, safe... by pcwalton on Monday August 29, 2016 @08:42AM (#12381439)
> Review the "Breaking Changes" of release notes

This is another instance where our being up front about any theoretical breakage leads to misunderstandings. Every new release of GCC and Clang has breaking changes in the Rust sense too--they just don't call them "breaking changes" (and we stopped doing that too, because of comments like yours). For example, I had some code break when upgrading GCC the other day due to newer versions becoming stricter about copy constructor semantics in C++11. In Rust we (used to) call this type of thing a "breaking change" out of an abundance of caution--even though it was nothing more than the compiler getting stricter about code that never should have compiled in the first place. Unfortunately being up front about it led to confusions and comments like yours.

Re:Zig: a system language which prioritizes optimality, safe... by eddyb on Monday August 29, 2016 @05:40AM (#12380904)
Are you referring to unstable APIs being deprecated and removed?

Technically we can remove those at any time (happens in librustc all the time), the deprecation period is to help and motivate migration of nightly users, nobody else can touch those APIs.

Re:Zig: a system language which prioritizes optimality, safe... by DSMan195276 on Sunday August 28, 2016 @09:15PM (#12379399)
I've thought alot about this before. Rust doesn't really have very great interoperability with C - most languages really don't, because the API for C programs revolves around functions, structs, macros, etc. Most languages can only really get part of that, and then the rest gets essentially a separate implementation in the language you're porting too - leading to a certain amount of manual duplication and intervention necessary (We don't call it porting when you use a C library in C). Design goals Rust has made compound the issue a bit more: For example, Rust uses the C++ ABI, not C. Rust can't generally use header files directly. The Rust<->C conversion loses a lot of information and can result in fairly messy code by Rust standards, etc.

This makes combining Rust into already existing C code a hassle, as it is generally worse then even just trying to use C and C++ together which is already fairly annoying and error-prone. This is compounded by Rust's safety goals which generally require a different design to a problem then the "C-style" approach would be - IE. You're probably not going to get good Rust code by simply replacing a .c file with a Rust file, because the entire time you're going to have to make use of the unsafe C functions that your program includes.

Perhaps the bottom line is that (for good reason) Rust is much more then just an improved C. And because of that, combining C and Rust is always going to feel like combining two separate systems together with a compatibility layer in-between - because that's really what it is - and that's just not very attractive.

I know little about Zig (I'm reading about it for the first time now) and don't at all predict it to replace C. That said, I would love a language which has a focus on maintaining very good compatibility with C while fixing the various pieces of its design that are fixable within the bounds of the language. For example, I noticed it mentioned in the README that in Zig that pointers are nonnull by default, and can be made nullable by adding the 'maybe' attribute. This is a feature I would love in C, but really isn't there (gcc has a nonnull attribute, but it is really essentially useless and nothing like you'd want). Essentially, I would like a language where it feels like writing code for the same system, but in a different (better) syntax - where I can drop it into a project and it works together with the system like any other .c file (Though with a different compiler, obviously), but the code inside is much better then what you can do in standard C. I think that such a language could retain the reasons why people (like me) still like C, while fixing a lot of the uglier sides to the language that everybody is aware of but aren't going to get fixed any time soon. And perhaps most importantly, I think such a language is definitely possible (Though it may not be able to employ all the features you may want) - but I don't have the time nor probably the skills to really do it well besides list off the things it should fix and shouldn't fix.

As for starting a project though, if you ignore difficulty to learn then I would agree with you: There's probably little reason not to just go for Rust if you're already willing to learn a new language. For most projects the issues I outlined above don't really matter assuming the entire project is written in Rust - besides the want for a language closer to C in design. That said, I readily concede that I have no idea if Rust's borrow-checking semantics could ever work without the extra features they added that take the design away from being 'C-like' - and considering that's one of the definitive features of Rust, is easily worth losing the 'C-like' detail if it is necessary.

But when considering a project like the Linux Kernel, the GNU coreutils, git, GTK, and other various large projects where conversion to Rust is probably impossible without a complete rewrite, being able to use a 'better' C while still retaining the aspects that people like would probably be a nice step in the right direction. The chances of it happening are nil, but it would still be a step in the right direction.

Re:Zig: a system language which prioritizes optimality, safe... by steveklabnik on Sunday August 28, 2016 @09:58PM (#12379521)

  > ust uses the C++ ABI, not C. 
Rust uses its own ABI, not any of the various C++ ones.

Our strategy for interop with C is two-fold: first, a very thin, direct wrapper. These are the various *-sys packages. They know how to link in (and maybe even build!) the underlying C library, and provide functions you can call from Rust. Then, on top, people can write a more idiomatic Rust wrapper, working in Rust's safety guarantees.

That takes some work and time to get right, but hopefully, it means that in the end, Rust users of C libraries shouldn't have to deal with the stuff you're talking about.

Re:Zig: a system language which prioritizes optimality, safe... by orbifold on Monday August 29, 2016 @02:08AM (#12380306)
That strategy is also used by Haskell, what typically happens is that the idiomatic wrapper ends up being out of date, poorly documented and if the library is prominent enough there might be even several competing ones.
Re:Zig: a system language which prioritizes optimality, safe... by steveklabnik on Monday August 29, 2016 @10:56AM (#12382236)
That's open source!
Re:Zig: a system language which prioritizes optimality, safe... by DSMan195276 on Sunday August 28, 2016 @10:05PM (#12379540)
You're 100% right. I was actually thinking of name-mangling when typing that, but that's still not quite right. The point I was trying (and failing) to make is that Rust uses things like name-mangling and a different ABI that makes interop more of a challenge - similar in challenge to interop with C++ from C.

C technically has name-mangling too, but generally speaking either there is no mangling at all, or they just add an '_' to every symbol name.

Edit: To address your second point, I think that is a step in the right direction - and I think that Rust's compatibility with C libraries is probably fine, and comparable to most languages. I don't consider that to be so big of a turn-off that I wouldn't want to use Rust for a new project, though obvious writing wrappers isn't always fun or error-free.

But, assuming I'm understanding what you're getting at correctly, I'm not sure that will really solve the core problem I'm trying to get at: Generally speaking, nobody wants to be maintaining compatibility wrappers for APIs that exist entirely within their program and are probably changing all the time, and that's really what you need if you want to replacing part of a C program with Rust.

For example, to take it to the 'extreme' - the Linux Kernel module API changes virtually every version of the kernel (And the ABI is not guaranteed at all), and maintaining a complete Rust wrapper for it would not be a fun time even if a certain amount of it can be auto-generated. The API includes a very complicated mess of functions, inline functions, macros, structures (with varying different types of alignment and padding). None of it is guaranteed to stay the same across versions. And I think it is fair to say that most C programs have internal APIs like this (Though not as crazy) that are changing all the time and not intended to be seen by the 'outside world'. It's these types of things that I see being a problem for interfacing with Rust - APIs that are changing all the time in complicated ways which make writing and maintaining a wrapper very annoying and error-prone.

Re:Zig: a system language which prioritizes optimality, safe... by steveklabnik on Monday August 29, 2016 @10:58AM (#12382247)
Oh, and to reply to your edit: yes, writing a wrapper for an unstable interface isn't exactly fun. But that also means that your C code is going to have to update with each release too, an unstable API is unstable for everyone.
Re:Zig: a system language which prioritizes optimality, safe... by DSMan195276 on Monday August 29, 2016 @02:47PM (#12384181)
A C API can stay relatively the same if you switch a function to a macro, or make it inline. In a lot of cases you don't actually care which of the options it might be to begin with - the syntax generally stays the same, and the situations where you care (Mostly just function pointers) are somewhat uncommon. Such changes would easily break a simple wrapper though. That said I do see your point - It's not like the C code is guaranteed to work either, so perhaps that is acceptable. The maintainer still has to weigh the disadvantages of supporting a Rust wrapper to the advantages of allowing Rust code.

My original point (which has gotten a bit muddled in the details) was just that there could be room for a language closer to C that offers to fix some of the more annoying issues, while still keeping very good compatibility with C overall and avoiding the need for 'wrappers' and such to interface it with C code.

Re:Zig: a system language which prioritizes optimality, safe... by steveklabnik on Sunday August 28, 2016 @10:10PM (#12379556)
It's all good! :)

Oh, and another thing that makes C interop hard that you didn't mention: unions. We have those coming, though, and it should help a lot.

Re:Zig: a system language which prioritizes optimality, safe... by grashalm on Monday August 29, 2016 @06:07AM (#12380971)
Nobody is ever 100% right. I have a hard time swallowing such sentences.
Re:Zig: a system language which prioritizes optimality, safe... by Nomentatus on Tuesday August 30, 2016 @10:23PM (#12395522)
Nobody is ever 100% right implies: Nobody is ever 100% right even when they say "Nobody is ever 100% right."
Re:Zig: a system language which prioritizes optimality, safe... by pcwalton on Monday August 29, 2016 @01:51AM (#12380251)
> But when considering a project like the Linux Kernel, the GNU coreutils, git, GTK, and other various large projects where conversion to Rust is probably impossible without a complete rewrite

Firefox is shipping Rust code right now. There are various examples of Linux kernel modules written in Rust. There are rewrites of the coreutils in Rust. I don't understand why you claim this.

Re:Zig: a system language which prioritizes optimality, safe... by DSMan195276 on Monday August 29, 2016 @03:09AM (#12380488)
> Firefox is shipping Rust code right now.

It is, and that is cool. That said, what it's shipping is essentially a separate library written in Rust that exposes a C API for parsing mp4s - it replaced one libraries usage with another, with a presumably sable API consisting of nothing but C struct's and C functions. Perhaps a key to point out is that it doesn't actually interface with Firefox very much at all - it isn't capable of accessing any Firefox state for instance. I also didn't list Firefox as you'll note - it's not a C project, it's a mishmash of languages already (Mostly JS and C++ from my understanding). I'm really talking about projects that are currently C-only adding in Rust code to the mix - they are structured much differently then a project like Firefox and the interfaces are generally much more complex as far as C goes.

I will add though, servo is making pretty impressive efforts. It is again a complete rewrite though, not an integration of Rust into Gecko which is the type of thing I'm talking about. A rewrite of a library in Rust against a stable API is generally possible, depending on the API - though the amount of work may make it prohibitive to get something usable.

> There are various examples of Linux kernel modules written in Rust.

I would like to see an example of a legitimate Linux Kernel module written in Rust that is in some form of use. I have never seen one besides a toy implementation and I can all but guarantee you it doesn't exist, for the reason that it would be way to much of a hassle to attempt to get it working with the mixture of macros, inline assembly, inline functions, gcc attributes, etc.

> There are rewrites of the coreutils in Rust

Coreutils is admittedly not a very good example, if only because coreutils isn't actually that big/complex of a project (Though supporting all the GNU flags and arguments is a pretty big task). They're also mostly just separate exe's anyway, so you could replace one or two with Rust code without much of a difference - I'd gladly remove Coreutils from the list if you would like.

Re:Zig: a system language which prioritizes optimality, safe... by Manishearth on Monday August 29, 2016 @07:41AM (#12381227)
> That said, what it's shipping is essentially a separate library written in Rust that exposes a C API for parsing mp4s - it replaced one libraries usage with another, with a presumably sable API consisting of nothing but C struct's and C functions.

This is true.

However, if we put aside the Rust code that is actually shipping, there's still plenty of work going on in sharing Servo's code with Gecko's.

One example of this is the ongoing experimental work to move Servo's style system into Gecko. Servo's style system doesn't have much of an "API surface"; everything is a surface. There's plenty of reaching in and grabbing Firefox state and vice versa.

Some of this is just done directly by reading or writing to structs. Some of this is done by writing small wrapper C functions (https://dxr.mozilla.org/mozilla-central/source/layout/style/...) and using bindgen. Bindgen has C++ method/ctor/dtor generation abilities that would obviate almost all of these manual bindings, though we aren't using them yet[1].

Overall, mixing C++ and Rust at a rough API surface in a large codebase hasn't been that hard. It's not easy either, but it's doable and I don't think it's anywhere close to being "nearly impossible without a complete rewrite".

[1]: The reason behind this has to do with name mangling -- Rust doesn't understand C++ name mangling, so bindgen generates the mangled function names and wraps them in a nicer API. This is all great, except that it means that the generated bindings stop being cross-platform, and you need to twiddle with the build system to either dynamically generate them or have a way to statically generate them for all platforms. Right now the servo-gecko integration uses a temporary build system that will be replaced soon, so this hasn't been a priority.

Re:Zig: a system language which prioritizes optimality, safe... by pcwalton on Monday August 29, 2016 @08:50AM (#12381481)
> macros, inline assembly, inline functions, gcc attributes,

Rust supports all of these features. So your complaint is that nobody has written a translator from these things to Rust yet. That's a matter of filing PRs against bindgen, not some basic problem with the language.

Re:Zig: a system language which prioritizes optimality, safe... by DSMan195276 on Monday August 29, 2016 @10:11AM (#12381934)
If you think it is so trivial, then why hasn't it been done yet?

Rust macros can't do all the same things that C #define macros can. You can't insert inline C code into Rust code without some serious work. And the fact is, as you pointed out, regardless of in the future, right now it doesn't work, and those features are absolutely necessary for writing something like a Linux Kernel module.

Re:Zig: a system language which prioritizes optimality, safe... by teacup50 on Monday August 29, 2016 @05:13AM (#12380831)
> I would like to see an example of a legitimate Linux Kernel module written in Rust that is in some form of use.

Seriously this.

I write an awful lot of kernel code. There's absolutely no way I'd try to convince folks to accept Rust code into the OS repository, where it will have to:

- Be maintained for decades

- Be ported to a litany of platforms, several of which may only have decent toolchains from the GCC department.

- Vend stable ABI

- Not require keeping a litany of compiler versions around just to keep older code building.

Re:Zig: a system language which prioritizes optimality, safe... by pcwalton on Monday August 29, 2016 @08:48AM (#12381474)
> - Not require keeping a litany of compiler versions around just to keep older code building.

Do you have a single example of this? We keep very close tabs on breakage in the wild even, and especially, for changes we were allowed to make (unlike say GCC, which happily breaks code if the language standard says the code never should have compiled in the first place). There are times when we've refused to make changes that we were allowed to make (because they were changes to unspecified behavior, which C/C++ has more of than Rust, and which GCC/Clang changes all the time) out of concern for breaking existing code.

As far as I'm concerned, honestly, this is just FUD.

Re:Zig: a system language which prioritizes optimality, safe... by teacup50 on Tuesday August 30, 2016 @05:14AM (#12388481)
Your own release notes contain descriptions of breaking language changes.

That may change as the language matures -- great. I keep an eye on Rust so that I can eventually try actually using it for kernel-level development work.

Re:Zig: a system language which prioritizes optimality, safe... by steveklabnik on Tuesday August 30, 2016 @09:27AM (#12389827)
Again, so do C compilers, so does Java, so does every language, even ones that "never break".

C11 removed gets, for example.

Re:Zig: a system language which prioritizes optimality, safe... by teacup50 on Tuesday August 30, 2016 @03:21PM (#12393065)
javac, gcc, and clang all support some form of -std=<lang_level>, and ABI/source interop between code that uses different language versions.

The post-gcc 2.95 C++ ABI breakage was pretty suck, though.

[edit]

And please don't misinterpret my criticism as intentional FUD.

I work on $MAJOR_OS and am achingly tired of writing C, so I keep my eye on Rust as one possible salvation, once it's had time to mature.

It's just a very hard sell when there's no ABI stability, we're talking about code that's expected to last for decades, etc.

Re:Zig: a system language which prioritizes optimality, safe... by steveklabnik on Tuesday August 30, 2016 @05:09PM (#12393919)
It's all good. I hear you. We'll get there :) It took C what, 17 years or something to have ABI stability?
Re:Zig: a system language which prioritizes optimality, safe... by pcwalton on Tuesday August 30, 2016 @05:57PM (#12394282)
As I explained, those are only "breaking changes" in the sense that some code might break in practice, not that we changed the language definition.

GCC did not provide a switch to get the old copy constructor behavior back when they made it stricter and broke my code.

Re:Zig: a system language which prioritizes optimality, safe... by pjmlp on Monday August 29, 2016 @10:58AM (#12382250)
Hence why it is a failure trying to convince UNIX users to use anything other than C.

C replacements will need to succeed in other OS not married with UNIX culture.

Which means Swift, .NET Native, Java/C++, C++17, on the OSes from Apple, Google and Microsoft.

Re:Zig: a system language which prioritizes optimality, safe... by teacup50 on Tuesday August 30, 2016 @05:11AM (#12388470)
> Which means Swift, .NET Native, Java/C++, C++17, on the OSes from Apple, Google and Microsoft.

You've seen where their OS code comes from, and what it's written in, right?

Just to be very clear, I'm very well-versed in life outside C and imperative programming. This isn't "UNIX culture". This is "systems programming" culture, and it's simply pragmatic.

Re:Zig: a system language which prioritizes optimality, safe... by pjmlp on Tuesday August 30, 2016 @07:39AM (#12389117)
Yes I have seen where their OSes come from.

Mac OS X and their predecessor might have an UNIX heritage, but C was always left for the very lowest layer. Already on NeXT the device drivers were written in Objective-C, being replaced by C++ on Mac OS X.

Anyone paying attention to their Swift talks during the last two WWDCs knows where the boat is steering. Cris is quite clear in stating Swift should be usable in all scenarios where C is being used and Sierra already got some adoption in userland components like the dock and launch deamons, now rewritten in Swift.

Microsoft has declared C89 as good enough with the future being C++ and .NET Native.

The C runtime library was rewritten in C++ with extern C for the public symbols.

The C99 compatibility and upcoming C11 are only done to the extent required by ANSI C++. For anything else there is clang.

As of Windows 8, the device driver framework has been changed to C++ and there was a talk from Herb Sutter where he mentioned the plan was to migrate the kernel to compile with a C++ compiler.

The idea of Core C++ Guidelines actually originated at Microsoft, before Bjarne and CERN guys got involved.

Google doesn't allow native code on ChromeOS and on Android they make pretty clear that the NDK is just to make game developers happy and nothing else.

They are all aware that C isn't going away tomorrow, but are driving efforts to make it as relevant in the future on their platforms as Assembly is today.

When I started working in IT, the only OS written in C was UNIX.

The language is not a sacred cow and the only thing preventing replacing it is the ubiquity of UNIX like OSes.

OS not bound to the UNIX culture and POSIX compatibility are free to chose other language as their systems language.

Not doing so, is usually a decision to cater to the status quo and ubiquity of existing developers and library (with their endless CVE entries).

Re:Zig: a system language which prioritizes optimality, safe... by teacup50 on Tuesday August 30, 2016 @03:23PM (#12393094)
C++ isn't substantially different from C when it comes to systems programming concerns regarding stability.

The biggest issue is that it's a bit harder to maintain ABI compatibility. You have to carve out reserved vtable space, avoid exposing STL in your interface, etc, but it's doable.

As for Swift, it has heavy userspace dependencies that make it non-viable for kernel work. Rust does much better there.

Re:Zig: a system language which prioritizes optimality, safe... by pjmlp on Tuesday August 30, 2016 @06:55PM (#12394619)
Depends if developers insist on C with Classes or are clever to take advantage of the language features that make it a safer language.

Regarding Swift, C developers used to tell me C++ would never be used to write kernels, because it had a runtime and was too bloated.

Since those days, I have seen BeOS, Symbian, Genode, Mac OS X IO Kit, Windows 8+ Kernel Driver Framework.

Also not to forget, Objective-C kernel code used in NeXT.

Also the biggest open source C compilers are nowadays written in the language which those C developers considered too bloated for any real work.