News for hackers. Stuff that matters.
Re:Integrating “safe” languages into OpenBSD? by akerro on Monday December 04, 2017 @08:59AM (#15843817)
>Such ecosystems come with incredible costs. For instance, rust cannot >even compile itself on i386 at present time because it exhausts the >address space.

Is cargo supported on i386 platforms? Also Rust complies itself, afaik there is no way to compile Rust/Cargo but to use previous version of it. If one of the past builds of Rust is backdoored, any version between then and now is backdoored, language is safe, environment... as safe as it was never compromised. OpenBSD compiles everywhere where C code works, Rust/Cargo works where it's supported and it will takes decades to catch-up on some architectures.

Re:Integrating “safe” languages into OpenBSD? by steveklabnik on Monday December 04, 2017 @09:53AM (#15844213)
Okay, there's a lot of misunderstandings here. Theo is right about some things, and wrong about some things. And people are misunderstanding what things he's right about. The things he's wrong about are very minor.

Rust absolutely works on 32-bit platforms, though we often use the i686 target rather than an i386 one. Platform support list is here: https://forge.rust-lang.org/platform-support.html

Theo is talking about building rustc, not compiling most Rust programs. That's the first distinction that it seems like many people are missing.

However, apparently the compilation process OOMs when building on a i386 box. I don't use those platforms, but I'd believe it. The Rust compiler is large. However, I thought (and looking at our CI, this seems to be true https://travis-ci.org/rust-lang/rust/jobs/311223817) we do compile with an i686-unknown-linux-gnu host (for this build), so I dunno. Maybe it was a fluke, maybe I'm misunderstanding, I'm not sure.

We often provide artifacts via cross-compiling, but this is unacceptable to OpenBSD. That's totally okay. They have good reasons for doing this.

Re:Integrating “safe” languages into OpenBSD? by HelloNurse on Monday December 04, 2017 @11:07AM (#15844874)
If compilation of rustc runs out of memory it represent an upper limit on the complexity of actually viable Rust programs, and given how much software is larger than a compiler it is a discouraging performance level.
Re:Integrating “safe” languages into OpenBSD? by steveklabnik on Monday December 04, 2017 @11:31AM (#15845083)
Rustc is more than just a Rust program; we compile LLVM from scratch, for example. Is the OOM in the Rust code, or in the LLVM code, or in the final linking, or what? It's not clear.

The compiler is one of the largest Rust programs that exist. Last I checked, it was three quarters of a million lines of Rust, but a quick loc shows 1.5 million lines of Rust, 2.3 million lines of C++, and 900,000 lines of C (again, mostly LLVM and jemalloc).

Servo is also very large, and they don't report having OOMs, though I'm not sure if they build on 32-bit or just cross-compile.

Re:Integrating “safe” languages into OpenBSD? by ratmice on Monday December 04, 2017 @07:07PM (#15848735)
FWIW, i've compiled llvm/rust with 2G, I never ran out of ram compiling rust, but in llvm gnu-ld would run out of memory, using the gold linker fixes that, as well as the configuration flag for enabling separate debug info -DLLVM_USE_SPLIT_DWARF=ON
Re:Integrating “safe” languages into OpenBSD? by MageSlayer on Monday December 04, 2017 @11:52AM (#15845277)
Agree. Moreover, Rust compiler contains some dark areas which nobody wants to deal with. See https://github.com/rust-lang/rust/issues/38528 for example. Basically it means that Rust compiler can suddenly take exponential time and space for compilation.

That bug really bites hard any code heavy on iterators (Rust often praised feature!). It has reliable reproduce test-case, but still it's already year old and was down-prioritized!

Hard to believe anybody uses Rust for real large project given so little attention to crucial details.

Re:Integrating “safe” languages into OpenBSD? by steveklabnik on Monday December 04, 2017 @12:04PM (#15845380)
I mean, that thread has a comment less than a day ago, and Niko says:

> I'm going to lower this from P-high to reflect reality. I'm still eager to investigate but haven't had time, and making it P-high is not helping =)

P-high means someone is actively assigned and working on it, so yeah in some sense this is a down-prioritization, but only from "Someone is working on this, so put your work somewhere else" to "this is open to work on"; the de-prioritization may lead to it getting fixed sooner, as Niko is a busy guy.

So, "nobody wants to deal with" feels like a mischaracterization to me here.

Re:Integrating “safe” languages into OpenBSD? by MageSlayer on Monday December 04, 2017 @01:01PM (#15845942)
Well, yes. The last comment says the issue is still there :) I mean this bug alone in fact nullify the entire incremental compilation effort. It's kind of weird.

> The de-prioritization may lead to it getting fixed sooner, as Niko is a busy guy

And Niko put "medium" priority month ago :)

Re:Integrating “safe” languages into OpenBSD? by oblio on Monday December 04, 2017 @11:46AM (#15845235)
True, but we're talking about 32 bit systems. Chrome stopped being able to compile on 32 bit systems years ago, as a C++ programs. I think Firefox is in the same situations.

It does sound kind of bad for Rust, but the competition isn't doing much better :p (weak excuse, I know)

Re:Integrating “safe” languages into OpenBSD? by zbentley on Monday December 04, 2017 @07:01PM (#15848711)
There is a really, really substantial difference in the "required for basic credibility/usage" qualifications of a web browser versus an operating system. Operating system instances can run for decades, executing arbitrary tasks, without needing to be restarted. Web browsers need to be updated incredibly frequently just to remain functional for parts of the internet.

That's not to say either one is "worse" or "better", but comparing the two on an axis like platform support is like comparing tomahawk missiles and tall ships. Totally different requirements and use cases.

Re:Integrating “safe” languages into OpenBSD? by glandium on Monday December 04, 2017 @05:09PM (#15848037)
Travis is running containers, so even if you use a i686 rustc, you still benefit from a 64-bits kernel, meaning processes still have the full 4GB of address space. On an actual i386 linux kernel, this would be limited to 3GB. Maybe it's even less on openbsd (I don't know, but technically, it could be as low as 2GB). That could explain why it works for you and not for them.
Re:Integrating “safe” languages into OpenBSD? by steveklabnik on Monday December 04, 2017 @05:13PM (#15848058)
Ah, right. Thanks.

I still thought that we generally kept it down to around 2GB of space, but maybe that's wrong.

Re:Integrating “safe” languages into OpenBSD? by hood_syntax on Monday December 04, 2017 @09:10AM (#15843892)
I'm pretty sure that attack you describe is mentioned in literature as essentially undefeatable. I really wish I could remember exactly what the name of it was; the gist is, there has to be a first compiler somewhere. If at any point in the chain, the compiler is infected with a self-propagating virus that hides itself in the byte code of the binary, it can ensure that the exploit is in every future version of the compiler.

I may be remembering the details a bit wrong, but it was a good read.

Re:Integrating “safe” languages into OpenBSD? by zdw on Monday December 04, 2017 @09:17AM (#15843942)
You're looking for "Reflections on Trusting Trust" by Ken Thompson, one of the original co-authors of Unix:

https://dl.acm.org/citation.cfm?id=358210

Re:Integrating “safe” languages into OpenBSD? by unhammer on Monday December 04, 2017 @09:17AM (#15843941)
There is a possible defense: https://www.dwheeler.com/trusting-trust/
Re:Integrating “safe” languages into OpenBSD? by steveklabnik on Monday December 04, 2017 @09:54AM (#15844223)
... and the hope is that https://github.com/thepowersgang/mrustc will let us do this for Rust.
Re:Integrating “safe” languages into OpenBSD? by edwincheese on Monday December 04, 2017 @09:16AM (#15843938)
I think you are referring to “Reflections on Trusting Trust” by Ken Thompson.

https://www.ece.cmu.edu/~ganger/712.fall02/papers/p761-thomp...

Re:Integrating “safe” languages into OpenBSD? by nickpsecurity on Monday December 04, 2017 @11:52AM (#15845281)
It's not undefeatable. The guy who invented it in 1970's, Paul Karger, told people the concepts on how to defeat it right afterward. Their advice for building systems in a way to catch a lot of subversion was encoded in the first standards for information security. I included most of those methods in my security framework here:

https://pastebin.com/y3PufJ0V

https://en.wikipedia.org/wiki/Trusted_Computer_System_Evalua...

One compiler made to the highest standard in development assurance is CompCert.

http://compcert.inria.fr/

It has specs of everything it does, proofs it does it in tool with minimalist checker, extracts to ML that can be compared against those specs in various ways (eg visually), can optionally be compiled with a mini-ML compiler or Scheme you homebrew yourself, and passed exhaustive testing by third party with only a few bugs in specs (not code). There's another using formal specs called KCC which could be ported to something bootstrappable like META II or Scheme.

The other requirement from TCSEC was that source be supplied to customer to build with their onsite, trusted tools. I looked into even having compilers done in Perl since it's already widely deployed. David A. Wheeler made brilliant suggestion of either bash or awk. I have put tools for those and more on rain1's bootstrapping page. rain1 or someone there called the concept "Ubiquitous Implementations." Note we've focused on technical content, not presentation, on that one being busy folks. Looks rough. :)

http://bootstrapping.miraheze.org/

You also need repo security to protect that source with it either cryptographically sealed and/or sent over secure transport. Link below on repo security from David A. Wheeler. Quite a few forms of transport security now.

https://www.dwheeler.com/essays/scm-security.html

After Thompson wrote on Karger's attack in 1980's, it took a life of its own among people that continue to mostly ignore the prior solutions. It's a problem absolutely solved to death starting with the person who discovered it in MULTICS Security Evaluation. Far as state-of-the-art, the current path of research is exploring how to integrate solutions for many languages, models, or levels of detail in one picture of a system with no abstraction gap attacks with proof of that for all inputs. That's a bit more complex but just an imperative language to assembly delivered and bootstrapped? Straight-forward but tedious, time-consuming work the first time it's done. :) Also, expensive if you buy CompCert which is only free for GPL stuff. Two of us are eyeballed CakeML's lowest-level languages as a cheat around that for verified bootstrapping.

http://cakeml.org/

EDIT: Btw, all that is technical discussion and argument. For fun, you might enjoy the story "Coding Machines" which is about only coding-related story I started reading and couldn't put down. Probably took an hour to read. It covers discovery of a Karger-Thompson-style attack along with how people might respond mentally and in terms of solutions. Some other stuff in that one.

http://www.teamten.com/lawrence/writings/coding-machines/

Re:Integrating “safe” languages into OpenBSD? by traverseda on Monday December 04, 2017 @09:13AM (#15843908)
You want the "trusting trust" keyword.
Re:Integrating “safe” languages into OpenBSD? by skj on Monday December 04, 2017 @09:13AM (#15843916)
A talk by Ken Thompson, "trusting trust".
Re:Integrating “safe” languages into OpenBSD? by mcguire on Monday December 04, 2017 @10:30AM (#15844531)
Technically, his Turing award lecture.