News for hackers. Stuff that matters.
Re:Don't run 'strings' on untrusted files by blinkingled on Saturday October 25, 2014 @01:54PM (#8508835)
It's not the utilities themselves that are the bigger problem - rather a library the utility uses. libbfd in this case. Problem with writing libraries in anything other than C of course is that they are not very usable by a large variety of software.
Re:Don't run 'strings' on untrusted files by steveklabnik on Saturday October 25, 2014 @02:07PM (#8508879)
It's pretty easy to call into a Rust library from C. The lack of a garbage collector makes embedding Rust quite nice.

Lately I've even been playing with a Ruby gem that's a C extension that calls into a Rust .a that exposes itself via extern C.

Re:Don't run 'strings' on untrusted files by Animats on Saturday October 25, 2014 @03:04PM (#8509018)
I just hope the Rust developers don't screw up. We badly need a safe, fast, low-level language to replace C.

Once Rust settles down, the next step is academic work to automatically translate existing C into Rust.

Re:Don't run 'strings' on untrusted files by pjmlp on Saturday October 25, 2014 @03:50PM (#8509192)
Ada, Modula-2, Object Pascal, Oberon, Modula-3, ...

Sadly, the list of better system programming languages that the industry decided to ignore is quite big.

Re:Don't run 'strings' on untrusted files by dbaupp on Saturday October 25, 2014 @06:03PM (#8509541)
Ada (and I guess the other ones) are unlikely to be memory safe in the presence of arbitrary pointers without a GC. That is, having a pointer/reference deep into a vector or some nested element of a tree requires a garbage collector or is unsafe (runs the risk of becoming dangling). So, while they are possibly better than C in many ways there are others in which they are lacking.

This lack of references means code is forced to do more copies or data-structure lookups.

(I dont actually know Ada or those other languages, but I did do some research a while ago and discussed this publically here and on /r/programming a few times, and have never been corrected, so I guess it is close to correct.)

Re:Don't run 'strings' on untrusted files by kyllo on Saturday October 25, 2014 @06:29PM (#8509601)
having a pointer/reference deep into a vector or some nested element of a tree requires a garbage collector or is unsafe (runs the risk of becoming dangling)

Yes, this is a problem, certain data structures need to be intrusive in order to be fast. Rust for example is a highly memory-safe language but its "unsafe" mechanism needs to be used in order to implement performant data structures for this reason.

http://www.reddit.com/r/rust/comments/2jec05/problem_with_im...

Re:Don't run 'strings' on untrusted files by dbaupp on Saturday October 25, 2014 @08:12PM (#8509856)
That is not what I'm talking about. e.g. (in Rust)

  let map: TreeMap<uint, SomeHugeThing> = make_map();

  let value: Option<&SomeHugeThing> = map.find(&123);
`map` is a (recursive) tree that associates a `uint` key with a value of type `SomeHugeThing`, imagine that is, e.g. 1KB, or otherwise expensive to copy. The `value` is a direct pointer to the memory of the value associated with the key 123 (if it exists), that is, it is extremely cheap to manipulate `value` because it is basically a machine pointer directly into the memory of the TreeMap. Rust gives you the power for that to be perfectly safe without a GC: there's no risk that changes to the `map` structure will cause the `value` pointer to be invalidated.

As far as I know, Ada etc. do not allow for this without a GC. That is, there's no way to have a safe pointer directly into the memory controlled by some dynamic data structure. Instead, one would have to search the `map` each time the value is wanted, or turn on GC.

> its "unsafe" mechanism needs to be used in order to implement performant data structures for this reason.

Implement some performant data structures. There's a lot of data structures that can be implemented performantly without begin intrusive (sure, one might want to use `unsafe` occasionally to optimise them fully, but the `unsafe` is almost always not being used to make it intrusive).

Also, I don't understand the relevance of that link, the top comment (which is mine, btw) clearly demonstrates that `unsafe` is not necessary.

Re:Don't run 'strings' on untrusted files by pjmlp on Sunday October 26, 2014 @02:28AM (#8510390)
Ada specification allows for a GC, but few implementations provide one.

Oberon and Modula-3 have GC and had real usable OS implemented in them, not just some kind of concept OS.

However those languages already offer the following in terms of security over C:

- String data type

- Open arrays (aka slices nowadays)

- Reference parameters (no need to pass pointers to functions/procedures)

- Bound checked arrays (you can bounds checking off, if you really need it)

- Pointer arithmetic is explicit operation

- Casting between types is explicit

- Enumerations are their own types, there is no implicit conversion to/from ints.

While they still don't cover all use cases in terms of memory safety, they already cover quite a few scenarios that in C just lead to unsafe code without the help of a static analyser.

Re:Don't run 'strings' on untrusted files by vezzy-fnord on Saturday October 25, 2014 @04:23PM (#8509280)
All but one of these are Wirthian languages. They all had overhead for their time period, and many of them were academic rather than pragmatic (consequently becoming influential for language designers).

Ada? You're trying too hard. It used to be a government standard and wasn't ignored in the least (see: http://www.seas.gwu.edu/~mfeldman/ada-project-summary.html)... much to a lot of people's chagrin. It's widely regarded as an example of a monster language, but it is still absolutely used where the safety is critical.

Re:Don't run 'strings' on untrusted files by pjmlp on Saturday October 25, 2014 @04:54PM (#8509366)
> They all had overhead for their time period, and many of them were academic rather than pragmatic

What overhead? The one spread by C crowed without experience in the said languages?

I used a few commercial compilers for those languages. They were quite comparable in terms of generated code quality to C compilers of the same generation, back in the day.

C was also a research language until AT&T made the code available.

> It's widely regarded as an example of a monster language, but it is still absolutely used where the safety is critical.

Actually, I would dare to say that Ada 2012 is smaller than C++14.

Its use has increased in Europe thanks to what is being discussed here, namely the amount of money lost in security issues thanks to the industry adoption of C due to its relation to UNIX.

I included Ada in the list, because most developers aren't aware that it still exists and is being used. Or that GNAT is just one of many compilers that are still available.

Re:Don't run 'strings' on untrusted files by blinkingled on Saturday October 25, 2014 @02:15PM (#8508903)
Things get uglier when I want to then call the whole shebang from - let's say - Go! Go -> C -> Rust. Now I am paying the penalty for 2 runtimes!
Re:Don't run 'strings' on untrusted files by steveklabnik on Saturday October 25, 2014 @02:54PM (#8508995)
Nope, you can use Rust without the runtime, so you're not _forced_ into paying the overhead.
Re:Don't run 'strings' on untrusted files by blinkingled on Saturday October 25, 2014 @02:59PM (#8509004)
So the Rust library you write is fully self sufficient without calling into any Rust stdlib functions or without using any runtime initialization mechanisms like ctors and dtors? That sounds too good to be true - If I am calling Rust then I am getting the Rust runtime just like if I can strlen I am getting the C library.
Re:Don't run 'strings' on untrusted files by dbaupp on Saturday October 25, 2014 @05:45PM (#8509502)
You can use a significant portion of Rust with only libc (i.e. no different to C) and this will be increasing. Things like "constructors" and destructors do not need the support of a runtime at all, they're entirely just conventional static function calls.

That is to say, the only overhead/problem with calling using C vs. using C and Rust is the increased binary size of having additional code included. There's no loss of flexibility (as long as you're not trying to run on a tiny microcontroller).

Re:Don't run 'strings' on untrusted files by blinkingled on Saturday October 25, 2014 @08:10PM (#8509845)
That does sound great overall - and I will take a deeper look at Rust - but this thread is about rewriting the core Linux utilities in Rust - if I had to do that with full featured Rust it would be hard enough. But if I had to restrict myself to a subset to avoid the "runtime penalty" for programs embedding my Rust shared library implementation of libbfd - then the rewrite just went from nearly impractical to completely impractical in terms of the efforts required.
Re:Don't run 'strings' on untrusted files by dbaupp on Saturday October 25, 2014 @08:26PM (#8509895)
> But if I had to restrict myself to a subset to avoid the "runtime penalty" for programs embedding my Rust shared library implementation of libbfd

The point is there is no 'runtime' penalty with Rust†. It doesn't have a compulsory garbage collector, it doesn't have a compulsory complicated IO manager, it doesn't have a compulsory multiplexed threading system. In some circumstances lacking any of those will be a downside, of course, but not having it built-in and compulsory means Rust gains flexibility. In several ways, there's actually lower overhead with Rust, because it is a modern language designed around the advances and research that has happened, e.g. dynamic (de)allocation can be faster due to putting some sizedness requirements on the very lowest level APIs (which one rarely calls directly, so the programmer will never notice the restrictions, just the improved speed).

My comment was trying to address the Go <-> C <-> Rust bridge, pointing out that C and C <-> Rust are essentially the same, other than the extra code one must have from writing in two languages rather than one. If one was to forgo the C (which is possible, Rust can easily be used to write a library directly exposing a C ABI), there won't be much difference at all between C and Rust.

In summary, in future, any restrictions required would not be very significant, and, even now before Rust's runtime has been excised, the restrictions will be things like "no network IO", one still gets access to all sorts of fancy iterators and data structures without requiring any runtime.

†Strictly speaking, this isn't quite true right now, but there is a concrete plan currently being executed to make it true.

Re:Don't run 'strings' on untrusted files by blinkingled on Saturday October 25, 2014 @09:12PM (#8509972)
Wow, in that case Rust is looking like it could be hell of a lot better C replacement! If I could just write all my libs in Rust using the full power of the language and then just call them easily from any language without ungodly overhead or complexity like in case of say Java/JNI, that's nothing short of fascinating!

Thanks for taking the time to explain.

Re:Don't run 'strings' on untrusted files by dbaupp on Sunday October 26, 2014 @02:11AM (#8510358)
It's not all brilliant though, there's still a lot of useful tooling that Rust needs to develop to make it really nice, e.g. there's no nice way to create a C header file describing a Rust library (since C headers are the lingua-franca of FFI libraries, in many ways), https://github.com/rust-lang/rust/issues/10530 .

And, being a young language, it's not too hard to be doing something no-one else has ever done, especially with this low-level stuff. :) (Meaning compiler bugs and some "I don't know" answers.)

Re:Don't run 'strings' on untrusted files by steveklabnik on Sunday October 26, 2014 @02:29PM (#8511800)
By the way, you may be interested in https://github.com/uutils/coreutils
Re:Don't run 'strings' on untrusted files by Dylan16807 on Saturday October 25, 2014 @03:37PM (#8509140)
You can use the core library to get standard functions with no runtime. I'm not sure what you're saying about constructors and destructors. There's nothing special about calling strlen that adds overhead.
Re:Don't run 'strings' on untrusted files by bronson on Saturday October 25, 2014 @03:14PM (#8509049)
C is fully self sufficient without calling into libc. Otherwise all the libc alternatives couldn't exist.

Sure, you don't get to call strlen (actually, you do if the compiler treats it as an intrinsic), but that's no big deal.

Re:Don't run 'strings' on untrusted files by blinkingled on Saturday October 25, 2014 @03:38PM (#8509142)
So we would need to write Rust libraries without using Rust stdlib? Not exactly a great advantage in that case. Especially so when you are rewriting a ton of C code.
Re:Don't run 'strings' on untrusted files by bronson on Saturday October 25, 2014 @06:37PM (#8509617)
So what are you complaining about? Do you want to use runtime libs or not? It sounds like you're dissatisfied either way.
Re:Don't run 'strings' on untrusted files by blinkingled on Saturday October 25, 2014 @08:01PM (#8509828)
It's about the context - rewriting ton of code that is in C libraries right now. If that had to be done using Rust without its runtime libs it is an huge disadvantage to solve the issue at hand.
Re:Don't run 'strings' on untrusted files by SwellJoe on Saturday October 25, 2014 @02:36PM (#8508956)
How high is that penalty, relative to, say, loading a variety of C shared libraries? At some point safety and security trumps saving a few kilobytes of memory...in a server environment, that point probably has already been passed a decade ago. Of course, if the penalty is hundreds of megabytes, then the equation might add up differently.

But, we already have a huge number of system tools and utilities running in a huge number of languages. Python has become the lingua franca of Linux management tools; Perl was in that role in the past, and still exists in a lot of places; bash and shell are integral. All have separate runtimes, and sometimes interact with system-level C libraries, either directly or through command line interfaces.

Is it really that big of a deal to have a Rust or Go shared library in a system that already has a half dozen different languages and a variety of shared and unshared libraries existing at once? I suspect it would be unnoticeable on a modern system. I'd choose safety over shaving a few kilobytes of memory used.

There would be a cost in having that interface friction for developers...and we'd be paying that cost for years. But, it seems like both Rust and Go have planned for the languages to be integrated with C libraries from the beginning, so it seems less of a problem. If I were going to attempt such a thing, I'd probably start with the front end utilities that use the libraries and then work my way down to converting the libraries (even though the libraries, in this case, are where the problems lie). But, maybe it's possible to make Go or Rust code that provide the same C interface as the existing libraries...I don't know enough to even guess.

Re:Don't run 'strings' on untrusted files by dllthomas on Saturday October 25, 2014 @03:28PM (#8509103)
Arguably 3.
Re:Don't run 'strings' on untrusted files by gizmo686 on Saturday October 25, 2014 @01:58PM (#8508851)
Most languages have some type of "extern C" as part of their FFI. It should be possible to use C as a langua franca between two non C languages.
Re:Don't run 'strings' on untrusted files by blinkingled on Saturday October 25, 2014 @02:12PM (#8508893)
> It should be possible to use C

Ugh, wouldn't that fall in "just because you can do something doesn't mean you actually should" category? :) Suppose I wrote the library in Go and exported some type of C wrapper and then I loaded that in Rust using Rust->C mechanism - that will load the Golang runtime in Rust! And you still got non-trivial C code to deal with anyways!

Re:Don't run 'strings' on untrusted files by gizmo686 on Saturday October 25, 2014 @04:23PM (#8509278)
If you want to interface two languages, then I do not see a way arround involving the runtime of both languages (in the same way that linking to C code still uses the C runtime). However, using my method does not actually involve any C code. "extern C" makes the compiler produce object code that can be called as if it was produced by C code; it does not produce any C code that would compile to the given object code.

Of course, it will also produce C headers to give the type information of the exported symbols, but I wouldn't call that code.