Meanwhile, you are just being snarky on a message board while using a whole stack of software that was built by the programmers whose work you are criticizing.
Rust looks great for apps and middleware. I don't see ffmpeg or x264 (both of which are implemented with significant amounts of architecture-specific machine code -- an environment where neither C nor Rust is going to be able to help avoid the kind of bugs we're talking about) jumping ship any time this decade.
I'm not claiming it's production quality, but I did write an MP2 decoder in an early version of Rust: https://github.com/pcwalton/fempeg
> Rust looks great for apps and middleware. I don't see ffmpeg or x264 (both of which are implemented with significant amounts of architecture-specific machine code -- an environment where neither C nor Rust is going to be able to help avoid the kind of bugs we're talking about) jumping ship any time this decade.
I grepped through the commit log for bugs marked "j00ru", as suggested in the post, and the first 20 I found were all mostly-architecture-independent, straightforward C code (often bitstream parsing code). This is the kind of problem that Rust is generally good at securing and making performant. None of the 20 patches I looked at touched any assembly code at all.
But the macro point is just empirical: the proof that Rust is an appropriate tool for codec development needs to come in the form of a production-quality codec implementation of some non-trivial standard (VP9, say), not a HN post.
I don't understand this. If you write in a safe language, the compiler rules out the memory safety bugs, regardless of the problem.
> But the macro point is just empirical: the proof that Rust is an appropriate tool for codec development needs to come in the form of a production-quality codec implementation of some non-trivial standard (VP9, say), not a HN post.
Of course, no argument there. I am reasonably confident that we can get there with tuning though, based on our benchmarks so far. :)
For performance sensitive code, you sometimes need to deliberately step out of the strongholds of the compiler.
...or just add more primitives to the language.
If you mean "constant" primitives, i.e. things which don't require to expand the language grammar or type system, then it's a question of future prevalence of usage of said primitives. If they become almost ubiquitous to any project, then they obviously need to be included in the standard libs. If you're talking about a new language feature, that depends on how much that new feature may help design and write future libraries, and also what are the aims of said language. I understand that rust aims to become a safer system language, so any feature helping the design of low level, performance critical code should be welcome.
OP is criticizing without a plausible alternative; Rust is at least more plausible of an alternative. But I also suspect that C+ASM will continue to rule the day here, especially now that these fuzzing techniques are become more widespread and industrial-strength.
"You will never find a programming language that frees you from the burden of clarifying your ideas."
> If you're a C programmer and somebody says like: I've that optimization that makes your program 3% faster you go: wow! Then you come and say: well, now I make it half as fast you go: w00t?! But then you remember that people actually use ruby to serve web pages.
I'd say this would be actually worth it for a far more secure FFmpeg.
It's been around since 2006, but I haven't really heard of anyone using it. I wonder why.
It never compiled on 64bit architectures, and to get it to compile with 4.0 or 4.1 (newer versions of gcc won't work) you need to fetch it from SVN (last commit is dated May 2009)... and obviously: no distribution still ship such an old version of the build tools... you'd need something like debian lenny, which does not even receive security updates anymore
This is like saying "my bulletproof vest will stop nearly every round fired at it". Too bad it only takes ONE...
But Modula-2 had Lilith to offer, while C had UNIX.
Compiler support across various platforms: are there compilers available and do they generate good code?
Mindshare: how big is the intersection of people who know the language and have the domain knowledge to contribute to the project?
Or are you perhaps just wishing that history had taken a different turn?
There'll be a HN topic A FFmpeg reimplementation in JavaScript soon enough ...
There is also Intel Lab's HRC (Haskell Research Compiler) that does have SIMD capabilities [2]. Unfortunately, there isn't a public release of HRC yet.
---
[1]: https://ghc.haskell.org/trac/ghc/wiki/SIMD
[2]: http://dorchard.wordpress.com/2013/10/14/automatic-simd-vect...
ISO/ANSI C also don't support SIMD, you have to go down to Assembly when writing portable code across C compilers.
also, I think you may be able to do reasonably well with icc and not much assembly
You don't need to work at the assembly level for this; you just need a language that can express vector operations in a way that doesn't require a compiler to solve the halting problem, and you need a compiler that knows how to compile such operations to SIMD.
Yes, ANSI C doesn't have such a construct, but GNU C (which I'd bet 95% of open-source code uses anyway) does.
(Plus x86's intrinsics are just plain ugly. It's easier to read asm.)
As for the architecture-independent SIMD in GNU C, it's something, but not quite flexible enough. Even C is really not a good enough language to consider implementing this in, because it has an overly-conservative memory model when it comes to char *, which aliases all other memory. For a library dedicated to parsing things which come in the form of bytestreams and 8-bit pixels, this means that most compiler optimizations are defeated right in the inner loops where you'd need them.
But restrict is only good in specific situations - if you have three pointers, and #2 can alias #1 but not #3, there's no way to express that.
It also tends to be used on function arguments, not at some higher up point of declaration. So when compilers inline the function and can see more global info about it, the restrict gets lost and optimization gets worse.
I think some kind of stronger 'typedef' would be better.
[1] http://msdn.microsoft.com/en-us/library/5ft82fed(v=vs.80).as...
map (uncurry (+)) (zip [1,2,3] [4,5,6])
The compiler needs to be able to see that the data conforms to all those restrictions (possibly more), and that's not a trivial thing to do.
Here's an example in Haskell: consider the case that the list of numbers was generated from a list of some other structure; something like "map x coordinates" to get all the X values from a list of X-Y-Z coordinates. For vectorization to work, those values need to be packed together in memory; however the structure isn't organized as such (at best, you'd have XYZXYZXYZ). That means a nontrivial memory copy, which will cost much more than the vectorization will gain you. So the structure needs to be reorganized, but then that might cause other pessimizations.
For the remaining 95% of the code, since 1978 one could use Modula-2, Ada, Pascal dialects, SafeC and many other system programming languages.
Back in the early 80's, C compilers were as crap as any other system programming languages when generating code for home computers.
- One hell of an awesome encoder/decoder library that can hit more formats than I knew existed.
:)
> There were lots of dynamic allocations
This surprises me; a lot of codecs are specified in such as way as to not require dynamic allocation at all.
I hypothesise that if it was written in another language, we would just see bugs appear in a different form.
Exactly! Programming is hard to get right, no matter what the language. I hate these people who come along saying "hurr, durr, C sucks, project X should have been written in language Y!" All the while ignoring the facts that a) they can go and rewrite X in Y any time they like, and b) if it was written in Y, people would be bitching about issues with Y (eg, speed). Shortsightedness and lack of will to put your code where your mouth is are two of the most despicable traits a programmer can have.
Language has a lot to do with what sort of issues programmers will have to tackle.
Using python as an example here makes little sense, since the constraints of the problem domain make it a particularly poor choice. Try choosing a language that doesn't suffer as much in performance (future Rust, maybe?).
I'm just making a point that language is relevant when discussing the type and frequency of errors that are surfacing in a project.
But the OSs created with them weren't as fancy as UNIX was at the time, or used hardware more costly than the PDP-11.
- NULL pointer dereferences,
- Invalid pointer arithmetic leading to SIGSEGV due to unmapped memory access,
- Out-of-bounds reads and writes to stack, heap and static-based arrays,
- Invalid free() calls,
- Double free() calls over the same pointer,
- Division errors,
- Assertion failures,
- Use of uninitialized memory.
But hey, any good programmer always writes perfect C code.