News for hackers. Stuff that matters.
Re:45 years since its creation. The C language still very po... by TickleSteve on Tuesday May 24, 2016 @11:59AM (#11762612)
Bare metal C code is what is running on 99.9% of the embedded devices surrounding you right now...
Re:45 years since its creation. The C language still very po... by vardump on Tuesday May 24, 2016 @12:35PM (#11762972)
Scary thought. I'd feel more at ease if it was Rust (or something analogous).

Just disassemble/decompile some random firmware code... you wouldn't believe what one can find from those piles of err... binary dump.

Re:45 years since its creation. The C language still very po... by TickleSteve on Tuesday May 24, 2016 @12:53PM (#11763173)
There is a lot of scaremongery around C.... don't believe it.

If you know what you're doing (and you should) then C is the best tool for the job.

Re:45 years since its creation. The C language still very po... by nickpsecurity on Tuesday May 24, 2016 @01:49PM (#11763814)
The military and defense contractors did empirical studies of various programming languages and their defect rates to put that to the test. That was mainly in 80's and 90's. They compared C, C++, Ada, and Fortran mainly. C usually had double the defects of the rest and with more severity despite pro's writing it. Ada usually outdid all of them with one showing Ada and C++ developers having same defect rate.

So, when we use evidence instead of feelings, what you said is a myth that's been debunked repeatedly in many ways for decades. And, yet, people repeat it. No, C use usually results in problems that safer, systems languages before and after it had less or none of. Despite professionals using it. The solid code is always an outlier.

Most amusing thing is Thompson and Ritchie, with help from Pike, later designed what they thought was perfect language with perfect set of features. The result, Go, was basically like Algol subsets that preceded C mixed with Pascal developed around same time. But, C compiled and ran on a PDP-11 fast. So, it's the best. ;)

Re:45 years since its creation. The C language still very po... by TickleSteve on Tuesday May 24, 2016 @01:58PM (#11763927)
Yes I remember the studies (and have done my fair share of defence s/w).

And I am not claiming that C is the best tool for the job because of the language.... its because of the 'environmental' factors, human resources, community, momentum, codebases, portability, etc.

I whole-heartedly agree that there are better languages out there, but the fact remains that the flaws in C are easily managed.

It would also be interesting to produce a study of the use of C specifically in embedded systems where the style of code is very much different than large desktop systems (i.e. much less heap usage and dynamic memory allocation for example).

Re:45 years since its creation. The C language still very po... by nickpsecurity on Tuesday May 24, 2016 @02:02PM (#11763989)
"And I am not claiming that C is the best tool for the job because of the language.... its because of the 'environmental' factors, human resources, community, momentum, codebases, portability, etc."

I agreed with you on that in another comment. Yep. It's also why NASA often uses it. The tooling covers dark corners pretty well these days.

"It would also be interesting to produce a study of the use of C specifically in embedded systems where the style of code is very much different than large desktop systems (i.e. much less heap usage and dynamic memory allocation for example)."

I'd love to replicate the old studies on modern languages with modern tooling for both systems programming and the embedded subsets like MISRA. We'd definitely learn some stuff from that. Further, I'd like to see specific metrics like in Ada/SPARK/CbyC example below on where defects were introduced or corrected with each technique and phase of lifecycle. Would tell us how low-level features, subset rules, and tooling interact with accurate assessment of how much problems they pose for real instead of in theory.

http://www.sis.pitt.edu/jjoshi/Devsec/CorrectnessByConstruct...

Re:45 years since its creation. The C language still very po... by TickleSteve on Tuesday May 24, 2016 @02:19PM (#11764151)
This is the thing...

Modern embedded C programming has evolved significantly in the right direction since studies were done and so I would love to see someone spend the time looking at 'modern' embedded C codebases.

Static analysis is now widespread and not theoretical, coding standards now take security seriously (tho can be outdated). Drawbacks to certain techniques have been surfaced and recognised. Tooling is much better, portability is recognised as a good-thing, pointer usage is now minimised when possible and contained to areas where they are appropriate. Casting is frowned upon, macro-magic is frowned upon. 'Clever' code is frowned upon.

Basically, I would think that certainly in the embedded domain, standards are now such that the story would now be significantly different.

Re:45 years since its creation. The C language still very po... by vardump on Tuesday May 24, 2016 @02:52PM (#11764457)
Any recommendations on static C analysis for embedded systems?
Re:45 years since its creation. The C language still very po... by TickleSteve on Tuesday May 24, 2016 @04:35PM (#11765244)
No one-tool does the whole job. Setup your CI system to use multiple tools (Lint, CLang-analyser, cppcheck, etc).

Dont trust just one.

Re:45 years since its creation. The C language still very po... by nickpsecurity on Wednesday May 25, 2016 @12:00PM (#11770726)
Have you tried Astree Analyzer? Papers I read indicated it was one of best but hard to find industry people to confirm or reject it. Want someone to clone it for FOSS. Meanwhile, I found these free ones for you two that have each found bugs in code with focus on minimal annotations. All are academic prototypes but Saturn was used on Linux kernel.

Saturn http://saturn.stanford.edu/pages/overviewindex.html

CSolve http://goto.ucsd.edu/csolve/

Calysto http://www.domagoj-babic.com/index.php/ResearchProjects/Caly...

Most academics are instead making compilers that translate C code into something mostly or totally safe while pushing the performance hit downward. Here's two of the top for you to try on various codes (or improve):

https://www.cs.rutgers.edu/~santosh.nagarakatte/softbound/

http://sva.cs.illinois.edu/downloads.html

Re:45 years since its creation. The C language still very po... by vardump on Tuesday May 24, 2016 @04:52PM (#11765366)
Pretty much what we already use.

I was hoping for something new.

Re:45 years since its creation. The C language still very po... by nickpsecurity on Wednesday May 25, 2016 @12:00PM (#11770734)
Here you go. Experiment away and give them constructive feedback as they need it. Esp Softbound and SAFECODE.

https://news.ycombinator.com/item?id=11770726

Re:45 years since its creation. The C language still very po... by vardump on Wednesday May 25, 2016 @02:48PM (#11772124)
Thanks!

I'll look into those.

Re:45 years since its creation. The C language still very po... by Profan on Tuesday May 24, 2016 @01:07PM (#11763322)
Best tool is a stretch, it's a tool.

If we can get tools like Rust which have better memory safety guarantees while offering much of the same performance, then hell, why not? It's easy to see why just by looking at the hundreds of CVE's throughout the years, often caused by memory unsafe operations.

Not that I don't like C, but there's many better alternatives out there.

And this is coming from someone who stubbornly sticks to writing D in C-style, but knows what Rust brings to the table.

Re:45 years since its creation. The C language still very po... by TickleSteve on Tuesday May 24, 2016 @01:40PM (#11763694)
OK, I'm biased as I've got over 20 years in embedded/real-time systems primarily written in C (and C++) and am currently architect for an embedded system containing ~million lines of code and rolling out to tens of millions of units.... (But I'm educated enough to also use Python, Clojure, C# and a variety of other systems).

...and C is currently the best choice for systems like this because of reasons not primarily to do with the language.

Currently, if you suggest to use a language other than C you will be laughed out because the only available embedded guys are C based. Yes, momentum counts and C has massive momentum.

There are also issues such as toolchains and tooling, familiarity, community, and of course the massive existing set of libraries, codebases and knowledge.

Basically, C is 'good enough'. (although Rust is interesting and on the horizon, its a long way off yet).

Although technically a better language, C's shortcomings are greatly exaggerated, for example memory management. In general this is not an issue in bare metal embedded systems as you don't have a heap, everything is statically allocated by design. There are many flaws in the C language (as in every language), but in day-to-day use, they are very easy to manage.

So... don't believe the (bad) hype.

Re:45 years since its creation. The C language still very po... by pcwalton on Tuesday May 24, 2016 @02:13PM (#11764092)
I agree with you that C is a lot better if you don't have a heap. But dangling pointers, undefined behavior, etc. are still issues. We know how to fix these problems in 2016 with better language design, via techniques we didn't know in 1978.

I'll be the first to admit that compatibility with an ecosystem, even if flawed, is important. (I work on web browsers, after all!) We can't change overnight, or even in two or three years. But we'll never get to a better future if we don't take the steps to start now.

Re:45 years since its creation. The C language still very po... by nickpsecurity on Tuesday May 24, 2016 @01:53PM (#11763863)
"There are also issues such as toolchains and tooling, familiarity, community, and of course the massive existing set of libraries, codebases and knowledge."

Social and economic factors that we C opponents say are why it remains and is often the pragmatic choice. These in no way show C language itself was well-designed, superior, etc. Just show how going mainstream can make things more practical. They did a great job on that part.

Re:45 years since its creation. The C language still very po... by TickleSteve on Tuesday May 24, 2016 @02:05PM (#11764023)
Completely agree.

TBH, my ideal language for this job would probably be Rust, but that is just not a mature enough ecosystem currently.

Don't get me wrong, I would consider myself a C proponent currently, but I would love to see change in a Rust-like direction.

Despite you classing yourself as a C-opponent, I suspect we agree more than disagree.

Re:45 years since its creation. The C language still very po... by nickpsecurity on Tuesday May 24, 2016 @02:13PM (#11764090)
Probably. I think you're a pragmatist rather than a true believer. ;)

Btw, what do you think of a reboot of Modula-2, Modula-3, or just Ada/SPARK? Your opinion on them would be interesting given your background. Closest thing to Modula's that's actively developed is Astrobe Oberon for embedded.

https://en.wikipedia.org/wiki/Modula-3

http://www.astrobe.com/Oberon.htm

Re:45 years since its creation. The C language still very po... by TickleSteve on Tuesday May 24, 2016 @02:53PM (#11764464)
(Certainly a pragmatist)

I started off with Turbo Pascal before graduating to Turbo C, so I do have a fondness for the Pascal family of languages. I also love type-safety.... TBH, I would prefer a hypothetical "fix" for the C language that improved type strictness/safety over "fixes" for resource/memory management issues which I consider should be bread and butter for any software engineer. So in that, I do not like large runtimes or garbage collection, more visibility to what the machine is really doing is needed. GC systems are also not predictable enough, deterministic runtime is absolutely essential to making reliable software.

I have poked my nose around Modula-2 in the past, but C took over for me personally. ADA I consider to be far to heavyweight for anything serious (and thats after being in a company that used it extensively in the defence world, tho I never had to touch it myself)

Currently Rust is my best hope for the future.... but C has survived well IMO.

Re:45 years since its creation. The C language still very po... by pjmlp on Wednesday May 25, 2016 @01:46AM (#11767881)
Also a pragmatist, as although I tend to bash on C on every occasion and would rather not use it, if a customer does require it, I will use it.

For me a kind of escape path from C, back when I was into Turbo Pascal, was to move to Turbo C++. I only got to use Turbo C around one year before getting to learn about this new cool language called C++.

It allowed me to keep some of the Turbo Pascal safety around.

What are your approaches when using plain C, coming from "so I do have a fondness for the Pascal family of languages. I also love type-safety" ?

My solutions were:

- Use translation units as TP units, only exposing functions and struct accessors (macros if function calls were too expensive)

- Always make use of debug functions to validate pointers in debug builds (e.g. _malloc_dbg, _CrtIsValidPointer on Windows)

- make use of const as much as possible

- write my own wrappers around strncpy and friends

- compile warnings as errors

- if given the option, just use C++ instead and prefer library types to inherited C ones (string, array, references, RAII).

Re:45 years since its creation. The C language still very po... by nickpsecurity on Tuesday May 24, 2016 @03:06PM (#11764580)
Interesting response. Thanks.
Re:45 years since its creation. The C language still very po... by pcwalton on Tuesday May 24, 2016 @01:34PM (#11763634)
> If you know what you're doing (and you should) then C is the best tool for the job.

Empirically, nobody knows what they're doing, if "knowing what you're doing" is defined as "writing large-scale C code without memory safety problems".

I hear a lot from C and C++ enthusiasts that there are lots of programmers out there who always write correct C and C++ code, and therefore we don't need new languages. But I've never found one of those programmers. Can you name one?

Re:45 years since its creation. The C language still very po... by VLM on Tuesday May 24, 2016 @02:26PM (#11764215)
"large-scale C code"

I hope to never own a microwave oven with a large scale codebase. Also my basement dehumidifier. And my waffle iron. And my clothes washer. And my digital dial caliper in my workroom.

There's a surprising amount of money in "you are now a timer" and "you are now a thermostat" and "you are now a thermometer" and very close analogies.

Re:45 years since its creation. The C language still very po... by vardump on Tuesday May 24, 2016 @02:33PM (#11764275)
Washing machines actually have quite a bit of code. Car ECUs have even more. Elevator control systems are pretty big. Medical devices have huge codebases.

You'll definitely use or own something that has a somewhat large codebase.

Re:45 years since its creation. The C language still very po... by nickpsecurity on Wednesday May 25, 2016 @12:31AM (#11767669)
Mine doesn't. When my washer broke, I called some locals that have a warehouse full of them and repair them. I asked whether it was their opinion, being experts, that my observation about the older ones being more reliable was true? The guy confirmed that older ones easily last a decade or so with new, computer-filled ones breaking all the time. Said they're good business for him. Sold me what he claimed was a good model of older ones he had for under $100. I was surprised when he told me the one that broke was around 20 years old.

That is both reliability and return on investment. :)

Re:45 years since its creation. The C language still very po... by imtringued on Wednesday May 25, 2016 @06:26AM (#11768783)
That is mostly because the newer washing machines replace what used to be steel with plastic to cut costs.
Re:45 years since its creation. The C language still very po... by nickpsecurity on Wednesday May 25, 2016 @11:46AM (#11770570)
I'd believe it. He said the electronics screw up, too. He was fixing a control system on one that was just a few circuits in the knob on older ones.
Re:45 years since its creation. The C language still very po... by TickleSteve on Tuesday May 24, 2016 @01:51PM (#11763833)
(see the reply to the above response).

Basically, in day-to-day use, the flaws in the language are easily manageable. Resource (memory) management is a skill that should be a given.

Yes, there are well publicised failures in this regard, but consider the number of problems versus the actual number of lines of code written.

Re:45 years since its creation. The C language still very po... by pcwalton on Tuesday May 24, 2016 @02:09PM (#11764062)
> Yes, there are well publicised failures in this regard, but consider the number of problems versus the actual number of lines of code written.

I think that's a pretty misleading metric when you take into account: (1) widely used codebases constantly increase in size constantly; (2) it only takes one exploitable mistake for an attacker to achieve remote code execution. One exploitable vulnerability per 10,000 lines means 1,200 remote code execution flaws in a codebase the size of Chromium.

Re:45 years since its creation. The C language still very po... by jjnoakes on Tuesday May 24, 2016 @02:23PM (#11764191)
No one "always writes correct large-scale code".

I have worked with many who write large-scale C and C++ code with no undefined behavior. Is that what you meant?

Re:45 years since its creation. The C language still very po... by pcwalton on Tuesday May 24, 2016 @02:58PM (#11764509)
Yes.

I've never seen it. All large-scale C and C++ codebases I've ever seen, from large companies to small ones, have had memory safety/undefined behavior problems [1]. If "just hire better programmers" were a workable solution, surely one company out of {Google, Apple, Microsoft, Facebook} would have succeeded at that strategy by now.

It's very easy to write C and C++ code that looks like it's free of undefined behavior, but in every case I've seen they end up falling when attackers actively try to look for problems.

[1]: Maybe qmail is the one exception, though even that had a famous debate related to overflow.

Re:45 years since its creation. The C language still very po... by jjnoakes on Tuesday May 24, 2016 @03:26PM (#11764738)
Have you ever worked with teams who use aggressive static analysis tools to detect and catch undefined behavior?

Because I have, and it works incredibly well.

Of course, the caveat is that once you turn up the static analysis aggressiveness (assuming you use a good static analysis tool), you will need to put in plenty of assert(index < len), ownership annotations, and other such items in your code to satisfy the checks.

But once you do, it's really hard (probably not impossible, but really hard) to trigger undefined behavior without the static analysis tool catching it.

Because if the tool decides it can't prove there's no undefined behavior, it is configured to complain, and you adjust the code until it stops.

I'm not saying many teams do this, but I am disputing that no teams do this.

Re:45 years since its creation. The C language still very po... by pjmlp on Wednesday May 25, 2016 @01:52AM (#11767901)
> I'm not saying many teams do this, but I am disputing that no teams do this.

If you watch Herb Sutter's talk at CppCon 2015, at a given point he asks the audience how many know and use such tools.

It is one of the most important C++ conferences, usually attended by the most savvy C++ developers in the world, the amount of the audience saying that they do use such tools was ruffly 1%!

1% shows how much the majority of C and C++ developers, or their employers, care about writing proper safe code in those languages.

The only way out is moving to programming languages whose safety must be explicitly turned off, and not having to be explicitly turned on, because most won't bother taking the effort to turning it on.

Re:45 years since its creation. The C language still very po... by jjnoakes on Wednesday May 25, 2016 @03:01PM (#11772221)
No one is arguing that the industry moving to a safer language will reduce the level of nasty bugs in the world. That would be silly to argue against, because that's basically a tautology anyway.

What I'm disputing is that no teams write good C or C++ code.

In fact, the 1% who raised their hands proves my point. It isn't 0%.

That's all I'm saying. Don't smear our names with the "all C and C++ developers write code with exploitable undefined behavior bugs in it" when you mean "most C and C++ developers write code with exploitable undefined behavior bugs in it".

Re:45 years since its creation. The C language still very po... by pjmlp on Thursday May 26, 2016 @02:19AM (#11775957)
C++ is my language to go, every time I need to go out of JVM and .NET ecosystems and I do take all efforts to be part of that 1%.

Once upon a time I gave C++ class to first year students at the university where I took my degree. Worked with C++ at some well known companies and research institutions.

Still, I won't say I never write exploitable undefined behaviour free code in C++, let alone C. Because I cannot control the code that gets used by third party libraries, written by team members or even having the whole UB use cases on my head.

Re:45 years since its creation. The C language still very po... by jjnoakes on Thursday May 26, 2016 @10:05AM (#11778021)
If you use the right tools (static analysis and sanitizers and fuzzers) your own code (and any code you rely on which you have the source for) will be handled.

If you rely on binary dependencies which have quality problems, then u can't help you there. But that could be an attack vector in any language.

And if you are worried about the operating system itself, then there really is no easy way around it unless you want to run a unikernel for everything.

And even then you may hit a CPU microcode bug it hardware bug...

Re:45 years since its creation. The C language still very po... by pcwalton on Tuesday May 24, 2016 @04:10PM (#11765084)
> Have you ever worked with teams who use aggressive static analysis tools to detect and catch undefined behavior?

Coverity is actually used on browser engines, and it has not been able to stem the tide of exploitable security vulnerabilities. Sound static analysis is just too difficult on idiomatic C and C++: it's effectively impossible, as the language was just not designed for it.

Re:45 years since its creation. The C language still very po... by jjnoakes on Tuesday May 24, 2016 @05:46PM (#11765797)
I don't agree. I work with static analysis daily and idiomatic code is the easiest to analyze.

If coverity isn't catching the bugs and they are undefined behavior bugs then coverity needs to be more aggressive.

But it probably doesn't support things like ownership annotations, and it probably gives up strict checking in order to avoid false positives.

The problem is developers. Developers are lazy. Most developers won't even turn on compiler warnings, never mind using the sanitizers.

So tool makers create sub-par tools to cater to them. Fewer false warnings means more customers, even if it also means fewer security holes.

It's why we use an in house tool and crank up the checks to 11.

Re:45 years since its creation. The C language still very po... by vardump on Tuesday May 24, 2016 @03:39PM (#11764847)
What tools do you recommend?
Re:45 years since its creation. The C language still very po... by jjnoakes on Tuesday May 24, 2016 @05:48PM (#11765806)
Unfortunately I have never used one available commercially which meets my needs. We author one in-house.
Re:45 years since its creation. The C language still very po... by nickpsecurity on Wednesday May 25, 2016 @12:26AM (#11767660)
Your tool is better than Astree Analyzer and Polyspace? I think you should consider GPLing or licensing it with commercial support. Judging from their prices & effectiveness, you'll make a killing doing better while undercutting their licensing.
Re:45 years since its creation. The C language still very po... by jjnoakes on Wednesday May 25, 2016 @12:44PM (#11771121)
The problem is usability and licensing costs. Our in-house tool is amazingly powerful and fits our needs but the usability would need improvement (no surprise) and we license some of the utility code we use (which is cheaper if we don't resell our tool).
Re:45 years since its creation. The C language still very po... by nickpsecurity on Wednesday May 25, 2016 @01:34PM (#11771547)
You couldn't replace the utility code with something in FOSS? And that's why your company is holding out on a static analyzer better than anything out there? The case for doing something to open that tool is just getting stronger.

Btw, what utility tool are you licensing? What makes it irreplaceable? I know of only one in this field that I truly couldn't replace for compilers or static analysis. Even then, there's quite a few that handle the job well enough to not need it. So, I don't use it.

Re:45 years since its creation. The C language still very po... by jjnoakes on Wednesday May 25, 2016 @01:55PM (#11771708)
The utility tool has OSS equivalents but we've evaluated them all. None of them have the features we need, and adding them would be person-years of work we can't afford.

My hope is to some day open source our internal tool, but we have to wait until the OSS features we need catch up. Until then, we aren't willing to pay the high license fee to redistribute this utility freely.

I'm speaking a little cryptically because I'd rather not have this tied directly to any particular company (neither mine nor the one we license this code from). I'm not speaking on their behalves officially; only my own.

Re:45 years since its creation. The C language still very po... by nickpsecurity on Thursday May 26, 2016 @12:21AM (#11775391)
Hmm. Love to know what that tool is but NDA's and policies are what they are. I have an email if you want to send it to me for what advantages it has over similar comparison. Then, on odd chance I see opportunity, I'll nudge someone in direction of trying to bring OSS up to par. No promises it will happen as opportunity has to happen first.