News for hackers. Stuff that matters.
Re:Kent Beck: “I get paid for code that works, not for tests... by Alex3917 on Thursday December 08, 2016 @09:03AM (#13130282)
> But we don't write tests to check if our code works.

I write tests to check if my code works. And tests that document how the code is supposed to work currently are usually enough to prevent code from breaking in the future.

Anything related to privacy or security should be fully tested. But for the typical startup, I'd posit that beyond that test coverage should be more closely related to the number of users and level of usage rather than to the amount of code.

Re:Kent Beck: “I get paid for code that works, not for tests... by e40 on Thursday December 08, 2016 @10:24AM (#13130828)
The number of times my tests have 1) found bugs in my code, and 2) exposed problems with the API I was about to publish... uncountable.
Re:Kent Beck: “I get paid for code that works, not for tests... by mannykannot on Thursday December 08, 2016 @03:11PM (#13133832)
More generally, testing effort should be distributed according to a risk analysis. Even a one-off statistical report can have serious consequences if far-reaching decisions are made on its basis.
Re:Kent Beck: “I get paid for code that works, not for tests... by aequitas on Thursday December 08, 2016 @02:48PM (#13133566)
> And tests that document how the code is supposed to work...

For me the most important word here is 'supposed'.

All the time I read documentation describe how code works step by step (what each 'if' does, but spelled out more verbose). And test that only test that a function does by mocking out everything else.

But I don't care reading what code does. I can see that by looking at the code. I want to know what the developer intended/expected the code to do, so I can validate it against what the code actually does. Most of the time assumptions are made with those expectations. And with those expectations you have a much better idea why a trivial refactor of a piece of logic could unearth a massive 'undocumented feature'.

Re:Kent Beck: “I get paid for code that works, not for tests... by hannob on Thursday December 08, 2016 @10:32AM (#13130876)
> Anything related to privacy or security should be fully tested.

How often do you write code that's not related to privacy or security? As soon as you connect something to the Internet it's related to privacy and security.

The only situation where privacy and security don't matter a whole lot is if your code runs on airgapped devices with very limited tasks.

Re:Kent Beck: “I get paid for code that works, not for tests... by EpicEng on Thursday December 08, 2016 @10:38AM (#13130908)
I rarely write code that connects to anything, so almost never? There's a lot of code that's not written by web devs.
Re:Kent Beck: “I get paid for code that works, not for tests... by hannob on Thursday December 08, 2016 @10:52AM (#13131040)
> I rarely write code that connects to anything, so almost never?

Sounds hard to believe. What kind of code would that be?

> There's a lot of code that's not written by web devs.

There's also a lot more than the web that has some form of connectivity with the Internet (even if it's not directly connected it may still parse data that comes from untrusted sources).

There is a widespread belief among many that "security is important, but doesn't matter for me". The most extreme example is obviously IoT ("Who would want to hack my coffeemaker?"), but there's a lot more. The unfortunate truth is: There is hardly any code these days that is not security relevant.

Re:Kent Beck: “I get paid for code that works, not for tests... by EpicEng on Thursday December 08, 2016 @12:03PM (#13131838)
I write code that interfaces with hardware (health care imaging systems), processes images, and a little computer vision here and there. Very few service calls in that stack.

I jump up 1000 levels of abstraction from time to time, and when I do, I agree that security is extremely important (FDA class III device and HIPAA compliance is mandatory.) I'm also a lead, so I have to know enough to call BS when I hear it from a team member.

Re:Kent Beck: “I get paid for code that works, not for tests... by ryandrake on Thursday December 08, 2016 @11:25AM (#13131401)
> > I rarely write code that connects to anything, so almost never?

> Sounds hard to believe. What kind of code would that be?

Device drivers, compilers, and some embedded systems come to mind immediately, there are plenty of others out there. I've worked on a lots of software where the only inputs were physical and sensor based, and the only outputs were to the screen. Device didn't even physically have network equipment.

Re:Kent Beck: “I get paid for code that works, not for tests... by RandomOpinion on Thursday December 08, 2016 @11:42AM (#13131581)
>...device drivers...

I hope you're joking but I suspect you're not. Device drivers have the highest level of privileged access in many operating systems and code quality for drivers is so uniformly lousy (certain large vendors whose names begin with "N", "A", and particularly "Q", I'm looking at you) that attempting to break the drivers would be among the first things I'd consider if I were trying to root a device.

Re:Kent Beck: “I get paid for code that works, not for tests... by ryandrake on Thursday December 08, 2016 @12:08PM (#13131889)
Please see my other reply. I was addressing "unconnected software" not "software without attack surfaces". I agree that security is a concern with all software.
Re:Kent Beck: “I get paid for code that works, not for tests... by boomlinde on Saturday December 10, 2016 @01:37PM (#13147099)
Well, in a sense, a device driver can usually be thought of as very connected to a highly sensitive part of a computer system. If something goes wrong in a driver it will have a negative effect on the whole (monolithic) kernel and the rest of the operating system.
Re:Kent Beck: “I get paid for code that works, not for tests... by hannob on Thursday December 08, 2016 @11:31AM (#13131454)
Device drivers are a bad example, they can be extremely security sensitive (particularly so because they usually have kernel privileges).

There are few things where security doesn't matter. But they are extremely rare. The situations where programmers think their code isn't security sensitive are probably vastly more common.

Re:Kent Beck: “I get paid for code that works, not for tests... by ryandrake on Thursday December 08, 2016 @11:46AM (#13131629)
Good point, but I was addressing the perceived lack of "code that connects to anything". I agree with you that security is a valid concern for most if not all code written.
Re:Kent Beck: “I get paid for code that works, not for tests... by nickpsecurity on Thursday December 08, 2016 @11:22AM (#13131364)
"Sounds hard to believe. What kind of code would that be?"

You do realize there's a bajillion non-networked apps in existence? Word processors, excel, editors/IDE's, system tools (esp monitoring/backup), media players that don't download stuff, compression libraries, MATLAB-style tools for numeric analysis, and so on.

Re:Kent Beck: “I get paid for code that works, not for tests... by hannob on Thursday December 08, 2016 @11:28AM (#13131432)
> You do realize there's a bajillion non-networked apps in existence? Word processors, excel, editors/IDE's, system tools (esp monitoring/backup), media players that don't download stuff, compression libraries, MATLAB-style tools for numeric analysis, and so on.

All of them parse potentially untrusted inputs. They don't have to be directly network connected to be a security risk.

Just pick the first example: A word processor. It is not a security risk only if you can guarantee that you'll only ever open documents that you created yourself. If you ever use it to open documents you got from someone else it needs to take security into consideration.

Re:Kent Beck: “I get paid for code that works, not for tests... by mwexler on Thursday December 08, 2016 @12:14PM (#13131950)
Sadly, almost everything you mention involve opening files, and files are often on networks. Standard modal file|open on windows allows http:// access to files on hosted servers, for example. Suddenly, all of these are potentially networked in the hands of users using the code for unintended purposes or in unexpected ways.
Re:Kent Beck: “I get paid for code that works, not for tests... by Jtsummers on Thursday December 08, 2016 @12:25PM (#13132072)
But you don't write those dialogs yourself. You write code that calls the system file open dialog. Then, presumably, you do some check on the file for sanity checking (is it the right type? does it parse correctly? does the internal size # equal the value the OS is claiming?). If those checks fail, you report an error. Otherwise, you process it. The source doesn't matter at that point. And if there's a glitch in the dialog for handling HTTP or similar sources, then that's on the system developers to correct, and on you to report if you discover it.
Re:Kent Beck: “I get paid for code that works, not for tests... by nickpsecurity on Thursday December 08, 2016 @02:11PM (#13133137)
Exactly. All the concerns of networking, the Internet, and the Web go away. It becomes an input validation problem. From there, one can do a format suitable for correct-by-construction auto-generation of the parser. Or can take the common route building a Turing machine into it to fight with over time. ;)

Still not writing or patching a networked app.

Re:Kent Beck: “I get paid for code that works, not for tests... by Jtsummers on Thursday December 08, 2016 @02:21PM (#13133247)
OT: Have you ever read Engineering a Safer World [0] by Nancy Leveson [1]? Seems like something that'd be of interest to you. Started on it this week, my sister is in her class this semester at MIT. I'll be tackling it over the next few weeks, and then her previous text (Safeware) after the new year.

[0] https://mitpress.mit.edu/books/engineering-safer-world

[1] http://sunnyday.mit.edu/

Re:Kent Beck: “I get paid for code that works, not for tests... by nickpsecurity on Thursday December 08, 2016 @09:19PM (#13136175)
Wow. Been studying high-assurance systems close to 10 years and just hearing about her. That's how scattered that field can be. Let's look at this.

"Started new area of research: software safety." That was Bob Barton in Burroughs B5000, Dijkstra on THE, and Margaret Hamilton on Apollo code. Maybe they mean first dept at MIT or just making status of sub-field more official. Then TCAS II. I recall reading that long ago as an exemplary work in formal specification & safety analysis but project was too heavy for me. Article says them too haha. Props to her for it & others. Article shares my view on scattered groups & methods. At least seen STAMP referenced once but unfamiliar with it. They wrote against N-version programming being re-invented... which I proposed for subversion resistance. Hmmm. I'm sure my variant is the one that works this time. ;) Also did SpecTRM at their company that looks a lot like state machine and modeling schemes I saw elsewhere in high-assurance. Not claiming a copy rather than inspiration or independent invention + convergence of multiple parties. Usually means a good idea.

Very interesting person. Thanks for the tip. Your sister is going to learn some wise things for sure given they've got sane methods and got results before. I especially liked how the article jokes about writing what she knew on high-assurance development then gotten wiser or more confused. I know the feeling where I'm redoing the foundations now with what a decade taught me. More slowly this time given I have more doubt than certainty.

Note: Just got to the last part. Wait, she was the one who wrote the THERAC paper? I just assumed it was some guy (male-dominated field) named Levenson since that name was all I saw in references to the report. Never saw it again. So, she wrote up an investigation we've been citing about software safety for decades, helped spearhead efforts to legitimize it as a field, did huge projects, and I basically never hear about her. Unreal. I'm bookmarking her stuff to go through it later.

Re:Kent Beck: “I get paid for code that works, not for tests... by Alex3917 on Thursday December 08, 2016 @10:56AM (#13131091)
Right, I meant that in the context of things that tests are designed to catch. E.g. almost no one writes tests to look for unsafe eval or whatever.

A good example is that I care a lot about making sure our search endpoint doesn't return private user data. But beyond that, I'd rather just know that the endpoint returns a 200 and let someone tell us if it's broken rather than have an extra three hundred lines of code to see if it's returning the correct results. If we get a ton of users then that will probably change, but for now the cost of writing and maintaining those extra tests wouldn't be worth the benefit.

Re:Kent Beck: “I get paid for code that works, not for tests... by mannykannot on Thursday December 08, 2016 @02:47PM (#13133560)
> The only situation where privacy and security don't matter a whole lot is if your code runs on airgapped devices with very limited tasks.

That used to be true for the software in cars, but it no longer is. The problems that result are not the fault of the original authors; that belongs to the people who decided to bridge the airgap without thinking through the consequences.

Re:Kent Beck: “I get paid for code that works, not for tests... by tlrobinson on Thursday December 08, 2016 @10:54AM (#13131070)
Most client-side web app code doesn't have much bearing on privacy/security, aside from avoiding XSS and a few other pitfalls.