News for hackers. Stuff that matters.
Re:Evidence-based Software Engineering: based on the publicl... by dang on Monday May 24, 2021 @08:00PM (#27271145)
Related previous threads:

Evidence-based software engineering: book released - https://news.ycombinator.com/item?id=25073218 - Nov 2020 (118 comments)

Evidence-based Software Engineering – book beta - https://news.ycombinator.com/item?id=23769930 - July 2020 (22 comments)

Evidence-based Software Engineering [pdf] - https://news.ycombinator.com/item?id=3213482 - Nov 2011 (9 comments)

Evidence based software development - https://news.ycombinator.com/item?id=2707618 - June 2011 (1 comment)

What we actually know about software development, and why we believe it’s true - https://news.ycombinator.com/item?id=901600 - Oct 2009 (54 comments)

I'm pretty sure there were others in years past... maybe someone will find one.

Re:Evidence-based Software Engineering: based on the publicl... by juskrey on Monday May 24, 2021 @02:16PM (#27267809)
Oh well, all of my 20 years of software engineering are a tale of fighting in the dark before anything works and heavy precaution after, without having any remote evidence of nearly everything readily available.
Re:Evidence-based Software Engineering: based on the publicl... by moksly on Monday May 24, 2021 @06:27PM (#27270422)
That’s the most interesting find of the book isn’t it?

We’ve taken a sort of controversial approach to software development in the public sector of Denmark because we don’t have enough resources. Small projects are build to run their own little lives with as little post-deployment development as possible. Because of this they aren’t build with best practices, whatever theories are the right way to do things in a current age, none of them are automatically tested because their functionally is so small it’s always going to be obvious where the flaws are and the only way for them to get additional development post deployment is if they turn out to be really good, and even then, we don’t expect them to live after 5-10 years.

It’s way of development goes against every theory on software development you’ll find being taught at universities and academies, I know, because I’m an external CS examiner at these places, and we’ve really only chosen to do this because we have 3 developers in an organisation with 10.000 employees.

Now the data shows us, that we’re actually better off doing the wrong thing.

Re:Evidence-based Software Engineering: based on the publicl... by whatshisface on Monday May 24, 2021 @07:38PM (#27270995)
Small programs are by far the best approach. The only time that doesn't work is when the problem itself requires a very large program. That is when you need things like automated tests.
Re:Evidence-based Software Engineering: based on the publicl... by InfiniteRand on Monday May 24, 2021 @10:36PM (#27272057)
In theory the obvious solution would be to make everything modular small programs but then you pay the price of coordination (ie, apis between modules, dependency, maintaining uniform practices etc.) so... shrug
Re:Evidence-based Software Engineering: based on the publicl... by 908B64B197 on Monday May 24, 2021 @08:34PM (#27271399)
> because their functionally is so small it's always going to be obvious where the flaws

That, sadly, won't scale.

> we're actually better off doing the wrong thing.

You aren't actually doing the wrong thing. At 3 employees the maximum complexity your team can handle is pretty low. It's also trivial to pass on institutional knowledge since well... the whole team can share one extra large pizza.

Re:Evidence-based Software Engineering: based on the publicl... by moksly on Tuesday May 25, 2021 @02:34AM (#27273537)
> That, sadly, won't scale.

We never really need it to. That’s one of the few benefits of writing software in the public sector. We don’t need to expect an explosive growth in users.

We’ve seen 10.000 additional citizens move to our area over 5 years, so there is growth, but it’s far from everyone of them who would interact with a single solution that we wouldn’t buy from the private sector through public procurement.

Re:Evidence-based Software Engineering: based on the publicl... by 908B64B197 on Tuesday May 25, 2021 @01:21PM (#27279605)
I meant feature-wise. My message was unclear.

There's a point where a team of three maintaining tribal knowledge is not enough to deliver certain project above a threshold of complexity. But you mention procurement, so I assume your team does a buy vs build pretty much systematically.

Re:Evidence-based Software Engineering: based on the publicl... by dmos62 on Tuesday May 25, 2021 @04:37AM (#27274417)
>> because their functionally is so small it's always going to be obvious where the flaws

> That, sadly, won't scale.

You're contradicting the UNIX philosophy: make programs so small that they're inherently simple (or so simple that they're necesserily small). I think that's what the other commentor meant and that does scale.

Re:Evidence-based Software Engineering: based on the publicl... by tempodox on Tuesday May 25, 2021 @05:44AM (#27274835)
It depends. Systems can display so-called emergent properties, i.e. the system's behavior is “more than just the sum of its parts”. Components that are simple in isolation can still form complex interactions. When that happens, you'll need to use the bigger guns.
Re:Evidence-based Software Engineering: based on the publicl... by m1sta_ on Monday May 24, 2021 @11:27PM (#27272302)
Heavy development processes don't scale when it comes to volume of projects.
Re:Evidence-based Software Engineering: based on the publicl... by AnimalMuppet on Monday May 24, 2021 @04:22PM (#27269247)
I love the idea, and the honest assessment of the quality of the available data. But good grief, cut to the chase already! This reads like they felt they had to give a thorough discussion to every possibly relevant tangential idea.

I recommend skimming rather than straight-up reading.

Re:Evidence-based Software Engineering: based on the publicl... by whatshisface on Monday May 24, 2021 @07:06PM (#27270738)
They had an entire chapter on general statistical knowledge! I wouldn't mind an abridged version that assumed all prerequisite general math knowledge and covered only the topic material.
Re:Evidence-based Software Engineering: based on the publicl... by denton-scratch on Tuesday May 25, 2021 @09:31AM (#27276539)
I've been skimming forward. There are three chapters on general statistical knowledge - maybe 1/3 of the book.

There are chapters on Probability, Statistics, and Regression. None of those chapters illuminates the subject at hand; they provide an explanation of the author's techniques. It's background. There are several other "background" chapters, that aren't about statistics, but also aren't about the subject at hand.

So it looks as if I read most of the meat yesterday.

I read most of the chapter on Regression, because I didn't know much about it, and the author relies rather heavily on regression analysis.

Re:Evidence-based Software Engineering: based on the publicl... by derefr on Tuesday May 25, 2021 @02:11PM (#27280226)
I'd love to read a book that teaches concepts "in anger" — where you're given a big problem statement with lots of subproblems; and then, in each chapter, the author:

• introduces a subproblem, that would be solvable but for one tool you're currently missing;

• introduces the tool you'll need to solve the subproblem;

• explains the tool in the process of using the tool to solve the subproblem. Only explaining as much of the tool as is necessary to solve the subproblem — more can be explained in later subproblems.

Sadly, I find that most books like this first introduce all the tools you'll need; teach them all thoroughly (including teaching stuff you won't need at all, because it's part of the explanation as traditionally taught); and then go about solving all of the subproblems using all of the tools, in a pretty random order. So the first half ends up boring, and the second half ends up messy and hard-to-follow.

Re:Evidence-based Software Engineering: based on the publicl... by neves on Monday May 24, 2021 @05:37PM (#27270018)
This well written book from O'Reilly covers the same subject: https://www.oreilly.com/library/view/making-software/9780596...

It is from 2010, I don't know if their is really new things on the subject

Re:Evidence-based Software Engineering: based on the publicl... by jdlshore on Monday May 24, 2021 @11:55PM (#27272443)
I’m not so sure about that book. It includes Steve McConnell’s claims about 10x productivity, which Laurent Bossavit thouroughly and convincingly debunked in his book, The Leprechauns of Software Engineering.

A bunch of citations isn’t the same as evidence. When I dig into software studies, I find all kinds of issues, ranging from subjects of convenience (the professor used their undergrad class for their study) to industry studies that don’t define their terms. (e.g., MS did a large empirical study of TDD, but never defined how TDD was practiced—this is relevant given the misunderstandings of TDD that exist, particularly at Microsoft).

I didn’t read the rest of the book—just looked at the blurb, winced at the inclusion of McConnell repeating his 10x claim, and read his chapter—but my default stance is to be highly skeptical of academic software engineering, and even more skeptical of popularizations.

Edit: I’ve just glanced through OP’s book, and it looks like an impressive piece of scholarship. I remain skeptical of academia’s ability to teach much of interest regarding software engineering, though. (They spend a lot of words on Waterfall and traditional project management. Their words on the Agile-kinda stuff companies actually use today? One paragraph, paraphrased: “there is no data, we have nothing to say.” Not the authors’ fault, but sums up the irrelevance of academia to real-world SE, in my mind.)

Re:Evidence-based Software Engineering: based on the publicl... by feoren on Wednesday May 26, 2021 @12:47PM (#27292860)
I'm not sure how you can "debunk" something that so many people have directly observed, but our ability to measure a software engineer's productivity is so awful that I'm willing to consider the possibility that we're all just wrong. Are 10x engineers just good at appearing productive then?
Re:Evidence-based Software Engineering: based on the publicl... by jdlshore on Wednesday May 26, 2021 @06:45PM (#27297089)
He debunks McConnell's claims, which are specifically about 10x research, not the 10x claim itself.

But I'll happily go down that road. The 10x claim ("the most productive engineers are 10x more productive than the least productive") is ridiculous on its face; the least productive are zero or negatively productive. There are people who can't program at all, and there are people who actively detrimental. Therefore, two things are clear:

1. The correct statement is "the most productive engineers are infinitely more productive than the least productive."

2. This is useless information.

A more useful statement would be "the most productive engineers are x% more productive than the median." Or even a histogram—that would be super useful!

But, it turns out, there is no way to measure real-world engineer productivity, so any quantitative claims are, in fact, utter bullshit.

And, if you look at the original research that McConnell cites, that turns out to be true. IIRC, the study was comparing the how long it took to debug programs when using a time-share system vs. a batch-processing system.

(To be clear: I'm arguing about the claim, not the variations in productivity. Obviously, people have variations in productivity. But there is no rigorous evidence for the magnitude or nature of those variations.)

Re:Evidence-based Software Engineering: based on the publicl... by muench on Monday May 24, 2021 @09:26PM (#27271708)
One of the editors of that book wrote about Derek Jones’s book “I think (at least I hope) this will influence software engineering education [more] than anything I've ever done”

https://twitter.com/gvwilson/status/1326190553738977281?s=20

Re:Evidence-based Software Engineering: based on the publicl... by denton-scratch on Tuesday May 25, 2021 @06:32AM (#27275152)
I picked up this book yesterday, and I'm now about 2/3 the way through (I skipped or skimmed about 1/4 of what I've got through - I'm reading for fun, and the returns seem to be diminishing).

I appreciate the author's frankness about the limitations he's working under: if there's no data available about some topic, he doesn't discuss that topic. In fact he's pretty rigourous about 'following the data'.

I think I would like just a little bit more speculation about e.g. developer motivations. And it's a shame there's not enough data to support an analysis of modern development paradigms such as TDD, pair-programming and so on. Of course that's not the author's fault; anyway, in a sense, the absence of data is itself interesting.

I've found the book interesting. I think there are too many 'data dumps', i.e. descriptions of datasets that don't cast much light on anything.

I think I'd find a review or abstract that distils the book useful and engaging.

Re:Evidence-based Software Engineering: based on the publicl... by gawi on Monday May 24, 2021 @02:59PM (#27268352)
I really like the cover where you can find illustrations of anti-patterns such as stovepipes, gas factories, reinventing the wheel, etc.

Can you names the others?

Re:Evidence-based Software Engineering: based on the publicl... by bckr on Tuesday May 25, 2021 @12:23PM (#27278817)
Oh God, now I must know.

I see a ticking time-bomb and the blind leading the blind. Not sure if those are canonical.

About three dozen other vignettes that I definitely can't pin to anti-patterns. Like man-eating plants, teaching magic to rabbits, gearheads, touching a lightbulb... someone please help.

Re:Evidence-based Software Engineering: based on the publicl... by harveywi on Monday May 24, 2021 @04:22PM (#27269246)
A cover when the document is digital.
Re:Evidence-based Software Engineering: based on the publicl... by swiley on Monday May 24, 2021 @04:29PM (#27269333)
I wish more stuff came with fun illustrations as canonical "covers." Also the file looks like it's probably exactly what they gave to the printers (the left and right margins are different for odd/even pages.)
Re:Evidence-based Software Engineering: based on the publicl... by bckr on Monday May 24, 2021 @02:02PM (#27267642)
I'm glad this has been published. It would be tempting to delay such a publication until more data is available. However, the author(s?) are doing it in the way of honest science, and hopefully this will catalyze further work on EBSE. Who knows, maybe the big companies will decide that the rising tide effect is worthwhile and fill in the gaps.
Re:Evidence-based Software Engineering: based on the publicl... by lifeisstillgood on Monday May 24, 2021 @04:03PM (#27269036)
>>> based on an analysis of all the publicly available data This aim is not as ambitious as it sounds, because there is not a great deal of data publicly available.

Amusing and revealing at the same time. Bet they did not take long to agree on that senrence

Re:Evidence-based Software Engineering: based on the publicl... by klausnrooster on Tuesday May 25, 2021 @06:17PM (#27283446)
"Those seeking discussions written in the style of a stimulant for mathematical orgasms, will not find satisfaction here."

Hahaha, classic. Like his style in general.