News for hackers. Stuff that matters.
Re:Why are we so bad at software engineering? by ken on Wednesday February 12, 2020 @06:15PM (#22313828)
Fashion. We're not bad at software engineering. We're bad at correctly labeling what is "software engineering" and what is not.

"Software engineer" today is just a job title. It makes us sound cool, but it has about as much to do with engineering as "Scrum master" has to do with rugby. In lots of places, you don't need to have an engineering degree or be a certified engineer to claim the title.

Teams that actually practice Software Engineering, like the famous Lockheed shuttle group, are as good at it as we are at any other branch of engineering.

It's wildly misleading to use the same word for an app written by a few programmers in a couple of months (like the Iowa Caucus app) and the Space Shuttle control software (maintained by hundreds of people, with an average of 1 bug per release), and everything in between. It'd be like letting anyone call themselves Doctor.

Re:Why are we so bad at software engineering? by westoncb on Wednesday February 12, 2020 @07:45PM (#22314355)
> It's wildly misleading to use the same word for an app written by a few programmers in a couple of months (like the Iowa Caucus app) and the Space Shuttle control software (maintained by hundreds of people, with an average of 1 bug per release)

You are drawing a distinction here between two vastly different project types—not between the roles of people on the projects.

Of course something that has a million times the budget has the potential for significantly higher reliability.

Additionally, the particular requirements of the space shuttle software project make reliability extremely important.

If a computer programmer were to allocate similar priority to reliability in an app for chatting with friends, I'd argue this would make them less qualified for the title of engineer.

My understanding of what characterizes an engineer is that they make effective decisions about technical tradeoffs, and come up with plans that can be realistically executed while optimizing a set of project requirements (technical and non).

In my experience, software engineers are (often) appropriately labeled as such, insofar as the work they do matches that definition (if the definition is inadequate, that's another matter—happy to hear an alternative).

My hypothesis is that people start getting fussy about the 'software engineer' label for two reasons:

1) There's a lot of variety in software work being done. Not everyone writing code is an engineer. Not sure about the situation at large companies, but at least in the startup world, in my experience, everyone I've worked with who had the title deserved it in the sense that they were doing engineering work according to the above definition.

2) The requirements for software projects—especially in terms of the importance of reliability—are very different from those in more traditional engineering domains, which can create a superficial appearance of poor/non-existent engineering. But this is exactly what you'd get if the engineer did an extremely good job, but with project requirements that don't place high value on e.g. reliability.

It would be great if every app could have formally verified code etc.—but the reason they don't has to do with businesses funding these projects, not with the engineers working on them.

Re:Why are we so bad at software engineering? by AnimalMuppet on Wednesday February 12, 2020 @10:12PM (#22315037)
> It would be great if every app could have formally verified code etc.—but the reason they don't has to do with businesses funding these projects, not with the engineers working on them.

You can look at it from the company's point of view: Net profit would go down if they formally validated everything. Put that way, it sounds horribly selfish of the business. But you could also look at it from society's view: Net value to society would go down if everything was formally validated. Why? Because much less software would be created. And, for something like a chat app, we (society) don't actually need the level of reliability that we'd get from formal verification. Don't waste the effort doing it, because it's a net loss. Go spend the time building something else.

The result is that we get chat apps that crash. But we also get much more total stuff. I think that's a net win, even though it's annoying when the chat app crashes.

Re:Why are we so bad at software engineering? by walleeee on Thursday February 13, 2020 @06:45PM (#22322538)
> ...much less software would be created.

I'm not convinced this is necessarily a bad thing.

Re:Why are we so bad at software engineering? by ken on Wednesday February 12, 2020 @10:30PM (#22315099)
> If a computer programmer were to allocate similar priority to reliability in an app for chatting with friends, I'd argue this would make them less qualified for the title of engineer.

While it's entirely correct for an engineer to use different materials in different circumstances, I don't think I've seen professional engineers in other fields choose methods which they know will fail under expected stresses simply because it's not considered an important project. There's still a safety factor.

> Of course something that has a million times the budget has the potential for significantly higher reliability.

I've seen software projects with funding everywhere between $0 and $outrageous, and the latter never use their extra funding for reliability. It's always for scope. They build software in exactly the same flimsy way -- just a lot more of it. You never see a press release that says "We took $5M in funding so we can finally upgrade from SCRUM to SEI 3, and fix all those dang bugs."

Re:Why are we so bad at software engineering? by westoncb on Wednesday February 12, 2020 @10:51PM (#22315191)
> I don't think I've seen professional engineers in other fields choose methods which they know will fail under expected stresses simply because it's not considered an important project. There's still a safety factor.

It's not that the project is unimportant so the engineer doesn't care about it, it's that the project's requirements have a certain tolerance for failures. The project dictates that tolerance, not the engineer; the engineer develops and implements a plan that works within the tolerance.

And no, typically there is not a safety factor, which is part of why the tolerance is higher for these projects.

> They build software in exactly the same flimsy way -- just a lot more of it

Again, this is not something determined by the engineers—it's just part of the project requirements the engineers are employed to work from.

I think your gripe is more with something like software product managers than it is with software engineers.

Re:Why are we so bad at software engineering? by thu2111 on Thursday February 13, 2020 @08:24AM (#22317502)
Nobody writes code that fails reliably under expected conditions. Reliability is a question of robustness in the face of unexpected or rare conditions.

BTW, chat apps are a bad example. Not sure why this thread is using them. WhatsApp has higher uptime than the telephone system itself, if I recall correctly. Making highly scalable and available message switches is a solved software/computer engineering problem.

A better example is most bespoke business software. If it collapses in a messy heap because a microservice ran out of disk space, people shrug and put in alerts to make sure disk space doesn't run low again without an operator being notified. Job done. Not exactly a rigorous approach to failure but it'd make no sense to spend weeks or months on making the entire system fail gracefully in that situation when it's easily avoided.

Re:Why are we so bad at software engineering? by city41 on Wednesday February 12, 2020 @06:58PM (#22314096)
Whether we're bad at software engineering or not doesn't seem to be the point to me. I think the point is economics and other factors causes us to forego engineering almost every time. The space shuttle control software is a massive outlier in a giant body of software.

edit: although looking back at the article and paying closer attention to the title, I can see why the point I replied to was made.

Re:Why are we so bad at software engineering? by macintux on Wednesday February 12, 2020 @07:27PM (#22314282)
> Teams that actually practice Software Engineering, like the famous Lockheed shuttle group, are as good at it as we are at any other branch of engineering.

I’m not sure that’s true, or perhaps more accurately, I’m not sure that bar is high enough.

No other branch of engineering, at least none that come to mind, have to deal with the number of variables that any medium-sized software application does. Software, being infinitely malleable and running on a seemingly infinite variety of hardware in an infinite number of environments, is vastly harder than any other engineering.

Re:Why are we so bad at software engineering? by bigger_cheese on Wednesday February 12, 2020 @08:17PM (#22314510)
>No other branch of engineering, at least none that come to mind, have to deal with the number of variables that any medium-sized software application does. Software, being infinitely malleable and running on a seemingly infinite variety of hardware in an infinite number of environments, is vastly harder than any other engineering.

I'm not going to argue if this is right or wrong as I don't have a Software Engineer's perspective. My background is in chemical and materials engineering in heavy industry (mining and manufacturing sectors).

I've worked in Operations and Process Engineering positions (https://en.wikipedia.org/wiki/Process_engineering) and in more recent years I've been employed as a "Technology engineer" which is essentially integrating new technology and process improvements and optimizations into existing industrial applications.

The scope and complexity of what Engineers deal with daily might surprise you, software is complex so to is something like a smelter or refinery.

Re:Why are we so bad at software engineering? by ggregoire on Wednesday February 12, 2020 @10:27PM (#22315089)
Pretty sure I can spend 15 min on Wikipedia and find dozens of engineering fields vastly harder than software engineering.

Also, your definition of software is really far from reality. 99% of software don’t have to deal with hardware directly and are compatible with a number of environments I can count on one hand.

Re:Why are we so bad at software engineering? by bkanber on Thursday February 13, 2020 @01:47AM (#22315821)
There are engineers who have to worry about quantum effects that physicists barely understand. I think, by definition, that's harder than Turing-complete software.
Re:Why are we so bad at software engineering? by watwut on Thursday February 13, 2020 @02:11AM (#22315925)
Wast number of hardware and infinite number of environments is not contemporary issue with software. Contemporary issue is that software does not run properly even when it was meant to run at one specific piece of hardware, is often not tested at all, contracts are made for impossible deadlines and even software practices known for years are still not made (as much as management claims to do them to customers).
Re:Why are we so bad at software engineering? by mLuby on Wednesday February 12, 2020 @09:40PM (#22314879)
There exist cheap, dangerous DIY aircraft and expensive ultra-performant bug-free programs, but mostly we see expensive, safe airplanes and buggy but cheap(ish) software.

Also SWEs do a lot more "on-the-fly" modifications than physical engineers, simply because it's so much easier in software.

Re:Why are we so bad at software engineering? by ggregoire on Wednesday February 12, 2020 @10:31PM (#22315105)
Agree. Also, nowadays everybody is “senior software engineer” after 2 years of experience, which is ridiculous.
Re:Why are we so bad at software engineering? by agumonkey on Wednesday February 12, 2020 @07:41PM (#22314342)
I wonder how different was the situation before internet and to a point before personal computers.

Computer engineering was probably slower and deeper thought out. Or am I imagining an ideal past ?

Re:Why are we so bad at software engineering? by kingludite on Wednesday February 12, 2020 @08:02PM (#22314440)
It use to be this: Making something that works is step 1, then you rewrite it again and again until performance is what you need from the application. What seem like bad ideas work sometimes and what seems like a good approach may not work. A lot of silly trial and error. It was good fun seeing hacks that far exceeded what was thought possible. Oh? Its 1000 times faster than my best approach?? what the hell??

Ease of development was an utopian dream.

Re:Why are we so bad at software engineering? by tastyfreeze on Wednesday February 12, 2020 @06:55PM (#22314085)
100% agree. Software Engineer/Architect gives plans to the software developers. Software developers actually build the thing to the specifications of the engineer just like a construction crew. But, all of that is only true if there is an actual license to be a Software Engineer that is held in the same esteem as other engineering fields. Until then the conflation of software engineer and programmer/developer will persist because it makes sound cool.
Re:Why are we so bad at software engineering? by zeroonetwothree on Wednesday February 12, 2020 @08:57PM (#22314690)
This is the dumbest comment in the whole thread. You can’t design software if you aren’t building. There’s just so many unforeseen problems that come up when you’re actually in the thick of it. It’s not like a house that you build 1000 times from the same plan. Every single piece of software is unique in some major way.
Re:Why are we so bad at software engineering? by mLuby on Wednesday February 12, 2020 @09:42PM (#22314894)
There are so many CRUD apps out there, they could be spec'd with such specificity that they only need a "coder" to complete the tickets. But the tech industry's combined architects/engineers and construction workers into a single developer role.
Re:Why are we so bad at software engineering? by pwm on Thursday February 13, 2020 @12:58PM (#22319797)
There are no such thing as a CRUD app and every app is a CRUD app.
Re:Why are we so bad at software engineering? by mLuby on Thursday February 13, 2020 @02:17PM (#22320497)
Reductio!

Someone who can make a boxy UI, hook those boxes up to {{text templates}}, and make GET/POST/PUT/DELETE calls (to RESTful API just thinly wrapping a SQL DB) can recreate a shocking number of popular web and mobile apps. Those are CRUD apps.

Non-CRUD apps have an unusual (non-boxy) UI, non-trivial backend processing/integrations, or both.

Re:Why are we so bad at software engineering? by wruza on Thursday February 13, 2020 @02:47AM (#22316048)
I wish I was a software developer you’re describing. You wake up in the morning without a stress about 20 yesterday’s daily decisions, take your todays tasklist and just do what you did everyday with your well-known tools and knowledge for fifteen years in the field. That’s how most of us want it, no sarcasm or something.
Re:Why are we so bad at software engineering? by WrtCdEvrydy on Wednesday February 12, 2020 @07:25PM (#22314271)
Funny thing, the PE software engineering exam was so unpopular they discontinued it... https://ncees.org/ncees-discontinuing-pe-software-engineerin...
Re:Why are we so bad at software engineering? by xnx on Wednesday February 12, 2020 @06:38PM (#22313965)
Agreed. An insult to real engineers. As misleading as calling programmers Software Doctors.
Re:Why are we so bad at software engineering? by exdsq on Thursday February 13, 2020 @03:55AM (#22316298)
You can become a chartered engineer based on software engineering expeirence and a CS degree. Ergo, you can be a software 'engineer'.
Re:Why are we so bad at software engineering? by junar on Wednesday February 12, 2020 @09:16PM (#22314766)
Ironically, the title "Doctor" originally meant "teacher".

https://www.merriam-webster.com/words-at-play/the-history-of...

Re:Why are we so bad at software engineering? by wruza on Wednesday February 12, 2020 @07:00PM (#22314105)
What does SWE actually do? I mean, I have some ideas how e.g. building or bridge engineers work and how they plan their projects. Standards, blueprints, legal safety requirements, physics, calculations, approval, etc. But what is software engineering in short?

Upd: is there AutoCAD for software?

Re:Why are we so bad at software engineering? by tracker1 on Wednesday February 12, 2020 @08:03PM (#22314449)
Most software and for that matter, most software developers don't need that level of detail. It's the difference between building a nice doghouse and a skyscraper. The problem is too few know the difference, fewer still know how to hire properly, and many seem to think that one developer is interchangeable with any other.

Of course tethering to education requirements doesn't mean much. I think a formal guild where you build your representative value by the endorsement and status of others, and if you fail, not only are you brought down, but those that endorsed you will lose cred as well. Setting up such a system would be difficult and take time, and I'm unsure if it would ever reach critical mass.

Also, unlike doctors and other fields, I don't think you should have to be formally licensed to work in software. Most initial developers are, were, and continue to be cross trained from other fields.

Re:Why are we so bad at software engineering? by wruza on Thursday February 13, 2020 @01:55AM (#22315860)
>It's the difference between building a nice doghouse and a skyscraper. The problem is too few know the difference

That’s what my question is about!

Building engineer knows or reads N54.s-1e/624 “on public safety in a first floor lobby” to design it. It may specify various requirements like no sharp edges, glass walls need stripes on them, non-slippy floors, etc. Then when they need to add an electronic lock to all main entries, they contract with a seller whose locks are licensed with LS9-41.3/FLL. Then firefight inspection validates these entries for public safety again, involving standardized legal rules.

I don’t even try to describe the actual building from the ground to the concrete structure process, because that will be too naive. Lots of standards, requirements, ready-to-use plans, or a project approval process that requires half a year for a shallow review of all documents in many controlling departments.

What I’m interested in, is there such a level of engineering in software at all. I don’t know about anyone who did something like a regular building engineer does at the / before the construction site, but in software. Is there such a thing like Software Engineering at all? All popular software doesn’t seem to have signs of no-dog-house, when you read their code.

Re:Why are we so bad at software engineering? by jinglebells on Friday February 14, 2020 @05:31AM (#22325446)
I'm not sure it answers your question but in software, the requirements are almost never codified up front because extracting the rules is part of the project.

In terms of the dog-house, there's that rule "Build one to throw away". Since no two software projects are similar, the only way to truly know what you're building is to build it, see what works, throw it away, then build it properly.

Part of what makes software so good is that you _can_ upgrade the skyscraper while people are in it, metaphorically hardening floors or adding new levels.

Re:Why are we so bad at software engineering? by jinglebells on Friday February 14, 2020 @05:35AM (#22325462)
I think the software metaphor for your building codes is basically "Software coding standards and guidelines" and most large platforms will have them. For example https://www.oracle.com/technetwork/java/seccodeguide-139067....

And yes, good software engineers are expected to know those practices.

Re:Why are we so bad at software engineering? by tracker1 on Saturday February 15, 2020 @05:01AM (#22334333)
Aerospace, higher end weapons systems and most medical hardware+software... It's usually when in conjunction with life/death physicality that these things come into play.

Unfortunately, the efforts on some of these systems are much harder to work through in practice than something created more organically.

Re:Why are we so bad at software engineering? by closetohome on Wednesday February 12, 2020 @08:21PM (#22314530)
This is also where non-software companies get into trouble trying to do software projects. They think they're building a house so they hire a carpenter. They don't realize they're also going to need a general contractor and maybe a carpet guy.
Re:Why are we so bad at software engineering? by thu2111 on Thursday February 13, 2020 @08:27AM (#22317518)
I think a formal guild where you build your representative value by the endorsement and status of others, and if you fail, not only are you brought down, but those that endorsed you will lose cred as well.

You mean like, a tech firm? Or a software consultancy business? :)

Re:Why are we so bad at software engineering? by tracker1 on Saturday February 15, 2020 @05:06AM (#22334352)
No, I mean a guild... Closest modern representations would be somewhere between a professional board and a union. Bob was trained by the great Tim, who created Foo and Baz. Tim endorsed Bob as a Journeyman from an apprentice. Steve and Phil (both master level) later endorsed Bob as a Master craftsman.

Bob screws up big time... complaint filed with guild board for review... board revokes Bob's master status (back to journeyman) and notes on Steve and Phil's record.

If Steve and/or Phil get too many hits, they lose their status as well.

Re:Why are we so bad at software engineering? by thu2111 on Monday February 17, 2020 @05:36AM (#22346707)
My point is that what you're describing is a firm, just with different terms.

Bob was trained by the great manager Tim. Tim endorsed Bob from internship to junior engineer to senior architect. Steve and Phil both later recommended Bob for promotion to CTO.

Then Bob screws up big time. The board of directors gets involved. Bob is fired and eventually gets a new job as a middle-manager or lower in some other firm, or if the screwup is large enough, takes early retirement.

The "guild" is doing the same things as a company does. But judging skill, success and failure is quite subjective. That's why we need so many competing firms to get results. A single guild would be like everyone working for a single company. It'd yield very poor results.

Re:Why are we so bad at software engineering? by _jal on Wednesday February 12, 2020 @08:18PM (#22314521)
One big difference is that, at least in many jurisdictions, new (non-software) engineers need to survive what amounts to an apprenticeship. A second one is passing standardized exams and demonstrating experience before achieving the title.

Now, I don't think apprenticeships are unmitigated positives - another function they serve is to limit field entry. But they are, I think, far better than what "we" do, and sorry, pair programming is not a substitute.

I don't know what does AutoCAD have to do with anything. I use it for modeling parts that I machine as a hobby; that doesn't make me a structural engineer.

Re:Why are we so bad at software engineering? by wruza on Thursday February 13, 2020 @02:12AM (#22315929)
By AutoCAD I mean some app that helps you to design architecture and components and then validate these against existing rules. Not a literal mechanics/blueprints AutoCAD, but ?-CAD for software development.

>But they are, I think, far better than what "we" do

They should be far better, but an engineer is not an abstract position. If all it needs to be an SWE is to know 3-book algorithms and some patterns (just my uneducated guess), then it is not engineering. See my other comment for details. Yes, a CS degree adds to ones competence, but it is like saying that physics degree allows you to build an elevator. It doesn’t, it will be just a nice science-grade dog house level elevator, because learning physics doesn’t make you know all standards that elevators require, legally and practically.

Re:Why are we so bad at software engineering? by exdsq on Thursday February 13, 2020 @03:17AM (#22316153)
Maybe something like a UML or data-flow diagram could be similar? There are more formal specifications you can do, as part of 'formal methods', which I think might be closer to some engineering disciplines [0] - here's one from where I work.

I do think software engineering as a discipline exists. Looking at the definition written by the American Engineers' Council [1], there have definitely been teams I've worked on where we met this criteria. There are teams like the Lockheed shuttle group who average 1 bug per release that clearly embody this criteria too.

Also, my CS Masters is accredited as meeting the educational requirements for Chartered Engineer status (which I'm hoping to achieve once I finish) so it's clearly accepted by some in the engineering community!

[0] https://hydra.iohk.io/build/789825/download/1/ledger-spec.pd...

[1] https://en.wikipedia.org/wiki/Engineering

Re:Why are we so bad at software engineering? by vonmoltke on Thursday February 13, 2020 @08:25AM (#22317507)
> One big difference is that, at least in many jurisdictions, new (non-software) engineers need to survive what amounts to an apprenticeship. A second one is passing standardized exams and demonstrating experience before achieving the title.

"[M]any"? Outside of a handful of really anal jurisdictions like Canada I am not aware of any where this is a requirement. I chose not to go this route because I didn't see the value in it for the type of electrical engineering I was interested in.

Re:Why are we so bad at software engineering? by HeyLaughingBoy on Wednesday February 12, 2020 @09:09PM (#22314736)
Delve into the courses listed here, it might help: https://www.msse.umn.edu/MSSE-Master-Science-Software-Engine...
Re:Why are we so bad at software engineering? by thebean11 on Wednesday February 12, 2020 @08:10PM (#22314480)
> is there AutoCAD for software?

JetBrains IDEs?

Re:Why are we so bad at software engineering? by 0xdead on Thursday February 13, 2020 @03:21AM (#22316169)
Totally agree. SWE is the only field where people can call them "Engineers" without having an actual engineering degree.