Maybe we have to teach some of these upstarts about Therac-25 and depress another generation just to keep the harm from spreading :(
I say this as someone who programs, teaches programming/computer science, and has worked professionally as a software development engineer - the comment isn't made to denigrate anyone, but because software "engineering" doesn't have any certification / professional credentialing that "real" engineering (e.g., civil engineering) apparently does have.
Also - I'm curious about how effective it is to tell the engineers to do the right thing (even at the expense of personal gain). It seems like the people most responsible for making decisions that build harmful products aren't the ones who are actually tasked with building the harmful products. Does anyone have any information about how effective such training / indoctrination is in preventing bad situations from happening?
https://www.ordemengenheiros.pt/pt/a-ordem/colegios-e-especi...
No Portuguese university is allowed to call their computing degrees engineering without their approval, and in certain kinds of state related projects, the one putting their signature into the legal contracts has to have the professional title acknowledged.
Germany and Switzerland have a similar process for those carrying engineering on their title.
Either that or you have done it via a trade school, and people on the job have to have a kind of teaching certification for the school to approve the company as teaching place.
If you happen to be someone like our former prime minister, or being lucky that no one bothers to sue because of how long everything takes.
However, depending on the company, it might be a matter to lose the job.
Where it definitely matters is signing legal binding contracts.
Also not everyone and their dog gets in unless they prove their skills.
Most private university are seen as scams.
You should start by shooting down all the university scams in US to start with.
Yeah and I hope it stays that way. Imagine needing a license to write software. It would mean the end of free and open source software as we know it.
> Does anyone have any information about how effective such training / indoctrination is in preventing bad situations from happening?
It's not about training or indoctrination. It's about legal responsibility and liability. Make professional programmers directly responsible for the code they get paid to write. If society finds they're causing harm, it will hold them accountable for it.
We must increase the risk associated with building unethical technology. Programmers need to think twice before they build the next surveillance capitalism nightmare.
Our society does a really bad job at holding people accountable for the harm they do. What makes you think it would be any different for software?
i don't think so because as you said:
> It's about legal responsibility and liability. Make professional programmers directly responsible for the code they get paid to write.
we urgently need liability for for-profit software in order to stop softwares race to the bottom.
I'm indirectly liable, in that I can lose my job, however.
If I were a contractor, I would also buy liability insurance.
If an engineer negligently (note: not knowingly) signs off on a bridge that collapses, they face fines, expulsion from their profession and jail time. At this point, every developer at Facebook should know the harm they’re causing. The most they face is job loss. That’s the problematic asymmetry.
software quality is not some kind of "force of nature" or "accident" but a direct result of cutting corners/costs.
insurance for beeing lazy will not help.
If we're going to do that, there's no good reason to give a special exclusion for not-for-profit software.
Besides, it's trivial to launder everything if you give such an out. The regulation can apply equally to everyone, or don't expect it to go anywhere at all.
Amateur programmers are not professional engineers. A profession is an activity people are compensated for. If you're not being compensated, it's not professional work. If it's not professional, you should not be held liable for it.
Someone publishing a personal project should not be held liable for anything. A professional software engineer deciding to use that project at work without thoroughly reviewing it and causing a security breach due to vulnerabilities in that software should absolutely be held liable for negligence.
> An engineer designing bridges is liable regardless of his salary, and same must apply to a programmer.
I'm not an engineer but I can draw up a bridge right now. Would it stand or collapse? Who knows. I think it looks nice though so I post it online. Then an actual engineer sees my drawing and makes an actual bridge out of it. The bridge collapses and kills thousands.
Who's liable? Me, a random guy who drew a bridge? The engineer who should have known better?
> Besides, it's trivial to launder everything if you give such an out.
That would require companies to not only refuse to compensate their programmers for their labor but to also open source their source code. Doesn't seem trivial to me.
Right, it's not the publishing but the use which is proscribed - same applies for engineering. But that would apply regardless of whether the software you're using is open-source or not, and regardless of how much you're paid.
A social-media site operated by an amateur in the bedroom being paid nothing and using only open-source software would still be liable under any sensible and realistic regulation.
>>Besides, it's trivial to launder everything if you give such an out. >That would require companies to not only refuse to compensate their programmers for their labor but to also open source their source code. Doesn't seem trivial to me.
You can compensate for code with this regulation, it's not difficult. e.g. create a non-profit, and pay the non-profit which pays the programmer. So long as the programmer is one step removed from you it's fine.
Open source isn't much of an impediment either, just have a suitably restricted licence. Or don't, it's not an impediment for many types of businesses.
Exactly.
I think there should be an exception for free and open source software though. We have lots of free software designed to be used by people. I don't think these programmers should be held liable. Many of them are amateurs and volunteers. The chilling effect would simply be too great.
I'm reminded of Greg Kroah-Hartman's talk about kernel contributors.
https://youtu.be/fMeH7wqOwXA&t=326s
> My goal, of me, as a developer and to make Linux succeed, is to help [amateurs, unknown individuals] out.
> If you're hired to do this stuff, you're on your own and you better know what you're doing.
I think he's right. At the time, roughly 20% of the kernel was done by individuals. That work is extremely valuable yet it is different from professional work. It should be treated differently. with a lot more compassion and understanding. On the other hand, it is fine to expect professionals to be disciplined.
> A social-media site operated by an amateur in the bedroom being paid nothing and using only open-source software would still be liable under any sensible and realistic regulation.
Yeah, I agree. That's not just software anymore, that's a service.
> e.g. create a non-profit, and pay the non-profit which pays the programmer. So long as the programmer is one step removed from you it's fine.
It doesn't matter how indirect it is. The important fact is the programmer is getting paid for their work. That makes it professional work.
Do kernel contributions getting paid by the Linux foundation count? What if the offending company is donating to the Linux foundation?
>I think there should be an exception for free and open source software though. We have lots of free software designed to be used by people... The chilling effect would simply be too great.
I understand distinguishing between publishing and use. But we're talking about use here. The person responsible for the use of the bad program is usually not the contributor. So there's no chilling effect here?
then follow-up with whether they would feel morally/spiritually at peace with their choices and methods after the fact (do your ends justify your means?)
in my experience, the people with the IQ to earn a yacht responsibly end up settling for a speedboat anyway.
A software developer glues together 400 node modules they don't understand and ships to Other People's Computers, then hands them their debit card when the load goes up. A software engineer knows that every line of code is accounted for, because they've actually done the accounting, and knew the scale requirements in advance (with error bars) and provisioned the infrastructure accordingly - with a budget.
IIRC, they do, and it doesn't work. The "Californian Ideology" has a strong hold on the tech world and it's incompatible with that kind duty,
Wikipedia says the oath that goes with it is:
I am an Engineer.
In my profession I take deep pride. To it I owe solemn obligations.
As an engineer, I, (full name), pledge to practice Integrity and Fair Dealing, Tolerance, and Respect, and to uphold devotion to the standards and dignity of my profession, conscious always that my skill carries with it the obligation to serve humanity by making best use of the Earth's precious wealth.
As an engineer, I shall participate in none but honest enterprises. When needed, my skill and knowledge shall be given without reservation for the public good.
In the performance of duty, and in fidelity to my profession, I shall give the utmost.
---
I get the impression this ring is not for "software engineers". I really dislike that term, I call myself a computer programmer to anybody who asks, regardless of the title my employer flatters me with. I think "software engineer" is leaching off the social prestige real engineers have earned for their professions. Prestige the software industry in general does not yet deserve. Programmers take jobs making shitty socially harmful products then deflect all blame to their employers, denying their own responsibility to society that oath describes.
I am sure there are some programmers who deserve it, but by in large, the term is used by companies to flatter code monkeys making gambling apps, spyware, social media skinner boxes, etc. I am not without sin here, so I refuse to call myself an engineer.
Anyway, at my university (CS/computer eng. major) everyone in the engineering school goes through the same induction ceremony that I think had a similar oath. I still have the Wash. U. Engineer's Creed card in my wallet. I vaguely remember that Tau Beta Pi had a similar thing. Honestly, I appreciate the ideals, but I don't think the oath made folks from my program any more ethical than those from other schools.
This is of little comfort to the many victims of US foreign policy over that same period
Machine translated to English (FI MUNI is heavy into NLP anyway ;-) ) it looks like this:
a) at the bachelor's degree: “I solemnly promise to dedicate my life to the service of humanity; I will keep Masaryk University, where I obtained a bachelor's degree, and its teacher in the memory and respect that belong to them; I will be faithful to my profession and I will be fair and accommodating to his colleagues; I will develop the projects I participate in to the best of my ability so as to serve man; I will not tolerate evil, bad practices or bribery; I will not allow that issues of religion, nationality, race, party politics or social status my professional decisions; I will not abuse my professional knowledge and skills even under duress. So I promise to my honor, freely and of my own free will. "
b) at the master's degree: “I solemnly promise to dedicate my life to the service of humanity; I will maintain my love and gratitude for Masaryk University, where I obtained a master's degree; I will responsibly perform all my professional duties and consider the ethical implications of my professional activity; I will not allow my activity to be in conflict with the rights of individuals, groups or organizations to respect their privacy and integrity and will not allow their knowledge to be misused and the ability to enable their violation; I will not abuse the properties of the processing systems information or knowledge of them for their personal benefit; I will be in my professional career to act with awareness of the limits of my professional competence and the field in which I work; I will help deepening awareness of the nature and possibilities of their discipline in society. So I promise myself honor, freely and of one's own free will. "
c) at the doctoral degree “I solemnly promise to dedicate my life to the service of humanity; I will preserve love and gratitude to Masaryk University, where I obtained a doctorate; I will carry out all their professional duties responsibly and consider the ethical implications of their actions in their profession and scientific field; I will expand and develop knowledge in my discipline; I will work in this direction to deepen / deepen the awareness of ethical responsibility for the consequences application and use of procedures and knowledge of informatics in society; I will respect protection intellectual property rights and to weigh it responsibly in relation to the free flow of the open scientific knowledge in international public ownership. So I promise to my honor, freely and of their own free will. "
I strive to respect this oath but I'm not sure if all the other absolvents do or even remember taking it. :P
I really don't want anything to do with them. I'd still work for a military software developer. But not Facebook.
Frankly, I was also surprised recently, though I shouldn't be, that there are (or were) some quite 'famous' developers / comp-sci researchers working there.
Would you use open source packages developed by them?
At some point the brand will become so toxic that engineers who've worked there could find themselves persona non grata in many tech roles.
If they can convince me that they stayed there in a ‘change it from within’ capacity, great. Otherwise, yeah it’s a negative signal.
The majority of startups is still "growth and engagement" as opposed to producing real products/services that people pay for, so ex-Facebook is the perfect match.
"Once the rockets are up, who cares where they come down? That's not my department,"says Wernher von Braun."
Smart people know that you gain nothing by sacrificing yourself for society. Even if you cared enough to say no, all that would happen is that Facebook would pay somebody else to work for them.
Nothing? Not even a clear conscience?
The "but someone else will do it" argument is used to justify so many wrong doings… We hear it all the time about fighting climate change, building unethical weapons, etc.
You also have the choice to not personally actively contribute to whatever unethical thing is being discussed. It's a circular game theory problem, that "other person that will do it anyway" might also be justifying it because they think you would do it if they didn't.
Serious question: Do you vote? Because this argument seems like people who say “my vote doesn’t matter, it’s just one vote out of millions.”
It’s true that at the individual level, one vote basically never makes the difference, but elections are won and lost. In the same way, if one person says no, maybe it will barely make a difference, but teams are made of more than one person.
I've worked in banking and my ex worked for Facebook, and I genuinely don't see the comparison beyond rather inaccurate tropes. Bankers aren't tearing apart society and sowing division like Facebook is.
It will attract thd clueless and the most ambitious. People who are ready to defend it for their own profit.
Its a trap where the hole they are in just keeps growing deeper.