News for hackers. Stuff that matters.
Re:Everyone is still terrible at creating software at scale by pdimitar on Saturday April 10, 2021 @08:01AM (#26760864)
You do have an excellent point for part of the cases.

We should IMO also entertain one more possibility: namely pressure from the top.

A lot of people who I knew as sympathetic and calm before they took management roles turned into something I could code in one minute: namely a program that asks "how much is this going to take?" and if your answer is above N hours/days then they say "no, we're not doing it". And that's not because they are stupid or suddenly non-sympathetic. It's because their bosses optimize for those metrics and measure them (and their salary, and their bonuses, and even their vacation days) by the same metrics.

Perverted incentives play a big part in the current state of things where everyone just looks baffled at thing X and is like "why in the holy frak isn't this working better?".

Truth is, MANY, and I mean really A LOT of people know that their work is sub-optimal but pressure from above prevents them from making it better.

And people are not independent -- for most people outside IT and other prestige / high-skill professions, standing up to the boss means 3 to 9 months of unemployment and maybe no food on the table. Hence everybody learns to suck it up and do the bare minimum that keeps them employed, perpetuating the cycle.

Multiply that to at least 90% of our world and it's IMO quite easy to understand why it is like it is.

Re:Everyone is still terrible at creating software at scale by zaphar on Saturday April 10, 2021 @08:19AM (#26760953)
Some of what you say about management is true. But if you really want to fully understand then you need to keep following that trail of incentives. Your bosses boss doesn't optimize for those metrics "just because". There are reasons and persuading them requires understanding those reasons. A company has only so much revenue. Wasting that revenue in the best case reduces profits. In the worst case it kills the company. Changing a perverse incentive requires understanding whether it is actually perverse or not and then proposing a compelling alternative that results in better results for the company.
Re:Everyone is still terrible at creating software at scale by pdimitar on Saturday April 10, 2021 @09:30AM (#26761358)
I am sure that what you say is very true for not a small amount of businesses.

My issue is mostly with my profession -- programming. Sometimes we are so micromanaged and our time is so sliced and diced that all of this short-term thinking easily gets 5x more expensive on a short time frame anywhere from 3 to 6 months.

This has been observed numerous times yet IT managers seem to want to never learn.

It's OK to try and save money. But saving money short-term in order to have a huge expense down the line is just being irresponsible from those managers.

Re:Everyone is still terrible at creating software at scale by hallway_monitor on Saturday April 10, 2021 @10:04AM (#26761559)
Having led a team for about a year now let me tell you it's not easy. You are always trying to balance many priorities and even with over a decade of coding experience, it's not always obvious which refactorings or tech debt stories or architectural changes are worth prioritizing now, which can wait till later, and which are engineer pipe dreams that I need to gently but firmly kill as quickly as possible.

I have had similar managers to what was described above, in those types of organizations. It may be a lost cause but it may be an opportunity to work on your influence skills; being able to explain and sell your solution is perhaps even more important than coming up with it in the first place.

Re:Everyone is still terrible at creating software at scale by tacitusarc on Saturday April 10, 2021 @11:18AM (#26762067)
This is really difficult. Perhaps the best thing is to track the proposed technical debt stories, and do not discard them when they are rejected. Instead, also keep track of how much pain is hit that can be associated back to that particular technical debt.

3 years into major project, my team finally got prioritized to work on and improvement that will take us a couple weeks, that we were not able to work on previously because we could not justify its value. But for the past 3 years the organization has lived with pain that this would have resolved. So now we can justify it. And the thing is, it's not like we didn't know 3 years ago when we said we wanted to make this happen but things are going to be terrible if we were not able to prioritize it but in many organizations it's not good enough to just say how bad things will be. People need to actually have felt the pain.

Honestly, I think that's okay on its face. My bigger issue is that frequently there is no or very little trust around engineers who predict what the major problems will be an advocate for them and then it's not like when they're proven right management now listens to them the next time. I think that's the real problem.

Re:Everyone is still terrible at creating software at scale by mavelikara on Saturday April 10, 2021 @12:47PM (#26762833)
> My bigger issue is that frequently there is no or very little trust around engineers

This is a problem. But at any given time management hears programmers arguing for both sides - the "if you're not using Kubernetes you're a moron" and the "learn how to sysadmin and you'll realize you don't need Kubernetes in the first place" (quotes from the top of the thread). In many cases folks in senior management do not have any idea of these things, and they rely on one of these two opinions, resulting in the other half claiming here on HN that management is all bull.

What the profession lacks is a code of practices - like a Joel Test accepted by every accredited engineer.

I am not sure that is net positive for such a rapidly evolving space. Even now people are debating if unit tests are needed and other basic topics. The only "settled" thing about the profession seems to be that gotos are bad.

Re:Everyone is still terrible at creating software at scale by psahgal on Saturday April 10, 2021 @01:40PM (#26763334)
I find that the best senior engineers don't say "if you're not using Kubernetes you're a moron" and "learn how to sysadmin and you'll realize you don't need Kubernetes in the first place", but instead say "Kubernetes is fantastic for this use case because of these reasons..." and "sysadmin would be easier to use than Kubernetes for this problem because of these reasons..."

I'm not sure how one gets software engineers to justify and back up their decisions like this on a regular basis, but I suspect it would help management trust their engineers more.

Re:Everyone is still terrible at creating software at scale by mavelikara on Saturday April 10, 2021 @04:09PM (#26764674)
> I'm not sure how one gets software engineers to justify and back up their decisions like this on a regular basis, but I suspect it would help management trust their engineers more.

More balanced senior engineers (I know anecdotally that there are lots of them) speaking up and challenging their fellow-engineers to justify their extreme positions will be a good start IMO.

Re:Everyone is still terrible at creating software at scale by lanstin on Saturday April 10, 2021 @03:07PM (#26764109)
(Bad) Management chooses to listen to engineers that say agreeable things, not true things.
Re:Everyone is still terrible at creating software at scale by psahgal on Saturday April 10, 2021 @07:11PM (#26766134)
If your manager listens to the engineer who says "Kubernetes is awesome and does all the things!" and doesn't listen to the engineer who says "I think Kubernetes is a bad idea for this project because x, y, and z", then yeah, it's time to find a new manager. Good managers listen to good advisors.
Re:Everyone is still terrible at creating software at scale by spockz on Saturday April 10, 2021 @03:03PM (#26764080)
Anecdotally, we had the opportunity to get better gear for developers than an outdated single 24” 1080p screen and slow computer. Higher management invited a few engineers to share Ygritte ultimate setup. These varied widely between one large ultra widescreen and three small screens. The developers in the meeting started arguing vehemently in the meeting and in the end they couldn’t agree and all developers were left with the same stuff. If you were enterprising enough you could gather two of those 1080p screens until someone from workplace it saw that and installed to desk to spec.
Re:Everyone is still terrible at creating software at scale by cosmodisk on Saturday April 10, 2021 @05:27PM (#26765288)
It was a mistake to give an option to discuss this to start with. It's always if not this than that, x brand is better than y,etc. What matters: is it a Windows/Linux/Mac shop?

No matter what brand,the base unit comes with enough RAM, processing power and spacious SSD.

Everybody gets the same brand,as it's easier to support from the IT perspective.

Keyboard,mouse,monitors should be an individual choice. Some want to ruin their vertebrae by staring at a tiny 13" screen and be happy about it,while some need 3 monitors and 2 keyboards to feel the power,why not.

Not sure why some companies are so tight when it comes to the equipment for developers,as it's always a small fraction of the overall cost of the position.

Re:Everyone is still terrible at creating software at scale by mavelikara on Saturday April 10, 2021 @04:07PM (#26764659)
That is copy-book Sayre's Law: https://en.wikipedia.org/wiki/Sayre%27s_law
Re:Everyone is still terrible at creating software at scale by pdimitar on Saturday April 10, 2021 @02:40PM (#26763849)
This is definitely a very difficult spaghetti to untangle, agreed. But IMO a very good start would be to never accept such polar statements in the first place.

"If you use extremist and simplistic generalizations you are not worth listening to in a discussion" would be the first policy I'd put into a team.

Balanced arguments with pro/con analysis is what makes an engineer senior and a contributor to an argument. Everybody else shouldn't participate the next time around.

Re:Everyone is still terrible at creating software at scale by mavelikara on Saturday April 10, 2021 @04:04PM (#26764635)
Agreed. But the balanced people are the least vocal. As Bertrand Russel noted:

  "The whole problem with the world is that fools and 
  fanatics are always so certain of themselves, and 
  wiser people so full of doubts."
I am someone who moved from a programer role into management. My appeal to the balanced senior engineers is to speak up more. We don't have to swing as much as economists, enough to frustrate President Truman to say:

  "Give me a one-handed Economist. All my economists 
  say 'on hand...', then 'but on the other..."
But we can bring more nuance into discussions - especially calling out those among us with polar positions.
Re:Everyone is still terrible at creating software at scale by cosmodisk on Saturday April 10, 2021 @05:42PM (#26765406)
As a manager, you need to find a way to pull information from people: with some it's a team meeting,with others it's over a meal during the lunch,etc. I even do this sometimes during the management meetings,when I see people clearly not quite happy about something but reluctant to speak,so I ask them directly what's on their mind and that's often enough for them to tell way more than you could have gotten by just expecting for them to speak up.
Re:Everyone is still terrible at creating software at scale by pdimitar on Saturday April 10, 2021 @04:12PM (#26764695)
All of that is sadly true and it's a fundamental problem with humanity. I'd say that the only solution I am seeing is that the managers must proactively shut down the extremists early and hard.

If you demonstrate that you will not submit to extremists, I've found that a good chunk of the more balanced people are then encouraged to speak up.

---

TL;DR: Silence the bad and loud voices, and the wiser and quieter voices will start speaking.

Re:Everyone is still terrible at creating software at scale by ethbr0 on Tuesday April 13, 2021 @11:11PM (#26802524)
> at any given time management hears programmers arguing for both sides

I think it was Andrew Grove (of Intel / High Output Management) who boiled this down to the following algorithm:

(1) Invite all sides to a meeting. (2) Ask them what their decision is. (3a) If they agree, implement the decision. (3b) If they disagree, tell them to discuss it, then give you their decision at the next meeting. GOTO (1)

His point being that managers are the least informed (by virtual of organizational pyramids), and therefore the least effective at picking solutions. Build a team that can arrive at consensus, then trust the team.

Re:Everyone is still terrible at creating software at scale by kclay on Saturday April 10, 2021 @11:29AM (#26762166)
In these cases wouldn't it have been to work on it slowly as a "side project" outside of the team normal priorities? I always tell my team if they can fit it in and not hinder the current sprint by all means go ahead and do it on the side.

I know this goes against the normal and people want to attach "right now " value (hours it took reflected on check) but going beyond the call can also give you leverage when your yearly review comes up for example.

Re:Everyone is still terrible at creating software at scale by hpoe on Saturday April 10, 2021 @04:14PM (#26764717)
I'll also add I only have so much energy in the day, and after a couple of years of fighting against problems I coudn't do anything about and managers who couldn't make the changes, and customers that wanted it now, I burnt out, and instead of spending a useless zoom meeting to work on an improvement I use it instead to make maps for the D&D campaign I am running.
Re:Everyone is still terrible at creating software at scale by Too on Sunday April 11, 2021 @03:10AM (#26768156)
While I've seen several examples of this being the only way to get out of never prioritized debt-cleanup that the team can't bother explaining to management in non-technical business-value terms. The reality is a sprint is always over-packed with stuff to do with zero room for side-projects and expecting people to work weekends on side-projects to solve company business needs is not exactly a sound strategy.

What one can do is is deliberately allot time to the team to work on any side-project as long as they are somehow attached to the business. There are ways to do this more organized, for example innovation-weeks or the famous Google-fridays.

Re:Everyone is still terrible at creating software at scale by tacitusarc on Monday April 12, 2021 @11:16AM (#26780308)
That can work assuming the team isn't already overloaded. But when people are being asked to work 60h a week or more to accomplish their tasks, it becomes untenable.
Re:Everyone is still terrible at creating software at scale by pdimitar on Saturday April 10, 2021 @02:41PM (#26763869)
That's exactly what I am doing and aside from a few wrist slaps people have been okay with such an approach. I always tell them I did this for a few hours during the weekend or while I was waiting our DevOps to resolve an outage these particular 2 hours the last Friday. Seems to work really well.
Re:Everyone is still terrible at creating software at scale by z3t4 on Saturday April 10, 2021 @08:10PM (#26766508)
> People need to actually have felt the pain.

Just like those that never have been hacked dont care about security. Or those that have never lost data dont care about backups. Or those who has never had more then 10 simultaneous users dont care about performance. Basically we do stuff in order to avoid pain.

Re:Everyone is still terrible at creating software at scale by pdimitar on Saturday April 10, 2021 @02:34PM (#26763796)
I wholeheartedly agree that we the engineers must work on our non-tech talking skills!

I've been working on it for a few years now and it's REALLY difficult but even the small wins feel very rewarding and are motivating me to refine my comm skills further.

Re:Everyone is still terrible at creating software at scale by tremon on Saturday April 10, 2021 @12:11PM (#26762535)
Changing a perverse incentive requires understanding whether it is actually perverse or not and then proposing a compelling alternative that results in better results for the company.

While that's true, considering those incentives and changing them if necessary is part of my boss's job, not mine. My job is only to report the misaligned incentives I perceive to my boss. If my boss isn't able to make me understand his decisions, that's my cue to find a new boss. And I concede, those bosses are rare. I've only had two in twenty years of employment.

It takes time to recognize the qualities you need in a boss to perform well. I know myself well enough in that department to actively look for those qualities. Even to the point where I prefer a less challenging job under the right boss than the perfect job under the wrong leadership.

Re:Everyone is still terrible at creating software at scale by alisonatwork on Saturday April 10, 2021 @01:14PM (#26763111)
I agree that often there are good business reasons for killing projects, and plenty of so-called technical debt is actually just the result of pragmatic decision-making. Imo developers should get used to maintaining that stuff, because that's a huge part of the job in most profitable companies.

But there is also a flip-side where I see the opposite happen - especially in startups and VC-funded companies. These companies can take massive losses while at the same time investing huge amounts into elaborate solutions that appear to primarily exist for the purpose of creating impressive bullet points on the resumes of careerist employees. In these companies, even in management, the work seems to be driven by everyone trying to maximize the impact of their own, personal "tour of duty", then making a profitable exit. These incentives are not the same as those of people working at more legacy/boring companies.

Re:Everyone is still terrible at creating software at scale by pdimitar on Saturday April 10, 2021 @02:44PM (#26763894)
> These companies can take massive losses while at the same time investing huge amounts into elaborate solutions that appear to primarily exist for the purpose of creating impressive bullet points on the resumes of careerist employees.

Yep, that's one of the things I had in mind when I said "perverse incentives".

Re:Everyone is still terrible at creating software at scale by bobthepanda on Saturday April 10, 2021 @01:26PM (#26763218)
It also takes time to turn a big ship, and by the time one realizes one needs to make the turn it may be too late.

To extend this to non-tech examples, failing big companies, like Sears, do not disappear overnight; the decline was in full view for over a decade but ultimately the ship could not be turned. With tech the problem may be even worse; Uber still has not turned a net profit, but with tech investors like to throw money at losing businesses. Some turn around, but that's not guaranteed.

Re:Everyone is still terrible at creating software at scale by marcosdumay on Saturday April 10, 2021 @10:19AM (#26761645)
> Your bosses boss doesn't optimize for those metrics "just because".

Your bosses optimize for those metrics because it's the one they can communicate higher up on the short time they can get higher ups attention.

If it were because the metric was any good, there wouldn't be so many cases of companies bankrupted by technical and cultural debits pilling up.

Re:Everyone is still terrible at creating software at scale by pdimitar on Saturday April 10, 2021 @02:46PM (#26763915)
> Your bosses optimize for those metrics because it's the one they can communicate higher up on the short time they can get higher ups attention.

That's their problem and not mine, is it?

A good manager doesn't just reflect the lightning bolt to their underlings. They absorb part of the shock and process / transform what they will transfer down below. If they aren't doing that then they aren't managers; they are uneducated opportunists in suits.

> If it were because the metric was any good, there wouldn't be so many cases of companies bankrupted by technical and cultural debits pilling up.

Really well articulated, kudos.

Re:Everyone is still terrible at creating software at scale by lamontcg on Saturday April 10, 2021 @02:59PM (#26764045)
There's an implicit assumption here that everyone is "doing a bad job" because of "perverse incentives".

But I'd argue that most 'computer nerds' have pretty deeply seated perfectionism flaws.

To take a simplified example, we tend to want perfect security, and like to prove that we're smarter than each other by understanding complicated attacks.

When it comes to running a business, though, a 10 person startup cannot afford to act like their threat model includes defending from hostile nation state actors willing to do Stuxnet-levels of effort to hack them. If they do that much work on security their product will never get off the ground.

The goal of most companies when it comes to security is never to have perfect security, but to have good enough security to stay out of the headlines.

It is like tradeoffs that have evolved in nature. You don't see organisms that maximize one feature at the expense of everything else. That energy always needs to go into competing concerns.

So sometimes as technical people we fetishize the perfect solution to some problem in the Enterprise, when actually doing that solution would be horrible for the Enterprise due to the costs.

(Of course the flip side is sometimes management only cares about window dressing the company so they can bailout with some golden parachutes and that has nothing to do with running a healthy company -- but not every example of a company not being perfect is due to deeply cynical and perverse incentives like that, but may be entirely rational).

Re:Everyone is still terrible at creating software at scale by pdimitar on Saturday April 10, 2021 @03:09PM (#26764129)
> But I'd argue that most 'computer nerds' have pretty deeply seated perfectionism flaws.

My goal here absolutely was not to turn this sub-thread into a ping-pong of "but managers bad at X" and "but but but programmers bad at Y".

I am very aware of the nuance you introduced and I've been guilty of being a tech perfectionist much more times that I'd be comfortable with sharing. But finally I learned.

(Granted, many haven't and maybe never will. In this regard you do have a solid point.)

> So sometimes as technical people we fetishize the perfect solution to some problem in the Enterprise, when actually doing that solution would be horrible for the Enterprise due to the costs.

Absolutely. Not denying it. It's just that what gets me is the reaction to some of the engineers' flaws is to go to the other extreme.

Which is just as bad state of things. Both extremes are unproductive.

Re:Everyone is still terrible at creating software at scale by Xunjin on Saturday April 10, 2021 @08:33AM (#26761015)
"Hence everybody learns to suck it up and do the bare minimum that keeps them employed, perpetuating the cycle."

In my point of view, that's how you lose solutions to stay in the comfort zone, and I'm not saying the people whom actually "just obey orders" but those who make them.

As I gain more experience, with age and some wisdom (I hope), I see that most top-down decisions are actually the worst way to decide things, people who are "at the bottom" maybe not understand how a entire system works however they do really knows (and suffer) on how their work is sub-optimal.

Re:Everyone is still terrible at creating software at scale by dasil003 on Saturday April 10, 2021 @12:04PM (#26762466)
"Top-down" has become an epithet that disgruntled engineers throw around, but the reality is that there are problems at all layers of an org, and they should be solved at the appropriate level.

Trying to solve every problem top-down manifests as micro-management that beats the morale and initiative out of strong ICs. But on the other hand, trying to address everything bottoms-up leads to local maxima, no standardization across tech or product, and warring factions within the company.

There is real art to this. Being able to grok the incentives and structures of a vertical slice through the org (C-level, VP, Director, Mgr, IC) as well as the tech (Brand, Product, Pages, Components, APIs, Middleware, Storage, Infra) is a rare and extremely valuable talent that allows solving problems that are impenetrable by any traditionally defined role.

Re:Everyone is still terrible at creating software at scale by pdimitar on Saturday April 10, 2021 @02:51PM (#26763962)
Everything you said is true. And it still doesn't change the fact that most organizations [I've been in] utilize the top-down approach on 99% with the occasional 1% of bottom-up.

That's an unbalanced state of affairs that completely loses very valuable feedback everywhere along the chain.

Part of the job of a manager is to find this perfect balance on the level they are working at. And most don't. They just pass the pressure put on them, down below to their underlings.

Re:Everyone is still terrible at creating software at scale by marcosdumay on Saturday April 10, 2021 @10:25AM (#26761689)
That's evidently correct, and all the empiric results point into correctness too. As always I recommend reading Deming. Or go directly to the Toyota production system. The best working system is one where management sets the goals and workers decide the means, while one trusting each other to do their job.

https://en.wikipedia.org/wiki/Toyota_Production_System

But somehow, management never accepts this reality. Even the Japanese auto industry is regressing into Taylorism.

Re:Everyone is still terrible at creating software at scale by NotPavlovsDog on Saturday April 10, 2021 @12:18PM (#26762585)
There may not be that much difference between Lean (Toyota) and Taylorism. Don't forget that Toyota (Toyoda) is not that far from Fordism - "you get whatever color you want as long as it is black". Employees get to tweak the process but the fundamental model stays the same. They don't get to change the assembly line, nor the parts, nor what is assembled. And Toyota guaranteed life-long employment when it was implemented.

I've worked the line twice - first time as a teen looking for any work, the second time as an undercover analyst hired to examine why Lean was failing at a plant, both doing the same type of job: electrical component assembly. I got physical injuries (minor) both times. The majority of the assembly line workers had repetitive stress injuries, including losing fine motor skills and wrist control due to use of impact wrenches and exposure to bad ergonomics. If a human delaminates, you can't glue them back.

Although rational (Taylorism, Management by Objectives, MBO, OKR, Milestones, management by goals ) vs. normative (Lean / Toyoda, human-centric, Agile, humanist, culture ) division of management styles/rhetoric has been proposed, they should perhaps be seen through the prism of Creative Destriction (not disruption, and surprisingly, CD comes from Marx).

In a vulgar approximation, imagine capital-backed entities served by the manager class, which are stressed by short deadlines and performance-tied bonuses and careers. They are constantly shifting basic approximations of solutions and perspectives, without challenging the system.

Give employees more autonomy, and output and managerial control suffer. Instill more rigid control, and risk losing the ability to innovate and "knowledge producing" (Drucker of MBO fame) talent such as programmers.

In an antagonistic system, whatever you come up with has its own roots of destruction on top of built-in conflict (Marx again). Hence the back-and-forth re-framing between "rational" (Taylorism) and normative "culture, baybe, Holocracy!" so fast that today it is a zombie that has both and neither.

You end up with projectification, corporate "culture" that demands the soul and meeting arbitrary "milestones" from employees, but the corporation itself can only guarantee arbitrary termination.

There is a great and influential analysis "Design and Devotion: Surges of Rational and Normative Ideologies of Control in Managerial Discourse" by Barley and Kunda. I would suggest reading "Management fashion" by Abrahamson as well. Both can be found through G.scholar, including no-registration copies.

https://www.jstor.org/stable/2393449 https://journals.aom.org/doi/abs/10.5465/amr.1996.9602161572

Re:Everyone is still terrible at creating software at scale by marcosdumay on Saturday April 10, 2021 @12:48PM (#26762840)
Sorry, but if all you got from Toyotism was Lean, than you got somewhere already fully geared into Taylorism. There is much more to it, and yeah, even one of the tenets you will get from Deming is that short deadlines and performance-tied bonus and careers can't work. He also goes a great length about empowering employees to change things like part sourcing, but yeah, not the final product.

(I don't know about worker health problems, and I imagine that poor as Japan was at the time, any empiric data on it would be useless. But, anyway, I don't think the management structure is the correct place to fix that one problem.)

And I'll have to break into your Marxist narrative, but all that was once incredibly successful in a fully capitalist framework. It never became non-competitive, it was only replaced by order of top-down command structures.

Re:Everyone is still terrible at creating software at scale by NotPavlovsDog on Saturday April 10, 2021 @01:01PM (#26762986)
I would appreciate an expansion on the "break into".

The approach I outline can explain and predict why the two Ts rotate, including at Toyota (Toyota going Taylor, or rather normative - rational).

I prefer not to base my experience on what Deming or anyone else wrote, but how their writing relates to real-life practice (though of course as a critic of Taylorism as represented by MBO and other Performance Evaluation rational schools, he is a welcome breath of fresh air).

Have you observed, preferably both as a participant and independent, Deming idealism in a functional real-life system? I have only seen a shift towards projectification and cultural hegemony.

Re:Everyone is still terrible at creating software at scale by marcosdumay on Saturday April 10, 2021 @01:49PM (#26763400)
What breaks the narrative is that the system is clearly not antagonistic. Giving the correct kind of autonomy to the workers increases every single item that management should care about, at least in a capitalist system. And both logic and evidence point that way.

> Have you observed, preferably both as a participant and independent, Deming idealism in a functional real-life system?

As a participant, never. As an independent, we are talking about the mentor of the movement that teached the Japanese management how to go from an industry that didn't produce anything good into the highly developed one that outcompeted everybody all ever the world.

Re:Everyone is still terrible at creating software at scale by NotPavlovsDog on Saturday April 10, 2021 @05:07PM (#26765117)
> What breaks the narrative is that the system is clearly not antagonistic

So why the shift to Taylorism at Toyota, the heart of the Toyota system? Why the shift to projectification from life-long employment at Japan? How can one explain chronic injury if workers get to plan and improve the workflow (or are wasted humans not considered waste)?

And the Toyoda principles at the heart of Toyota predate Deming. (Ohno, Taiichi (March 1988), Just-In-Time For Today and Tomorrow, Productivity Press, ISBN 978-0-915299-20-1 )

The list of paradoxes can continue indefinitely, or one can elect to look for the root cause.

Re:Everyone is still terrible at creating software at scale by dragonwriter on Saturday April 10, 2021 @02:26PM (#26763732)
> from Deming is that short deadlines and performance-tied bonus and careers can't work.

well, for the performance pay, Deming is more subtle: its not that it doesn’t work, its that the performance must be something meaningfully under the employees control for performance pay based on it to work (otherwise it can’t influence anything), and it must be a holistic measure of company value (otherwise you get perverse incentives around microoptimizations), and your measure of performance must take into account the degree of uncontrolled variation in the process (otherwise you give false negative and positive signals due to random variation rather than real problems/improvements).

Re:Everyone is still terrible at creating software at scale by yawz on Saturday April 10, 2021 @11:58AM (#26762416)
Pushing down the decision making is important for that reason. The “makers” know best most if not all of the time. The manager is there for support and for facilitating the learning. E.g. With decisions we make mistakes. As a manager it’s important to ensure that the team reviews past decisions and learns from them to improve the decision making process.
Re:Everyone is still terrible at creating software at scale by itsArtur on Saturday April 10, 2021 @09:56AM (#26761511)
> A lot of people who I knew as sympathetic and calm before they took management roles turned into something I could code in one minute: namely a program that asks "how much is this going to take?" and if your answer is above N hours/days then they say "no, we're not doing it". And that's not because they are stupid or suddenly non-sympathetic. It's because their bosses optimize for those metrics and measure them (and their salary, and their bonuses, and even their vacation days) by the same metrics.

I had this exact same thought recently when reflecting on my behavior in my new role as a "technical product owner". All of it was reflexive, as if I suddenly forgot all of the software engineering knowledge I accumulated over the years and became a deadline-driven cog.

I don't have a solution yet; I think it comes down to that I don't yet speak the same language that people I report to do, and thus I feel like I can't defend my position well enough. It comes with experience, I guess!

Re:Everyone is still terrible at creating software at scale by pdimitar on Saturday April 10, 2021 @10:03AM (#26761553)
Part of what makes you an enabler of programmers -- and not just a guy who screams "faster! harder!" -- is to be able to push back a little.

There's sadly an inherent conflict of interest between the people doing the job on the ground (programmers) and the rulers (managers).

Your job is to find the middle ground -- not to completely side with your higher-ups.

That would be a good starting point for you IMO.

Re:Everyone is still terrible at creating software at scale by bradleyjg on Saturday April 10, 2021 @10:31AM (#26761730)
I’ve been reading these Dilbert rants for twenty five years. I probably made some of them on slashdot.

We as an industry tried the all engineers thing and it didn’t work.

Have you considered the possibility that the boss’ bosses have a point? That the intelligent people you once respected have been convinced rather than intimidated?

Re:Everyone is still terrible at creating software at scale by tremon on Saturday April 10, 2021 @01:34PM (#26763282)
Have you considered the possibility that the boss’ bosses have a point?

It's the organization that shapes the individual, not the other way around. Changing a cog in an existing gear system only works if the cog is of a particular shape, and it will get chafed and dented if not sufficiently so. If the cog is too different, it will break and be replaced again.

Occam's razor says that if many different people fail within the same organizational structure, the problem is not with those people.

Re:Everyone is still terrible at creating software at scale by AlexCoventry on Monday April 12, 2021 @10:45PM (#26787609)
That's just begging the question, though, since the comment you replied to was essentially asking whether the promoted SEs are actually unthinking cogs, vs people who've been convinced by their new perspective.
Re:Everyone is still terrible at creating software at scale by tremon on Tuesday April 13, 2021 @09:01AM (#26791583)
But my point was that that's a false dichotomy: where is the option that they were thinking cogs that found themselves unable to change the shape of the gear system around them?
Re:Everyone is still terrible at creating software at scale by pdimitar on Saturday April 10, 2021 @10:56AM (#26761917)
I would be ashamed to call myself a thinking human if I never considered it.

And as said in other comments, for some companies and businesses they are right indeed.

But not in programming / IT in general. At least not where I worked (and where many of my former colleagues worked).

It's all optimizing for short-term and never the long-term. And I have seen people playing the long game gaining a lot so I also know I am not imagining it.

Re:Everyone is still terrible at creating software at scale by kansface on Saturday April 10, 2021 @11:47AM (#26762328)
Predictability in the short term has a definite business value in and of itself, regardless if it makes development slower - or perhaps, a value to management. Certainly the degree matters.
Re:Everyone is still terrible at creating software at scale by pdimitar on Saturday April 10, 2021 @03:01PM (#26764062)
You know, I am actually VERY okay with short-term thinking but not when it's 100% of all decision making.

Make it 75% short-term objective and give the other 25% to longer-term planning and/or simply playing with ideas and I'll be very happy to work in such a place.

I am not a fanatic -- and most programmers are actually quite understanding (if a bit too timid sometimes) people.

We just want the pendulum to not be on one of the sides 100% of the time. That's it really.

Re:Everyone is still terrible at creating software at scale by didibus on Saturday April 10, 2021 @01:39PM (#26763328)
> We as an industry tried the all engineers thing and it didn’t work

Example of when this was tried?

Re:Everyone is still terrible at creating software at scale by bradleyjg on Saturday April 10, 2021 @02:19PM (#26763671)
Re:Everyone is still terrible at creating software at scale by BurningFrog on Saturday April 10, 2021 @10:20AM (#26761657)
I think OPs point applies to organizational management as much as programming:

> if a solution exists today, and it has yet to be replaced, it's at least possible that it's over all the best solution, and the reason you're not seeing that is because of a lack of understanding.

Re:Everyone is still terrible at creating software at scale by jakub_g on Saturday April 10, 2021 @03:05PM (#26764091)
This. It's also really difficult to build a mastery of things when you're constantly moving from one piece of code to another according to shifting priorities.

And once you build mastery of $fwk/$lib/$tool, it's deprecated and you migrate to another one. GOTO START

Re:Everyone is still terrible at creating software at scale by zmmmmm on Saturday April 10, 2021 @07:43PM (#26766345)
> It's also really difficult to build a mastery of things when you're constantly moving from one piece of code to another

Conversely, it's surprisingly difficult to get engineers to master things they aren't intrinsically interested in. We've had a stable tech stack and core architecture for 4+ years and one of my major challenges is to stop employees constantly trying to introduce $new/$shiny or major refactor X because it wasn't built how they personally would have done it and get them to just learn and understand the stack we already have - which addresses all our needs at most 5% less optimally than $new/$shiny.

Re:Everyone is still terrible at creating software at scale by danieltillett on Sunday April 11, 2021 @02:25AM (#26767927)
When someone’s future salary is tied to acquiring $new/$shiny skill then you are going to get this sort of rational behaviour from the engineers. Smart people respond to incentives.
Re:Everyone is still terrible at creating software at scale by pdimitar on Sunday April 11, 2021 @07:05AM (#26769406)
This is very logical because engineers know they can be fired any minute with zero recourse. Inevitably, they want to invest part of their current work time into being able to to apply for the next job.

If people have a good social safety net and a good job security, the depth of their skills improves by a lot.

Re:Everyone is still terrible at creating software at scale by zmmmmm on Sunday April 11, 2021 @06:16PM (#26773980)
You make a really good point about the relationship between job security and motivation and commitment. In my case we actually do employ people under medium to long term contracts, so this is less of a factor. But I think the phenonemon you mention is widespread and endemic in the IT industry which definitely contributes to the technology fetishism and resume building type dysfunctional behaviours that plague us.
Re:Everyone is still terrible at creating software at scale by pdimitar on Saturday April 10, 2021 @03:13PM (#26764169)
Sadly yes. I really want to... stand still for a bit and learn things properly, you know?
Re:Everyone is still terrible at creating software at scale by analog31 on Saturday April 10, 2021 @10:19AM (#26761650)
If the manager is a former engineer, they might just be following a rule of thumb, that bigger tasks are impossible to manage, and above a certain size threshold, carry an existential risk for the business.
Re:Everyone is still terrible at creating software at scale by cascom on Saturday April 10, 2021 @06:26PM (#26765775)
I’m not sure I’d call the incentives perverted (only perverted in the sense that “perfect” software is not the goal).

Management and owners are a far from perfect group that make plenty of bad strategic decisions for a host of reasons, but they also have to deal with the real/perceived constraints of time, budget, market, competition, etc. that are easy to not appreciate when you are focused on the design/build portion of the business.