This is a lot different from than "no on besides them can have any insight on timelines, or budget, on which features are or aren't important.". I definitely value the input of other professionals when it comes to their field of expertise. All I want is the same in return.
But the way the conversation should go isn't the developer deciding when refactoring should happen. It's the developer giving an analysis of the cost and benefit of a refactoring (it will take X time, but will save Y work in the future). And the manager factoring that into all the other circumstances, and deciding whether it's worth the current cost.
This has been the ideal theory for decades and I really wish people finally stopped to think for two minutes, look around, read some history and finally understand that it's not working.
Is it really so hard to accept this reality? I am periodically mind-boggled by the amazingly resilient illusion that you can plan things as tidily. It never happened, not once, in my 20 years of career.
Estimating these costs upfront is more or less impossible. It's usually a can of worms that opens other brand new and exciting sub-tasks, the same kind that gives managers nightmares.
IMO you might have projected a little bit and arguing against some imaginary "us the programmers are better than everyone else". Meh. Only kiddos think and say things like that. By and large, almost all experienced programmers I've ever met are quite moderate and humble.
Problem is, they are too moderate and humble which leads to predatory businessmen micro-managing them to oblivion. I've personally witnessed high-profile programmers firmly saying "No!", putting their foot down and making a thing their way with more time budget. Their project and code was used, almost untouched, for 10+ years afterwards -- it was that good.
Many of us are expert artisans. You don't go to a professional blacksmith giving him wisdom how can he accelerate a certain process.
I recently made the mistake of deferring to the anxieties and fears of non-developers for so long that I almost forgot how we got to where we are today or what my true capabilities are.
You know exactly what those assholes are thinking when you say the word "refactor" or "iterate". It starts to become a psychological horror experience if you let it get out of control for too long.
Now don't get me wrong. At the end of the day I love and care for our customer too. I just don't let their minute-by-minute feature request stream live rent free in my head the same way the rest of the team does.
Ouch, dude, you didn't have to stab me in the heart. But yeah, 100% same. I am just now, at the age of almost 42, starting to finally crawl out of that. I am ashamed that it took me most of my life and career but oh well.
> I think the key point for me is that once you control for all common human flaws, the developer is still far better suited to make an executive decision regarding a technology concern than anyone else on the team.
True, but then you get blamed for "not understanding business realities". I got super tired of being hinted that I "don't get it" or whatever, especially having in mind that I launched products whose launch date I was not consulted about, repeatedly, and still get treated like dirt. I started pushing back on micro-management, fiercely and sometimes rudely.
You know what's the funniest part? It works. People start looking at you very differently and suddenly negotiations are an option. I guess it was possible all along and the business people are like "push him, he's a tech nerd, he'll fold".
We should start getting aware how are others viewing us and integrate this knowledge into our negotiation strategy.
I don't think either devlopers or managers can estimate future savings in most cases, but I still think it's necessary to refactor just to not drown in complexity and slow down overall development speed. My approach is to reserve about 20% for refactoring and technical improvements and let the team decide internally what to use it on.
Enough to get useful stuff done, small enough to keep most capacity for feature development. Also depends on the amount of technical tickets deemed relevant by the team
I think I just fundamentally disagree that this is the managers job. He tells me what needs doing, and I get that done as soon as possible within a given block of time (mostly sprints, so two weeks). They should be able to trust me to refactor as necessary within that block.
I think the more interesting scenario is large tech debts that take real time, like the change itself is weeks, or months. These need to be justified and planned and coordinated with managers.
In terms of refactoring there is no way to give a Cost Benefit analysis as that's too subjective. This should be a maintenance cost that's factored into the project at a high level. So you would say there a X days per year (normally about 20%) of the working days of the year spent on maintenance. Company I'm at has this and if you're doing this type of work then you have timecodes to book your time against.
That includes refactoring, re-architecture, tech debt, updating deps ...etc. PM's can't override this for their own project i.e. they have to allow for 20% off project activities in there Project Plan. Devs shouldn't spend more that 20% of their time doing these activities over the FY.
Example here would be in the UK we have something called a MOT which you have to legally get it's like a minimal maintenance you have to get on your car. Most people spend a little more though than the minimum.
Ideally, yes, though in practice this usually breaks in various ways, especially when problems occur. E.g. the President of a country probably doesn't need to be a firefighter, but if there's suddenly a massive fire, I assume he'd need to be briefed in far more detail to help call shots around the type of response.
> In terms of refactoring there is no way to give a Cost Benefit analysis as that's too subjective.
I mean, you probably won't get a very correct Cost Benefit analysis, but you have to do something. The rest of your post is you describing that something - if you elect to spend 20% of your time on maintenance, that's effectively giving it a certain amount of weight compared to other things you can be doing with your time. That's certainly one way to work.
Personally, I think a more flexible approach is better. E.g. new startup-ish projects should usually spend a lot less time refactoring, for multiple reasons, whereas more established long-term projects should spend much closer to 20%, or maybe more.
I feel like this is something a lot of presidents (or VP’s, or Directors, etc.) think, but really, instead of taking time away from the professionals doing the fighting to explain to them the minutiae of blowback, the best thing they can do is say “I trust you, put it out.” And then leave the pros to do their job.
In other cases, it is more important to get things right and continual refactoring is the fastest way to find an acceptable architecture to a novel problem.
In either approach, when speed is of essence, a limited scope helps avoid either producing insurmountable technical debt, or an unterminating research rabbit hole, respectively.
In the case I gave it's hours book on the maintenance project that is the test but you could very well have other measures and I've seen these in many projects. Things like linters, dependency bots, unit tests, e2e tests, code coverage, Cyclomatic complexity ...etc.
You work in a place where this happens. You could not.
Where does it not happen?
You could, if lucky, but such sane places are becoming more and more rare.
In the 90s (and probably earlier but I don't know) developers were viewed much more highly than now in all aspects except salary. There was a lot of autonomy and authority, the role was respected as a professional. With the rise of agile in the last ~15 years it has been transformed into a low-authority replaceable-cogs type of job, to be micromanaged by everyone else involved in the product. Salaries have gone through the roof though, so I guess that's the tradeoff.
Maybe there are engineering-oriented companies still with the 90s culture?
(If anyone knows of any such companies, please share, I'll head over to interview there.)
My only counter however, is that this often is the case, and the manager is right that the very overpaid engineer essentially sucks at self-management.
Or maybe management as a whole has simply realised that they need to gaslight engineers into thinking their required, or maybe a generation of engineers have been promoted into a role they're unsuitable for. All possibilities, all likely.
you follow guidelines and rules for the most part, if you performed surgery before a patient had scans same day, youd be reprimanded by peers. that seems analogous to "don't do major refactor on release day".
In the software industry there is a lingering stereotype of the eccentric genius who is really good with computers, and really bad at everything else in life, and their computer skills make up for lacking a lot of basic skills of a functioning adult, such as lack of judgement and perspective.
It's expected that software developers are odd and lopsided people who need someone else to babysit them and provide judgement and direction, and by now this has become the norm in the industry and is an established culture in the workplaces, so it also gets perpetuated and it's really hard to even be a well rounded software developer.
I've seen this too. It added tons of overhead before it was rightly cancelled.
> I’ve seen millions and millions of euros get wasted on things that don’t matter.
To be fair, I've also seen this from the business side. A great number of ill thought out unsuccessful projects, that are dispiriting to work on, as you know from the beginning that they will not be a success for concrete reasons, e.g. using an immature piece of software that doesn't support the features required.
Speaking as a lawyer, and as a programmer: your statement is utter falsehood.
In many countries lawyers are not 'managed' by other lawyers. In fact, most non-lawyers have no say in how lawyers practice, and are there to assist only in clerical/administrative matters. In some jurisdictions, non-lawyers are even barred from taking any cut in the outcome of matters (e.g. possible champerty/barratry), and definitely cannot override expert opinions.
We even have a derogatory word for those who 'manage' lawyers: touts.
In a similar manner, the non-professionals who think they can 'project manage' professionals often over-evaluate their own ability at management; I've seen them make plans that have no basis in engineering reality or technical compliance/requirements especially when it comes to development timelines. Your defensive posture of non-programmers is very indicative of the kind of mentality that true professionals must guard against. Professionals must do a job well, and not just be told to rush out buggy, unsafe products by non-programmers the majority of whom cannot even program to save their lives.
Still, what about cases like a legal counsel for a firm? Surely in those cases, the manager of the lawyer is the CEO, who might not be a lawyer himself?
As to your reference to the word 'legal counsel': a distinction must first be made.
The use of the word 'counsel' is often confused by non-lawyers. 'Counsel' is a specific term used to refer to lawyers appearing in court ('counsel of the courts', or 'court practitioners', or 'barristers').
Whereas 'legal counsel' is more specifically used in reference to in-house employees of companies, who handle regulatory compliance, general legal affairs of their companies, and so on. Such legal counsel may be legally trained/qualified, and may have had prior experience in legal practice. But they are not considered 'lawyers', in so far as they are not active/practising members of the bar at which they are practice, and in some instances are referred to as 'non-practising' lawyers. This is about the same distinction as e.g. a graduate of civil engineering, who has decided not to practice as an engineer and has gone on to some other field. Lawyering is however a very traditional profession, and the distinction between practising vs non-practising lawyers is very marked, and generally 'legal counsel' (as described above) are not treated as lawyers at all. It is not uncommon to find an in-house 'legal counsel' who has absolutely no idea what they're doing in practical/commercial reality, as they've never handled proper disputes or difficult contracts before except in non-contentious settings.
Thus in certain jurisdictions legal counsel are often prevented from performing any acts that must and can only properly be done by actual practising lawyers - e.g. 'legal counsel' cannot represent other companies/people/entities in courts (as they have no right of audience or locus to represent third parties), they cannot give legal advice as lawyers, etc. 'Legal counsel' as referred to in such context may be 'managed' by a CEO, but that's only because they are staff with legal knowledge, and not lawyers per se.
I tell them what i wanted achieved, and i pay. it gets done. If there's a delay, i expect the plumber to tell me promptly, and i re-evaluate the work.
Some programmers do get treated this way - consultants and such. But the majority is not - they are considered a cog in a machine.
But take plumbers working construction on a 30-story building. They're usually part of a team, and work together with dozens if not hundreds of other professionals. Do you think they don't have meetings where they get together with the other teams, discuss problems, progress etc?
I'm not defending stand-ups here, btw. Sometimes they make sense, sometimes they don't. But if your measure of "developers are treated badly" is "developers are asked to be in a 10-minute meeting to talk about their work progress", I think you're missing the forest for the trees in terms of what are and aren't good working conditions.
If you want to be treated as a payed consultant, become a payed consultant, with all the disadvantages and advantages which follow from that.
Does the law office receptionist get to tell a lawyer which pieces of evidence should be brought to trial, and which shouldn't? What are the parameters and limits around 'managing' attorneys? I worked in a couple law offices (office stuff and deliveries) and while there was some 'management', it was generally 'top down', and had to do with billable time and court deadlines. Coordination between lawyers to help each other or work on a specific case together happened, but even then the deadlines were generally based on external dates, and were generally not company-imposed ("gotta get all these items done by Q3 for the Q4 launch!")
If your workspace has shit smeared on the walls, you don't ask if you can clean it up. You just do it.
I highly disagree with this.
For some reason, many devs seem to think they are somehow better than everyone else, that they are the most important part of any team, that no on besides them can have any insight on timelines, or budget, on which features are or aren't important.
News flash - all of the above is none-sense. Developers are a profession that provides a specific skill, a highly valued one, but one that is a part of building things in most cases. There's nothing demeaning or wrong about devs having managers, whether or not they happen to also have been developers themselves. Some great software managers were never developers, and some great developers are horrible managers, just like in every other profession.
> It's nuts to me that a skilled profession - that not many can do - lets themselves get micro-managed like this.
Every skilled profession has people who are managed. Doctors, lawyers, they all have managers, and most of them interact with the public, who can override their expert opinions.