Yep and that's what I've seen be successful: someone who really knows the existing code inside and out, warts and all, needs to be a key leader for the part being broken out into a separate system. The hard part isn't building the new system in these projects, it's the integration into the existing system, which always requires a ton of refactoring work.
Indeed, the developer was one of the best programmers I've known and absolutely the key person on the system. The New Guys Clique were the sort of developers, you might know some, who come in, look at the existing systems, decide it's all wrong and terrible, and set out to Do It Right.
His proposed architecture wasn't without elegance, but it was also more complex and, more importantly, it didn't solve any problems that we actually had. So in the end it was more of an ideological thing.
He seemed to take it personally that we didn't take him up on his proposal, and he left a few months later (and went back to his previous employer, though to a different group). Don't think he was around for even a year. He wasn't young either; he was pretty experienced.
This would mean that when rolling out a new version, usually only the shared library would need to be replaced. So.. significantly more complexity, due to now having a more arms-length interface between the executable and the business logic in the shared libraries. And the payoff was.. when deploying, we would upgrade a shared library instead of an executable? Still doesn't seem worthwhile.
You can do successful rewrites but your rewrite has to be usable in production within like a month.
If you don’t know how to achieve that, don’t even try.
The quiet developer was able to get their own rewrite done because they understood that.
Looks like the director of engineering showed some classic inexperience. You can tell when someone has done something before and when it’s their first time.
And it was never constrained to rewriting the existing system. The rewrite plan was motivated by the entirely reasonable desire to make further improvements possible, an additional mistake was the attempt to add major improvements as part of the rewrite. The new guys made their disdain for the existing system obvious, to the extent that their intent for the rewrite ballooned into a ground-up rebuild of everything.
Things You Should Never Do, Part I: https://www.joelonsoftware.com/2000/04/06/things-you-should-...
I strongly disagree with this, and it reminds me of one of the worse Agile memes: "With every commit, the product must be production-ready.". [0]
The rewrite has to be generally not behind schedule. Whatever that schedule is is up to the folks doing the work and the managers who approve doing the work.
[0] I've worked for an Agile shop for a long time, so please don't tell me that I'm "Doing Agile Wrong" or that I misunderstand Agile. "No True Scotsman" conversations about Agile are pretty boring and pointless, given Agile's nebulous definition.
But there's no way to really describe it. It's like explaining to somehow how to parallel park or do a kickflip... you can only explain it so much.
I like to say "it should be usable in production soon" because it's generally a good approximation that takes into account what you might have to work with. It's an upgrade from advice like Joel's who just say "IT NEVER WORKS"
What you originally said ("It must be usable in production within a month") is equivalent to "Just don't do it, because IT NEVER WORKS" for all but the smallest, simplest projects out there in Professional Programmer land. [0]
> But there's no way to really describe it.
There really is a way to describe it:
"The rewrite has to be generally not behind schedule. Whatever that schedule is is up to the folks doing the work and the managers who approve doing the work."
Establishing that schedule is the same sort of cost/benefit and expected-level-of-difficulty analysis that should be done before planning any nontrivial work in Professional Programmer land. All but the most green or most sheltered-from-Process programmers are at least aware of this analysis. Many of those who are aware of it have participated in it.
[0] Or the most well-designed projects, which have small, easily understood pieces with easily-comprehensible interactions with the rest of the system, that can be quickly and easily replaced with new pieces. There's not much out there like that... and I'd imagine the task of "making major changes to how many of those pieces interact with each other" wouldn't be usable in production in a month for most of those systems.
But, man, sometimes software is fit-for-purpose and can really be just be left alone for extended periods. Other times, the users of that software upgrade on a hemi-annual or annual schedule (or even LESS frequently), so they'd never notice a three month delay in new releases.
I suppose if the software maker is a functional monopoly in that space and the customers are cursed withe vendor lock-in, sure. I can think of Major Software companies that barely maintain their products yet companies stick with the garbage because what choice do they have?
Given this statement, it might surprise you to learn that there are folks who aren't keen on software development for its own sake, and really rather dislike having to re-learn how to use their tools when they those tools get significantly change with no clear benefit to those users.
See also: the folks who are perfectly happily using "ancient" versions of -say- Photoshop, or WordPerfect, or Word or...
Rewrites from scratch never work with sufficiently large systems, and anyone that’s been involved with these things should be savvy enough to recognize this. The only question is around the exact definition of sufficiently large for a given context.
One of the benefits is that you get to compare the new vs old implementation quickly and easily, and it lets you raise questions about the old implementation; every time I've done this I've found real, production-impacting bugs because the old system was exhibiting behaviors that didn't match the new system, and it turned out they weren't intentional!
[0] http://sevangelatos.com/john-carmack-on-parallel-implementat...
I once came into an old codebase like this as a junior, thinking I can start again. And I was gently but firmly told by my boss that it wouldn't work, this software is crucial to operations that support all our revenue, and while it can be improved it has to keep working. And we improved the hell out of it.
If it’s a big new team that doesn’t know what they’re doing, working separately from existing codebase, with lots of meetings… I see no reason why it would finish at all.
The devs were good developers, too! Two people on the team went off to Google after, so it's not like this was due to total incompetence or anything; more just overambition and lack of familiarity with working on legacy code.
Bluntly, yes. And so is every other reply to you that says "no this isn't naive", or "there's no reason this project shouldn't have finished". All that means is that you've not seen a truly "enterprise" codebase that may be bringing in tons of business value, but whose internals are a true human centipede of bad practices and organic tendrils of doing things the wrong way.
Currently there. On one hand: lot of old code which looks horrible (the "just put comments there in case we need it later" pattern is everywhere). Hidden scripts and ETL tasks on forgotten servers, "API" (or more often files sent to some FTP) used by one or two clients but it's been working for more than a decade so no changing that. On the other: it feels like doing archeology, learning why things are how they are (politics, priority changes over the years). And when you finally ship something helping the business with an easier to use UI you know the effort was not for nothing.
Everything takes longer than you think and this sounds like it involves at least 2 teams (the php team and the scalar team). Every team you include increases time line factorially in the best case.
It takes a lot of time to have meetings with a dozen managers to argue over priority and whatever. Especially since their schedules are full of other arguments already
LOL why is this two teams!?
Yes.
This resonates. At one former company, there was a clear divide between the people working on the "legacy monolith" in PHP and the "scalable microservices" in Scala/Go. One new Scala team was tasked with extracting permissions management from the monolith into a separate service. Was estimated to take 6-9 months. 18 months later, project was canned without delivering anything. The team was starting from scratch and had no experience working with the current monolith permissions model and could not get it successfully integrated. Every time an integration was attempted they found a new edge case that was totally incompatible with the nice, "clean" model they had created with the new service.