I really mean nothing patronizing here, but I suspect OP does not have the corporate experience to handle this situation. This is a corporate equivalent of a double-black diamond downhill route. OP was hired by people who have little understanding of tech and already came in with guns blazing. I might almost wonder if OP's a sacrificial lamb.
But, the tech advice of not doing a rewrite, making tests, soothing any hurt feelings, creating local instances will help. Make the everyday coding experiences of the tech team nicer. Source control, local instances, and unit/integration/E2E tests are a gimme.
The old rule of thumb applies: pick only 2 of speed, cost or quality. You cannot have 3.
I agree with this. OP doesn't say, but reading between the lines, corporate at best doesn't understand the ramifications, but corporate doesn't care about the ramifications.
They're getting 20 million in revenue from 3 cheap devs. Things are going great, according to corporate. They're not going to learn, and OP is going to get blamed when things can't get done.
I just quit because I was placed in a similar situation. The CEO, who does have a CS background albeit ancient, insisted there was nothing wrong with the tech stack that couldn't be solved by vertically scaling and then horizontally scaling. We were at the limits of the former and the architecture made many important parts impossible for the later, but that's another discussion.
The problem wasn't tech scaling, it was process scaling. We really couldn't divide work easily because there were often conflicts. People would join, see the horrible code, then leave. We specifically had to hire off-shore junior devs who didn't know any better and snowball them. I felt the last part was unethical and didn't want to be part of it any longer.
OP is not doing any favors for themselves, and especially not for the junior devs on the team. This job is going to set back the career for the junior devs. They're wasting their time on ancient methods and technologies.
Could you define this phrase and what English dialect it's from?
I assume American English.
Prior to the sexual slang made popular by the movie Clerks, snowballing in the context I used basically means to blindside or con someone.
One definition on Urban Dictionary:
"A situation where a criminal has found themselves in possession of an easy target and proceeds to rob them and leave them mortally wounded for fun, a synonym for getting iced."
There's also snowballing meaning a problem getting bigger and bigger when unaddressed. I'm probably using it in an older, not-exactly-mainstream way.
Blindsiding someone means taking them unawares - hitting them when they aren't looking, physically or metaphorically.
A speedball has cocaine in it, which is sometimes known as snow.
"Iced" can mean killed, but the Urban Dictionary definition is oddly specific
I agree about "snowballing" meaning increasing in size or momentum, but it doesn't have to refer to a problem.
Even before that, s/he doesn't even seem to understand any measure of business. $20m/year with 3 people is BIG. Any disturbance to whatever makes that happen will hurt the business greatly. When a full rewrite shakes the boat and causes a few millions of revenue loss or the loss of potential market share or opportunities, they will rightly fire him.
Best route would be to improve things without hurting anything. It doesn't matter things are 'properly done'. It matters whether the organization survives and prospers. What's 'proper' is redefined every 3-5 years, most of the time without objective basis anyway. So no need to hurt the business to force the current software paradigm.
From what is told in the summary, it seems like the software stack and these 3 people constitute the core of the business that is going on. The business may be something totally different. But it seems to be running on that software.
> And... $20m is revenue
Even if they have lower margins than what you yourself imagined, $20 m revenue/year is still a gigantic amount. You can improve upon whetever margin or inefficiency in the process and increase the net profit - optimize vendors, costs, sales pipeline.
The difficulty in modern Internet business is getting to some place at which you can command $20 m revenue from any market. Finding the customers and the users. Not the margins.
It’s not difficult getting to a high revenue fueled by aggressive and expensive acquisition using money from investors who gets dazzled by growth numbers, but if your customer lifetime value is low and you’re not a pure SAAS business which means margins won’t automatically improve with scale, turning that company profitable can prove very difficult.
Edit: Typos. I suffer severe typing legasteny more often than not...
Otherwise, walk away immediately.
That is how it should be done in any case anyway. Improvements should get slowly rolled out without disturbing the users and the business.
Not exactly. IT management should be always telling people stuff like "did you notice that the integration with XYZ that never worked well stopped failing?" or "did you notice that we delivered those few last features at record time?" and explaining why.
My advice would be to listen to the developers, to understand them and the business. To understand what they really need. What a viable path forward would be. A complete rewrite, a second system for new developments, many more developers or something. Or maybe it is the optimum solution right now because the whole company is so messy and your job is not to change the company structure. Then you maybe you could support them by slowly enhancing their skill set and accept what you can't change. Doesn't sound like fun? Then leave soon, staying won't do you any good.
> This business unit has a pretty aggressive roadmap as management and HQ has no real understanding of these blockers. And post COVID, budget is really tight.
I've been at a company not unlike this... several MLOC of VB WinForms that was total spaghetti, but a highly successful app that brought in a lot of revenue. In our case the majority of the dev team was in agreement that the situation wasn't sustainable, and at first (meaning when I joined the company) we had engineering leadership who mostly agreed. They brought in several rounds of consultants to evaluate the code base and announced plans for a major modernization effort. But the consultants largely agreed with the dev team that the code was in such bad shape that it effectively needed a rewrite. At one point we did a prioritization and t-shirt sizing exercise and came up with _30 man years_ worth of items in the "critical tech debt" bucket. Apparently engineering leadership was not aligned with the C level suite about how much money they were willing to spend on this thing, because within the next year there was 100% turnover of management in the engineering org. A couple of (talented!) people who had been hired for the modernization effort left first, then the other engineers who knew the code base well followed. Last I heard the company was limping along relying on Eastern European and Indian contractors to keep the lights on.
In short OP, you can probably get some wins; maybe some major process improvements like using source control, maybe more minor things like introducing patterns or frameworks so that net new code isn't a total mess. But there is zero chance that you're to do anything like a rewrite or major refactoring without leadership understanding that they're in an unsustainable situation and a willingness from them to invest time and money to fix it.
The CEO/Founder wouldn't even consider a refactoring or cleanup session, let alone a full or partial rewrite. Only features drive sales, and we've sold so many things we don't have, so all he wanted was feature cram. Every so often, a VIP customer complained about some major use case that simply didn't work, so only in those cases was bug fixing permitted. And in those cases, the VIP customer would get a custom bespoke build made with the bug fixed. I was hired because the last person of my seniority could not cram features in fast enough and gave up in disgust.
They only got source control because I came in on a weekend, unpaid, to do it. I lasted a little over a year. Bootstrapped founder (and sole shareholder) eventually sold the company for ~$150M. Sometimes it seems there is no justice in the world :)
Don't get me wrong, if it works it works, but the question is for how long and who will suffer when it doesn't?
Also from a business perspective: If I were the CEO of that company I'd probably like to know that there is something built on sand and a huge technological dept. It is a cashcow now, but I'd like to ensure it still can be one in the future. And for this some level of maintenance would be expected.
Same thing for reliabilty. If as a CEO I knew the entire functioning of the thing that brings the cash hinges on one developer remembering whether index_john-final.php or index_peter-final-final.php is the latest code I would probably have a hard time sleeping.
That means the minimum OP should do is explain the situation neutrally and your point of view is certainly something he should weave into this. In the end the higher ups need to know this, and what they decide to do is their thing, but OP needs to make them aware of why this could potentially endanger the service in the future. If they then decide to take that risk — so be it.
Some developpers seem to think that their jobs is to engineer nice and beautiful systems. It's not.
As a developper, you're getting paid (fyi the minimum so you don't leave the company) in order to maximise total shareholders' returns. That's it.
The business doesn't care if the codebase is garbage, with massive technical debt, nor if you struggle working with it. That's literally not even a problem as long as it is concerned.
I was more reacting to OPs attitude, where he asks how to fix everything without even being clear if he is managing the team or officially in charge of the product.
I may be mistaken but the impression I have is that the management is perfectly happy with a bad system and expects the engineers to simply go along. This is then a management decision. It may blow up later and sink the business (or not, it can just hold for the lifetime of the product).
As far as OP is concerned, this situation would mean that he should probably leave the ship now. (As staying would probably result in 1. No marketable skill development (being stuck with a garbage codebase) 2. Burn-out has he takes on the Herculean task of cleaning it up. 3. Be blamed if something bad happens.
What I care about is the code quality because good code makes my job easier. I inherited the code, not the business.
It is possible that you’re already doing what you’re getting paid for. In that case you shouldn’t go out of your way to increase the company’s profits (and nobody expects you to).
(In my original post I didn’t say that as a developer you should code thinking of quarterly results, I just stated the obvious that you are employed for the shareholders to get money back)
The physicist said that God must be a physicist, because He had to know about matter and energy and so on.
The engineer said that God must be an engineer, because He had to do something useful with the matter and energy - turning chaos into order.
And the business guy asked, "Where do you think all that chaos was coming from?"
1. You focus on the advice here, adding tests (which IMHO is actually pretty difficult on a legacy poorly architected/documented system), source control, refactoring, etc.... with any time remaining in the schedule devoted to adding features.
2. Your doppelgänger chugs along and releases several more features than you do, giving them a market edge.
What happens next I suppose depends upon what difference those extra features make. If the delta is small, you may be able to pull out ahead, in the long run. But if it's large, then your company may start to lose revenue / customers. Then, the screws will likely tighten even harder, and you'll be forced to sideline refactoring efforts and double down on delivering features. And then those refactoring/cleanup efforts will bit rot and you'll find yourself back roughly where you started, except now you're behind your competitor.
There's a quote I can't seem to find atm that summarizes this. It was something along the lines of "With php our product is already up and gaining market share meanwhile our competitor is still futzing with their rails configurations." (If you know the actual quote I'd love to see it.))
This doesn't indicate how profitability, and seems to ignore that management/owners might have had something to do with it. A well-connected industry player is better poised to start/found/build/grow a company to that level than someone with 0 experience.
And... yes... quitting to something which matches the OPs expectations will likely be better all around than trying to 'fix' something people aren't asking to be fixed (it seems).
Even at FAANG salaries this wouldn't be that much compared with $20M
E.g. maybe it's an e-wholesaler or widget reseller, bought $19M goods and sold $20M. Or maybe it was much slower than expected, they actually bought $25M goods and are burning 500k/month on warehousing. Or whatever.
> My mistake, they didn't lay a golden goose--they built a money printer. The ROI here is insane.
We can not make that conclusion. Presumably the business still needs salespeople, support, etc.
sounds ok?
It's likely a losing up hill battle.
If the mess generated $20m last year and it's projected to generate $20m next year, that's a problem.
If the second case is true, I believe it's somewhat the responsibility of the OP to sell solving this long-term problem to the _rest of_ business. If they hired him as an expert in that area, they should listen to him.
If that fails, leave.
There is some consensus that you can't fix stupid.
Even if you are very talented at repairing airplane engines in flight.
Their pursuit of profits above all else have likely gotten people killed. They represent a clear and present danger to US National Security.
You don't use it because you haven't heard of it; = not competent.
I'm sure it happens. And then you get the next job with 5y PHP experience or whatever, employer doesn't mind no formal training (not that I'm saying they should in general - but if they're non-technical hiring someone to 'do it', or first hire to build the team or whatever, then they probably should as a reasonable proxy!), rinse and repeat.
> I'm sure it happens
Yeah it does happen. While using the Internet, quite frequently, you are looking at such products developed by such teams, making millions of dollars a year. Even as the good engineering that is being done at FAANG is now being questioned over profitability, with even Google talking about 'inefficiency'.
I think many people here are reacting to $20M forgetting not everything's a SaaS/in the business of selling software (but mostly still has some (in-house) software somewhere).
The shitty software is what sells the product, from the description. Even if the shitty software is a sales/inventory management tool or 'whatever', from the description it is obvious that it is vital to whatever business they are doing.
It doesn't matter whether it was built with Microsoft Access and Excel files. If its contributing a major part of that $20m /year, its not shitty, its golden.
Anyone who understands the trials of modern business, including any tech lead who had to deal with even merely stakeholders and low-level business decisions would prefer to have a $20 m/year sh*t before a well-crafted, 'properly built' architecture. The difficult thing is getting to that $20 m/year. The difficulty of rearchitecting or maintaining things pale in comparison to that.
> I think many people here are reacting to $20M forgetting not everything's a SaaS/in the business of selling software (but mostly still has some (in-house) software somewhere).
Everyone is aware of that. Many are also aware that getting to $20m/year in WHATEVER form is more difficult than architecting a 'great' stack & infra.
My point about Access (or Excel or whatever as you say) was that that would be the very early days of something starting to happen in-house, that wouldn't even be the hypothetical 'script kiddies'.
Nope. Not really. Your average SV startup idea in which the end users will do some simple, but catchy things with your app - yeah, go all no-code if you want to get it started.
But, in real business, in which there are inventories, sales, vendors, shipping companies, deliveries, contracts, quotas, FIFO and LIFO queues and all kinds of weird stuff, things don't work that way. You may end up having to code something specific in order to be able to work with just one vendor or a big customer even. They may even be using Excel. You do it without blinking because millions of dollars of ongoing revenue depend on such stuff.
From a business perspective, nothing is broken. In fact, they laid a golden goose.
> team is 3 people, quite junior. One backend, one front, one iOS/android. Resistance to change is huge.
My mistake, they didn't lay a golden goose--they built a money printer. The ROI here is insane.
> productivity is abysmal which is understandable. The mess is just too huge to be able to build anything.
But you just told me they built a $20M revenue product with 3 bozos. That sounds unbelievably productive.
> This business unit has a pretty aggressive roadmap as management and HQ has no real understanding of these blockers
You should consider quitting your job.
As far as the business is concerned, there are no problems... because well... they have a money printer, and your team seems not to care enough to advocate for change. Business people don't give a damn about code quality. They give a damn about value. If 2003 style PHP code does that, so be it. Forget a rewrite, why waste time and effort doing simple refactoring? To them, even that has negative financial value.
From their perspective, you're not being paid to make code easy to work with, you're being paid to ship product in a rats nest. Maybe you could make a business case for why its valuable to use source control, dependency management, a framework, routing outside of nginx, and so on... but it doesn't sound like any of that mattered on the road to $20M a year, so it will be very difficult to convince them otherwise especially if your teammates resist.
This, again, is why you should consider leaving.
Some developers don't mind spaghetti, cowboy coding. You do. Don't subject yourself to a work environment and work style that's incompatible with you, especially when your teammates don't care either. I guarantee you will hate your job.