Years ago I left a rather awful company - I left on great terms with my boss, though I hated the company. I was the last person who knew how to release X product. While I documented everything, after I left, the wiki lost its mind, backups apparently weren't a thing, and the director of IT moved to Bermuda.
Fast forward 6 months.. a contractual obligation existed to release the software. They still hadn't successfully built (yup autoconf, make and friends), let alone know where to build it. I got the call you always dream about, 30 days prior to the release being due.
My boss asked me to do him a favour - I reminded him that I quit due to a lack of promotion and raises. He asked what I want, and I asked for 125K, which clearly he scoffed at. Then I reminded him that it was a 10M deal they'd signed, because we had an all hands. The CEO got involved and screamed at me for 'torpedoing the company'. I walked out and calmly informed them both along the way the price was now 250K, non negotiable.
I ignored every single mail (and legal letter) sent me way. With 10 days left, they agreed. I had the company deposit a cheque in escrow with lawyers of my choosing. We signed contracts, and I did my thing. Four days of work. It was well worth it.
Dougie, if you're reading this I hope your cringing.
I have bosses at previous companies that I'd help for free today because they were good to me and we parted on friendly terms. One boss helped me move my house once, and I'm not about to charge him to fix some random issue.
I have other bosses whom I'd require a cleared wire transfer of money before I'd lift a pinky to help them, because they were not nice people and I don't want to associate with them or assist them with anything. If someone in this camp approached me, I'd have a story just like yours afterward.
That an overworked employee who had already left the company would stick up for himself? Or that a company would go to such efforts to try to manipulate a former employee?
I realize my tone doesn't hide my feelings, but even if we disagree I'd still be curious to hear more.
Walking away from a toxic situation is not "psychopathic." It's common sense and if we all did it more then the psychopaths who expect us to stick around through it all would finally learn how to treat other people with respect. It's a job, not a marriage.
Sadly, I've seen this. Not first hand, but it happened to a friend and I have no reason to doubt his version of the story.
"We tried to hire back an employee to build something that we didn't train or try and figure out ourselves and a deadline forced us to pay a large lump sum to that employee".
No matter how you slice it, that's really shitty planning on the businesses side. They should have spent the time and money learning how to maintain their shit. By not doing it, they paid a huge price.
Every negotiation is about the best alternative the parties have, and it's generally harder in real life than in fantasy because you underestimate the alternatives to a deal.
Applying maximum leverage has a way of creating "out of the box" alternatives. And vindictiveness.
I have no need to judge it if it were true.
Luckily for him they decided it's easier to just pay the fee.
Because here's the thing, even with how weak US labor laws are, one thing they don't allow is for a company to come back and sue your for incompetence. So long as they didn't go out of their way to sabotage things, they are free and clear.
- if he can prove the threat, the company will be in an even worse position
- given the rather lacking precedent (as pointed out by the sister comment), there's a good chance the suit would be thrown out pre-court and therefore be rather cheap
- given that the OP seems to be a valued engineer, it is reasonable to assume that he has the f-you-money to fight this out
- even if won, the drawn out battle would take far too long to fix the software in time and, given that they then lost a 10 million contract, they lack the motivation (and maybe money) to continue
- lastly, even if they won after loosing the contract, they'd probably not be able to extract much in damage payments
So it's a rather empty threat.
Especially ex-employees who've been gone 6 months already.
Shouldn't be hard to find a no win no fee ambulance chaser to happily take them on and get costs awarded and damages as well...
I think in general all of my comments on this thread have been horribly misunderstood - I was just trying to say that if the company was scummy, instead of paying him the 250k fee, they probably could have sued him instead. I'm not saying that should have done, I'm not saying it would have been the right thing to do, that he deserved it.....literally nothing of the sort. Just pointing out that the company could have made an argument against him in court. As many other people pointed out to me, it would have almost certainly failed. But I don't understand how you can deduct where moral considerations are on my priority list from this?
This is exactly the free market at work regarding labor. Have a problem with the free market if you like, many of us do but do not accuse this poster of criminal action equivalent to a protection racket.
To clarify, the market would likely not allow this specific work to be sold at this price normally. But the buyer wasn't just buying specific work they were buying specific work to be completed by a specific date. The work in isolation does not have this value but the work completed by a certain day does. Companies use this rationale all the time charging more for expedited services.
It's especially common in construction and manufacturing where costs double or even triple when expedited delivery is required.
The way the story is told the price doubled after and because of the CEO screaming.
The market has nothing to do with that except the good old "The fair price is the one that god paid, uh.” I suppose.
I'd be inclined to believe that if there were some other providers but as it is told working with OP was the only solution. And OP's first offer was 125 000. Two screams later it's 250 000 ? It has nothing to do with economical theory or free market interpretation.
Second, you speak as if his claims were irrational. The way I see it, however, you would set for 125k (probably much less judging from what you write) while he, the sentimental fool, scooped up double the money. Therefore, your decisions, which you personally claim are rational, would have left you with half the money the "irrational" player would get. Seems to me, the "irrational" is actually the rational one, as he netted more profits. Economy and market are not math games in a vacuum, and players are not formless ideas.
The way I see it you are putting words in my mouth and I am definitely not interested in having a discussion like that.
I never insinuated OP was a sentimental fool or wrong to raise the price. I am calling bullshit on the free market and econ 101 as an explanation of why the price got doubled and call it like it is: it's payback from OP because they were dicks to him first, everything else is pedantry and useless justifications.
edit: if anything, OP could be considered a fool since he obviously gave an under the market quote as his first proposal. Had he started with 250K from the beginning he would have gotten more since there likely would have been screams anyway and he would have risen the price all the same.
And yes. I completely agree with your edit and said as much elsewhere. Their acceptance of 250K showed that the OP under-priced their services at 125K.
And yes. Free markets produce monopolies. Especially when time is important. As I said above, the critical good here was not the delivery but the delivery within a certain time.
And yelling, made the prise jump closer to market value.
I find this explanation (from top comment) more in line with how the world spins:
> If they left under bad terms because the business was a bunch of dicks, expect to pay 10x market rate. If this is truly "fix this or the business is out of business" - then it shouldn't be a tough decision to make.
So, “being a dick” has more to do with price raising than any free market justification (which I am okay with, just don't try to convince me it's the only thing at play. The offer would have been 250k from the beginning then. Or 500K to get 250K, whatever.).
Any small or independent contractor in any business will eventually have a blacklist, or a list of troublesome customers who get charged extra. Most of my contractors charge me less because they like me (and I pay on time). My HVAC contractor charges a particular customer double just because the customer is a rich entitled asshole; the contractor has decided that it's only worth dealing with the man's unpleasantness if there's more money to be made. Apparently the man has been blacklisted by other HVAC companies, so he might be in a tight spot, but whatever the reason, he ends up paying a lot more money for the same service.
This is a completely market-based mechanism: the troublesome customer has limited options (in this case, maybe only one option), and is thus obviously disadvantaged in negotiating price. Regardless of the myriad motivations involved in price discovery, it all leads to the same place: if the price wasn't worth the service the customer was to receive, they would not have paid it.
Honestly, this isn't mysterious or profound. It's economics 101.
Yes.
> It's economics 101.
Not even that. It's payback, that's all it is. There's no need to call for a more elaborate explanation for what was at play here. Not that I disagree with it. It's humane nature.
> My HVAC contractor charges a particular customer double just because the customer is a rich entitled asshole;
Like you say. No need to even know about market dynamics to (wisely) choose that option. Because the other guy is an asshole.
This is literally the definition of price discovery in a free market. If the price was not worth the service the company was to receive, they would not have paid it.
The fact that they did pay it clearly indicates that the first price was far below what the "market" would bear.
I think so too, however maybe better play safe I guess, and be content with 250? The negotiation partner seemed a bit irrational to me
Anything distorting those conditions - e.g. there's only one seller, only one buyer, a time constraint, a skill constraint, an information disadvantage, will distort the pricing as well. One trade isn't enough information for a market to exist or to settle on a fair price. I agree there that the market has nothing to do with it.
But I disagree with your implication of "the fair price is the one paid" is somehow unfair in an ethical sense. (I also think OP is in no way obliged to be "fair" in any ethical sense to the company, but that's another matter). The company got a $10M contract for $250k, versus losing the contract, is that unfair? If that really was the pivotal factor between losing the contract and not, the company ought to benefit by spending right up to the point where the contract becomes a money loser, to not lose it. Would it be unfair for the company to get an $8M contract by paying $2M? Or a $2M contract by paying $8M? Is OP obliged to help the company make as much profit on the contract as they can, by charging them less for his services?
This is non-sense. Those conditions specifically define a market and make it distinct from any other. To say a market with any of these constraints is "distorted" is to say all markets are distorted and render the term meaningless.
Also, I never said this one trade signified a market. My point was that this is a consequence of free markets. You can make any single trade "irrelevant" by choosing arbitrary criteria by which to separate it from its market. The company could have certainly hired another engineer in hopes they could complete the work in time while running the risk of blowing the deadline. But it was freely and fairly worth at least 250k to them to guarantee the job could be done correctly and on time.
This kind of thing is such a common part of the free market, that there is a whole aspect of contract law that pertains to ensuring companies don't get into this position. It's why contracting completed work is so important. Leaving things partly done leaves you vulnerable to others eating right up to the tiniest margin of your profit.
This is pure slander
The company can avoid this if they only care to maintain the documentation / wiki left behind by said person
When they failed to do even just basic proper operations, it's their own fault.
I'm a business owner and a consultant, and I too charge more for these kind of asshole clients, because of the extra stress.
But nowadays I just try to avoid them, it's just too damaging for your own sanity. Their kind of stupidity, self-entitlement, and ignorance is on a completely different level.
<pedantry>It's libel (written) not slander (oral); "defamation" is the generic noun. </pedantry>
It seems that paying to skywrite (you know, small plane making artificial clouds) could be slander but not libel
Citation? By that light, a DVD containing defamatory speech would be libelous, but that's generally not how courts see it.
From the (highly-influential) U.S. Court of Appeals for the Second Circuit, summarizing New York law (which is quite typical): "Defamation, consisting of the twin torts of libel and slander, is the invasion of the interest in a reputation and good name. Generally, spoken defamatory words are slander; written defamatory words are libel. Libel is a method of defamation expressed in writing or print." [0]
https://scholar.google.com/scholar_case?case=601727562995661...
Coding standards, documentation requirements, meetings and code reviews exist for a reason, too. It's not like companies are defenseless.
This is a gross failure of leadership at the company, and the former employee has every right to negotiate compensation for the inconvenience of having to work for them again.
This person left the company, and documented everything. What should they do, come in and save the day for $15/hr?
The company could have apologised, and do a lot more ego stroking before asking a favour which they can pay for and maybe throw in an all expenses paid weekend with the SO as a thank you. Not only will you get goodwill but future problems are going to be significantly cheaper to solve. Even if you have a bad break up - as a business owner you should understand the value of mending bridges. It's cheaper than losing sleep at night and losing your steak dinner to a bad stomach.
If your company were in the hands of incompetents, but still kept going in spite of the incompetents because it had gotten its foot wedged in the door or whatever, and then fell apart in ways you could only scorn, how would you view that sort of situation?
You may be doing any of that already, since I don't know anything about you except your one comment.
But the answer to your fear of exploitation is avoid making people hate you/your environment enough to want to protect themselves from it.
If you do not have the money to pa for someone's services you cannot complain. You say you have a company? Have you ever let a client get your products by paying less than their value or nothing at all? Work itself is a product too.
I feel for them but they failed to protect themselves.
He left detailed instructions how to release the software. The company managed to screw up so bad they lost the manual he made sure to write on their internal wiki.
And you identify with this band of miscreants? That's embarrassing.
That's probably what they deserve, if they come asking for a favor and then start getting abusive.
People like that would much rather spend $2.5M on courts and lawyers than let a worker walk home with 250k, or even 25k. Agreeing to a deal like this is an incredible minefield that may blow up on you even years later.
For a while, I made it game to light the various communications on fire in interesting ways, but it got boring.
Did they have a legal reason to think so?
The company could say: we’re going to pay you 125 and you could sue us and spend 60k and two years or just settle for that, after he’s already done the work.
The guy could demand the payment directly to his account and then not deliver, and the firm would be in the same situation.
By depositing the funds in escrow, they’re making sure that the money is there and in the hands of people whose only incentive is to give it to the correct party upon fulfillment of the contract.
The contractor knows that the money has been paid by the company and won't be withheld after the job is done.
The company knows that the contractor doesn't have the money until the job is done.
This is a political talking point that has been repeated so often that people believe it is true.
During the recession of 2008, there were stores on local TV news in many places showing unemployed Americans waiting in long lines for the opportunity do do those jobs that the politicians kept saying no American wanted to do for those wages. Meatpacking plants, especially, had more than enough Americans to choose from who were willing to work for low wages.
Because these things happened in "flyover" states, they were only very rarely shown on television on the coasts, and so the meme stuck.
A similar oft-repeated lie is that American farmers are all a bunch of Republican hillbillies who hate brown immigrants.
Around 2010, I attended a conference in Seattle for farmers in the Pacific Northwest. The main topic of discussion was how to increase legal immigration from Mexico so that they could have more farmworkers to pick the cherries and apples and mint and whatnot. These people were talking about goals of making people from Mexico and Latin America U.S. citizens in staggering numbers.
The more you actually go places and do things and meet people and talk to them, the more you realize that the words coming out of the political organizations on the coasts are mostly for their own benefit, and do not reflect reality.
Your second point about farmers trying to increase legal immigration in 2010 clearly indicates most Americans won't regularly do this work for the wages being offered. Why else would farmers be looking to hire immigrant workers just 2 years after locals were lining up to get these jobs?
While it would be nice if there was a way to get enough immigrants working legally to fill those jobs that doesn't seem to have happened. Short of another large economic downturn I don't see Americans working those jobs either. That leaves people who are working in the US illegally to fill the gap.
> Around 2010, I attended a conference in Seattle for farmers in the Pacific Northwest. The main topic of discussion was how to increase legal immigration from Mexico so that they could have more farmworkers to pick the cherries and apples and mint and whatnot. These people were talking about goals of making people from Mexico and Latin America U.S. citizens in staggering numbers.
Farmers are, for the most part, single issue voters: Whoever has the best plan to support agriculture and small businesses gets their vote.
How do you determine from TV that the people lining up for meatpacking jobs were not undocumented immigrants?
Like, it is generally accepted there aren't any in "flyover states" or something?
I never really thought of it until this moment, but you know how within 100 miles of any border, your constitutional rights are somewhat compromised so DHS can operate? That ought to mean a pretty substantial incentive for people who are undocumented to migrate inward. So I would bet they do.
That and Republicans have grown way more hostile to immigration since 2010, so it's an interesting anecdote but not sure how well it translates across the US, where polling shows farmers are overwhelmingly Republican who tend also tend to hate immigration legal or not.
When you say it's an "educated" guess, educated how? It only seems to reflect the usual bigotry that comes out of the coasts.
polling shows farmers are overwhelmingly Republican who tend also tend to hate immigration legal or not
If you're going to make a statement that sweeping, you really should back it up with credible sources.
I lived first 28 years of my life in the rural south and now live in the midwest.
My entire family and most of my friends growing up are the people you claim I'm bigoted against...
"If you're going to make a statement that sweeping, you really should back it up with credible sources."
The fact that farmers are generally far more conservative and republican than the median voter or average American isn't exactly some shocking new insight.
You seem to have never spent much time around these folks or have done much research on them if this comes as a surprise to you.
https://www.washingtonpost.com/news/wonk/wp/2015/06/03/why-y...
Are these just bigoted facts "out of the coasts" too:?
source?
The workers were detained, deported, and so on.
After several raids, not so much as a misdemeanor was levied against a single person, nor a fine to the company in general.
"There was false testimony and willful manipulation, and that makes this a shocking case of prosecutorial misconduct."
"An unprecedented letter, recently sent to Kevin Techau, U.S. attorney for the Northern District of Iowa, signed by 107 former high level Department of Justice officials, FBI directors, federal judges and prestigious law professors attests to the extraordinary prosecutorial misconduct which exacted the most possible punitive sentence for Rubashkin"
https://www.desmoinesregister.com/story/opinion/columnists/i...
Trump commuted Sholom's sentence in 2017, apparently.
I'd suggest a different phrase than "prominent Jews"; the family are "ultra-Orthodox Jews of the Lubavitcher hasidic movement" not representatives of some sort of Jewish privilege in general.
That's easier said than done. The problem is that in order to reform immigration you have to do one of two things:
1. Start treating people fairly, with the result being higher food prices, which will make some of your constituents very unhhappy
2. Codify the current class structure into law so that you can continue to exploit cheap labor legally. The optics on that are really bad.
Exploiting people off the books is the path of least resistance for everyone, including the workers (manifestly so, or they wouldn't risk life and limb to be here).
It is also difficult because of the two tiers of labor. A farmer will try increasing wages by $1/hour, and still struggle to find enough workers. Wages would probably have to be doubled before new workers—Americans—would start to enter the industry.
I'm assuming you're implying a conversation about China and their manufacturing capabilities. But worst case scenario short of nuclear war, assuming US-China burns bridges and somehow for some reason now US is fully unable to import Chinese manufactured goods; how does this eliminate the US and all other countries from investing in an area that has now become more profitable? You could argue that prices would increase, but then again you aren't owed cheap manufacturing, and subsidies and tariffs are paid by either the taxpayer or the consumer (which tends to have a knock-off effect on the rest of economy, anyway). There was never a free lunch.
USD $5/hour is what I earned as a teenager working the fields by hand (picking corn, beans, and fruit for sale same-day at the farmers small stands throughout the area) in the early 1990s... That's insane. Is that USD $5 or AD?
Now, outdoor work with a high injury rate is a different story.
What I've seen first-hand in some farming communities was closer to $10AUD/hr, but that is still far below our actual minimum wage of $19.84AUD/hr.
Sure, farmers could pay $10/hr, but unless your Doritos were $10 a bag, they’d lose money.
If nobody wants to buy $10 doritos, that means the product can't exist in a market without cheap labor.
[1]https://www.ebooksread.com/authors-eng/andrew-carnegie/trium...
"Exploitation" is a relative term, some people would rather be exploited in the countryside of a first-world nation rather than the countryside of their homeland.
The Australian national minimum wage is $19.84/hr.
Getting paid about a quarter of minimum wage is very clearly exploitation. There may be other benefits for actually working in Australia, but that's not really relevant here when you're trying to attract native Australians to work for you.
It’s all relative.
And it's not just cheap labor, it's labor exploited under pain of deportation.
Edit: that was just based on my general memory of non-perishable food economics. But I googled it and found this (sorry, quora) from someone who would know, a 2 oz bag of potato chips costs 15 cents to produce and bring to (retail) market. Of that, a fraction is potato prices.
https://www.quora.com/How-much-does-a-bag-of-57g-of-Lays-act...
Edit: fix Verizon math.
Same reason I avoid organic whenever possible. We don't need boutique food.
Actually farmers could set any price if they colluded to manipulate the market. People will buy food at any price and everyday.
[0]https://www.cato.org/policy-analysis/candy-coated-cartel-tim...
I am telling this as an engineer working in the Canada's oil and gas sector. Since 2014, salaries in our sector have decreased and firms are struggling. "Just raise the price of oil" is not an actionable advice. The price of oil is not ours to set. It's the market's
https://www.lowtechmagazine.com/2020/04/fruit-trenches-culti...
You're absolutely right that perhaps not everything should be grown everywhere, but I strongly believe that a country should have a local source of all the calories and vitamins / minerals they need. In Canada, for example, maybe they encourage farmers to grow greens, brocolli or potatoes instead of citrus for vitamin c, but they absolutely need a source.
But imported products don't fly right into the country. There is customs, and denying/tariffing them is a policy that countries are very actively using. So at the end of the day, it's a political decision.
Edit: On the other hand, if you are exporting stuff (oil for Canada), then you can't really do that for the external market. Might explain why the government will let some sectors run with illegals (to remain competitive); the alternative being their ultimate demise.
[0] https://en.wikipedia.org/wiki/Chicken_tax
[1] https://www.npr.org/sections/money/2017/01/25/511663527/epis...
Australia does not use this approach, and instead seems to have pushed other countries to eliminate their own tariffs for the TPP.
Of course, you might get pilloried for the artificial scarcity, but hey... nobody ever likes the capitalist :)
That's certainly a simple solution.
The problem with simple solutions is that they are near guaranteed to not actually be feasible or realistic.
> The problem with simple solutions is that they are near guaranteed to not actually be feasible or realistic.
This. There is no amount of money anyone sane can be given to stay in Romania. Most recently i’ve read they (hospitals) were tying covid patients to their beds.
Couldn't help but comment on this as some of things i discovered are shocking for an eu member country (or any country for that matter), such as EU leading human trafficking, police corruption, collapsing health care, and the list goes on.
But sure, the simple solution is to give people money, not the constant abuse taking place at work and outside that makes people want to flee that place.
Companies paying more (offering more at least) would begin to slow this domino effect, which raises standards of living and provides more funding for the government to provide services and ultimately pressure for increases in quality of life by the voting public. Everyone wins.
Ultimately not everyone can emigrate - it's an impossibility. The brain drain will continue to be real, so it's incumbent upon these companies hiring to raise their wages and retain talent.
It is really not as easy as go "this is the exchange rate" and simply use that to compare pay in different countries.
Companies aren't going to say "I guess we're inefficient and should just die." They're going to fight to survive and thrive, which is what you're seeing.
Then again, some companies ARE just cheap. You never know.
They said the standard is 30 minutes per room per day. I figure at a very generous $50 per hour (100k / year) it should run $25 to have my room taken care of. Now that doesnt cover the shared costs - doing laundry, cleaning the pool, front desk, etc... but my room was something like $150 per night. How the fuck can they not find people to clean rooms?
Answer: they're too cheap to pay a decent wage. But why? My suspicion is that all these places are actually leveraged far too much and paying rent/interest/dividends so much they can barely operate.
I suspect low interest rates and other incentives to "stimulate" economies are actually causing these problems.
Sorry, I though I was on the Romania subthread.
sorry for pun. But that's not always a solution. Sometimes employer is greedy. Sometimes his business just does not allow to pay enough salary.
Usually, since admitting that they were stupid to let all the knowledge of the system walk out the door requires more emotional and intellectual flexibility than managers who let this sort of thing happen possess, the biggest obstacle isn't even the money... it's getting the managers to admit they're idiots.
Anything but pride, I guess.
> The company that built this ERP solution went bankrupt 6 years ago, and we don't have the source code. It uses PostgreSQL and MySQL, and also has a built-in key-value database for which none of us has any credentials. We don't have the source code for this software (it's built in C and delivered to us as compiled packages..).
Sounds like finding their old employees wouldn't help. They need to find old employees of the company that wrote the system and hope they kept a copy of the source code.
Which from a legal perspective might be fraught with peril, at least in theory. In practice, if the company folded then it'd depend entirely on the litigiousness of the current owner of the copyright, which could range from "nobody" to "some holding company consisting almost entirely of lawyers chomping at the bit to eat you for dinner".
I know of a hospital which is still refusing to migrate off of a system which is not supported for a decade. Pay more money? 3 people worked on it: Adam had a stroke, Bob retired with enough money, Charlie left the country.
The reality is company execs like to talk about fixing problems with money but rarely are they actually willing to fix a problem with money. In my career I dealt with the exact three companies where execs decided to throw money at the problem bypassing all the fiefdoms and bureaucracies created to avoid rapidly solving problems using money in favor of slow moving, don't step on Jackson the IT manager's toes, approaches. All three solved their problems, destroying the local fiefdoms in a process.
After 25 years, I'm an expert at making small, efficient software tools to fix local IT workflows. God knows, in the Fortune 250's I've worked at, there are endless opportunities. And, while I've written many solutions, I've seen this political game of thrones thwart my efforts many times. This is how you get the kind of 20-year legacy mess described in the original post: People sacrificing long-term success of the company for short-term success in their careers.
I guess there are 2 kinds of people. Those that think this is just how the world works, and find this to be a perfectly-cromulent way of getting through it, and those, like me, who find it odious. Luckily, I now work for people like me. We do what we can.
The ERP system was a nightmare, though, and the company was difficult to work with (key, long term employees). It was a system I inherited, built out of garbage, that I tried to improve. Before I left, I found a very experienced contractor to take over, on-boarded him, and stayed as a support resource for a couple months. Six months later, he's retired and I'm getting panicked phone calls. I valued (and still do) my current job far more, even though I would probably have done better financially in the short term with the alternative.
A few years ago, I heard they migrated to a new ERP at long last.
In context of this case - if they don't have money to fix their backups (https://news.ycombinator.com/item?id=26539988) they likely couldn't drop lots of cash without a serious credit line.
A new system isn't going to do them much good when they don't even know who their customers are. I doubt any customer is going to hang around for a year while you rebuild your system from scratch, ignoring the fact that if they're an MSP the customers may not even have their own copy of the data.
All of that is ignoring the likely contractual obligations they're likely under to maintain the data on their systems.
>Also it's a public hospital - for a fraction of $1M they could get people who's only job is to replicate that system running around with pen and paper - they wouldn't have $1M to just drop on a support issue the same day.
Hospital? He said they're an MSP on reddit:
>Yeah we're an awful MSP. I've been trying to find a new job for quite some time.
https://www.reddit.com/r/sysadmin/comments/ma4mwl/the_absolu...
This thread is about a different case.
He had retired, but took another 12 or 18 month contract flying around the country servicing these things. Apparently they asked, and not wanting to do it he quoted what he thought was an insane "go away and leave me alone" fee, but then they said yes. (I have no idea what that fee was or what multiplier he applied, though it definitely brings to mind patio11's standard advice [1]).
[1] https://www.kalzumeus.com/2006/08/14/you-can-probably-stand-...
If the long-gone ex-employees hated their employer and would be happy to see if go out of business then you're in quite a bit more trouble.
Asking because, while, yes, throwing money at the problem might be a solution, I think the obvious bigger issue is how does one allow themselves to get into this situation?
Or rather, how does one avoid getting into this situation. Because it seems to me that the software industry of late has become so mired in esoteric layers of abstraction that this feels less like an outlier scenario for a mismanaged company but more like an inevitable one for many otherwise well managed companies in the industry.
At a lower level, it's allowing individuals to become knowledge silos (or having a bus factor [1] of 1); lack of documentation; not having+testing disaster recovery plans; neglecting systems, code bases, and platforms; letting technical debt accrue endlessly (or worse: not even realize that's what you're doing).
How to avoid? Make sure there's a strong, competent technology leader with a vested interest in the company succeeding, and empower them to make the changes they need to.
What should that person be doing? Making sure all the stuff I just said doesn't happen.
The relevant part of the article:
> To anyone outside of the ballpoint pen manufacturing world, it might seem hard to understand what, exactly, is so surprising about this development. China already produces 38 billion ballpoint pens a year, according to China Daily, which is about 80 percent of all ballpoint pens in the world. That's a lot of pens, but there was a catch: China had long been unable to produce a high-quality version of the most important part of the pen, its tip.
> The tip of a ballpoint pen is what makes it a ballpoint pen. At the tip, a freely rotating ball is held in a small socket which connects it to an ink reservoir that allows the pen to write or draw lines. Manufacturing a ballpoint pen tip that can write comfortably for a long period of time requires high-precision machinery and precisely thin steel, but for years China was unable to match those crafted by foreign companies.
IMO, the problem is technical debt. Most likely, in the majority of these cases, the engineers working on the system knew perfectly well that it was a mess, was very difficult to learn how to maintain, and many parts were dangerous to modify. Most likely they told management multiple times that this was a problem and what the consequences of it would be. They would have refused to budget resources for improving it, so everyone either saw the writing on the wall and quit, or gutted it out and learned how to manage the system eventually. Maybe the system limps along for a while longer, but it's only getting worse and the knowledge needed to maintain it more siloed and specialized. Eventually you lose a key person, something breaks that nobody left can figure out, or maybe the whole thing just implodes like the main story here.
To avoid it, you must understand the pattern, hire good technical people, and give them the resources needed to improve the system in an orderly fashion.
I guess any company where leadership got there without progress on the technical side. Law firm partners get there after being lawyers. Consultant partners get there after consulting.
But if you are the CEO of an IT company without ever having to code, then you may never know the things you must take care to prevent a washout.
Mentorship and documentation.
You need to set aside time and resources for these two things, and make contributing to them worth something in the performance review.
If all you do is count JIRA tickets closed, you'll forever end up in this situation.
Documentation?
Significant documentation is also not something easily written in slack time. With any non-trivial project good documentation is a full time effort. It also needs to be updated as the project evolves lest it become out of date and incongruous with the behavior of the system.
There's dreams of in-line documentation but that's only going to cover individual methods and classes and not necessarily whole modules or major subsystems of a project. It's also not going to necessarily cover in-line documentation from other parts of a project in different languages e.g. comments in config files.
There's lots of technical ways to help documentation but there's no replacement for documentation being a high level project goal with resources allotted to it.
For example, in code: good naming, simple structure, unit tests, and useful comments all can count. A set of easy-to-understand unit tests that cover real scenarios is much better than even the greatest documentation ever written.
A fully scripted build and ideally also deployment (CI/CD pipeline) beats a checklist with tens of items that have to manually followed.
An issue system with good descriptions of changes (including why and what), reproduction steps, expected outcomes, links to related issues, etc also can go a long way.
All of this stuff has the side effect of making day-to-day work simpler, overall, while actually just being a part of that work.
1. Naming things is hard. It's a bit of a joke but good descriptive yet usable naming is hard. It can get harder after refactoring or adding in features you didn't expect the last time you were in the module.
2. Project structures start out simple. Then you need to add some feature or fix some major issue with a due date of last week. As a project gets bigger the simple structure of yesterday might not serve the needs of today. Resources for refactoring are usually somewhere around the priority level of "Good Documentation".
3. Comments need maintenance like explicit documentation. Tight schedules lead to "# TODO add documentation here". Then you get the assholes that believe comments or in-line documentation is something you put in check in notes.
4. A "unit test" that covers a "real scenario" is not a damned unit test. It's an integration test (or however you label it) and it's not likely something your CI/CD system can run on the regular. If a real scenario is touching a production-load DB or something it's a big deal to run the test. Tests like that need resources to run and non-trivial resources to write. You're not just making sure a method returns a float or throws an error correctly.
I'm not saying your suggestions are bad or shouldn't be used. The issue is management rarely sees any value with documentation, tests, or even code cleanliness. They're rarely incentivized to care about those things so they don't. They get their bonus on feature checkboxes or hitting a deadline. At the end of a project they'll schedule a "documentation" sprint or some dumbass thing that accomplished little useful output. It's worse when you've got a "move fast break things" chanting moron doing the scheduling.
The specializations common on a team even 20 years ago was much closure to the "Surgical Team" model from the mythical man month than what we have today.
you handle this by having and exercising a disaster recovery plan and having (and listening to) competent people that would spot this issue in a jiffy.
You'll continually be amazed at the human tendency to double down instead of admitting a mistake.
That last part may not be necessary though, as customers will jump ship and the company will probably sink anyway.
Depends, they might have a captive market or a customer base that's non technical and doesn't understand how bad this is.
If their systems are like this, chances are their offering isn't anything groundbreaking and the competition already provides a better service for possibly even cheaper, so the customers who are willing to jump ship would've done so long ago and the fact they're still in business suggests they have a complacent customer base that's likely to stick to them even despite this incident.
The same reason this technical debt was left unchecked for ages applies to customers. The task of migrating to another provider will most likely rot forever in a Jira board somewhere and they will keep paying their bill in the meantime.
I would anticipate a mix of: the people who know where the right bodies are buried, the right incentive (yes, that includes money), but also people to look over the entire program, a designer to make sure it is designed correctly, an organized person to make sure it is properly documented, and someone to make sure it is adhered to in the future.
Everything they list that's an issue seems to revolve around the fact that the employees with domain knowledge for the system have left the company.
Reach out to them and hire them on as contractors. If they left under bad terms because the business was a bunch of dicks, expect to pay 10x market rate. If this is truly "fix this or the business is out of business" - then it shouldn't be a tough decision to make.