News for hackers. Stuff that matters.
Re:Moved a server from one building to another with zero dow... by Spooky23 on Wednesday August 05, 2020 @10:31AM (#24061185)
The real lesson is that the teller of the tale sort of did initally -- fire the customer.

If the story is true, the client is a stereotypical know-it-all small business owner who gets by on bullying. You see them frequently in businesses that pay low-skill workers a small premium that is hard to replace. (ex: cleaning services, pool guys, mechanical contractors that do low-end maintenance work, etc)

As a contracted SME, taking a job like this is dumb. The chances of failure, where "failure" == the server going down is high, and the customer will just stiff you.

Re:Moved a server from one building to another with zero dow... by paulsutter on Wednesday August 05, 2020 @11:45AM (#24062101)
Key is to set the price. If the job is a hassle, you're not charging enough. That will also filter out bozos. Sometimes people really do have extreme requirements. And when they do, they're willing to pay 10x for it.

Agreed though, this particular customer disqualified himself as soon as he said he wont pay if the server goes down. He should have offered a big bonus if the move succeeds without downtime.

Re:Moved a server from one building to another with zero dow... by kenhwang on Wednesday August 05, 2020 @02:40PM (#24064082)
When I did consulting, we always got unreasonable but technically not impossible asks like this. We never "fired" the customer, because that's just bad business and customer service. What we did instead was tell them their options, our recommendation, and "appropriate" billing estimates. Your job is to consult them to the best of your ability, not stop them from bad decisions despite your advice.

So, 1 hour billed for 5 minutes of downtime, or 40 if you want absolutely none. Happy to do either, but highly recommend the former. 99% of people pick the cheaper recommended option.

In this case, I would've tried to put the server on WiFi which would seem like less a hassle for me. Equipment acquisition cost billed to the customer.

Re:Moved a server from one building to another with zero dow... by michaelt on Thursday August 06, 2020 @04:54AM (#24069232)
> Your job is to consult them to the best of your ability, not stop them from bad decisions despite your advice.

If you offer a client an option with a 1% chance of a very bad outcome and they understand and accept the risk, sure.

But if they have not understood and accepted the risk? Implementing something you already know the client misunderstands ain't always a smart move.

Re:Moved a server from one building to another with zero dow... by HeavyStorm on Thursday August 06, 2020 @11:55AM (#24072117)
That's consulting for you.
Re:Moved a server from one building to another with zero dow... by kenhwang on Thursday August 06, 2020 @03:07PM (#24074341)
Yep. Then it's either extra billable hours for you to fix the mistake you knew was going to happen, and/or the lawyers get involved and point out the mistake being requested in whatever statement of work. Even better if you called out the exact failure scenario in the risks section that you suspect they didn't read.

At the end of the day, it doesn't really matter if they listen/understand or not, as long as you documented telling them or it makes it into some legal document and they still pay the bill.

Re:Moved a server from one building to another with zero dow... by derefr on Wednesday August 05, 2020 @02:38PM (#24064059)
> Key is to set the price. If the job is a hassle, you're not charging enough. That will also filter out bozos.

Sure. Just, there are some verticals where charging a "positive-ROI" amount gets you no business at all, because all the potential clients in that vertical are businesses that operate on such razor-thin margins that they don't actually have the cash-flow to pay for the extreme requirements they also have. They've been getting along until now purely by begging/tricking/manipulating people into doing negative-ROI one-off tasks for them. If forced to get contract all the services they need out on the free market, their business would cease to exist.

(Therefore, you say, they should cease to exist. I'm not arguing!)

Re:Moved a server from one building to another with zero dow... by rdslw on Wednesday August 05, 2020 @04:35PM (#24065281)
> there are some verticals where charging a "positive-ROI" amount gets you no business at all,

if you do, you're just selling dollar bill for 80c. You may drink growth-kool-aid, or someday-monopoly-hope or VC-subsidizes-business.

In the end, somebody pays for it either from stupidity or hope.

Re:Moved a server from one building to another with zero dow... by derefr on Thursday August 06, 2020 @10:32AM (#24071230)
To be clear (and I realize this wasn't quite clear), I wasn't talking about pure monetary ROI, but rather "return" in terms of satisfaction of your preferences.

I.e., a job paying a lot of money is "return" on investment; but so is a job that pays a lot of experience, or satisfaction. Meanwhile, a job that takes a lot of time to do requires a larger "investment" (in terms of the set of BATNA jobs you could otherwise be doing with your skills); but so do jobs that don't take very long, but are more physically demanding/emotionally draining, or where the client keeps jerking you around by changing the requirements. If working the job makes your utility go up (i.e. makes your life better), it's a positive-ROI job. If it makes your life worse, it's a negative-ROI job.

So, let me restate: in some verticals, while the clients do pay a livable time-and-materials wage for the region and experience level... they expect far more labor (and far more highly-skilled labor) for their dollar than that dollar "should" be able to get them, such that the net utility from anyone taking that job is negative, even if they're "making money."

These businesses get by mostly on exploiting young IT ingenues who don't yet realize their worth, and so are willing to give a try to e.g. "program a social-networking app" for $2000; or to e.g. "set up a PXE Linux automatic memory-testing boot env and Windows system-image deployment pipeline" for $300.

(These are both things I naively got myself into when I was younger. Amusingly, I was in over my head on the social-networking app, but managed to complete the hardware qualification+deployment env just fine. And my only major obstacle for the social-networking app, was that the client insisted that it needed to be hosted on some free ISP hosting they had, and thus needed to be written in PHP4. Which I didn't even complain about... it was just a language I wasn't very familiar with, so I tried to learn it while working. Sigh.)

In other words, these companies try to get newbies to do senior-level work, paying only newbie costs. Sometimes, surprisingly, that works! And they survive entirely because it sometimes works.

It's basically the "my nephew knows computers, he can certainly repair mine" mindset applied to corporate planning.

Re:Moved a server from one building to another with zero dow... by spydum on Wednesday August 05, 2020 @07:55PM (#24066877)
Evidenced precisely by all these ransomware attacks where victims have to pay up because they didn't invest in backup resources (maybe they had some tapes, but didn't find the staff or process to test any of it.. because they haven't needed them so far).
Re:Moved a server from one building to another with zero dow... by Dylan16807 on Wednesday August 05, 2020 @02:57PM (#24064257)
> Agreed though, this particular customer disqualified himself as soon as he said he wont pay if the server goes down. He should have offered a big bonus if the move succeeds without downtime.

I don't know, that sounds too close to encouraging the attitude of "it's not worth it, I'll just take my normal pay". 10x vs. 0x is a significantly stronger incentive than 10x vs. 1x.

Re:Moved a server from one building to another with zero dow... by noobermin on Wednesday August 05, 2020 @02:36PM (#24064043)
This is probably the best compromise. Sure if you're moving a server with, idk, top secret government spy files or something then you charge 10x or some multiple. But that would obviously be the exception. The vast majority of these people are just self-entitled dipshits who have an inflated notion of self-worth.

See my comment elsewhere on "the customer is always right."

Re:Moved a server from one building to another with zero dow... by donmcronald on Wednesday August 05, 2020 @11:38AM (#24062000)
I deal with a fair bit of customers like this (not via my business), enough that firing them all isn’t an option because it’s a large portion of the market where I am. It’s something. They’ll have low end servers with no redundancy, terrible or no backups, and no contingency plans for anything. They won’t spend a nickel and are the most likely to lose their minds if anything goes wrong.

It’s so frustrating and stressful.

Re:Moved a server from one building to another with zero dow... by eldavido on Wednesday August 05, 2020 @02:46PM (#24064124)
This is why services like Google Apps or managed exchange hosting exist. Most people are terrible at IT management. So bad they're far, far from realizing how bad they even are.

When you consider you're getting like 1000 of the smartest tech people in the world to manage your infrastructure for you, for $5/month per user, it's really such a no-brainer. If people are too stubborn to see that, or want to waste time trying to do it better themselves "because it's cheaper", with redundant power, OS patching, zero-downtime changes/deploys, proper capacity planning, proper redundant connectivity, provisioning the right network around it, physically securing the server room, ensuring things are properly cooled, not wet...I could go on forever, and this isn't even the main focus of the business...I'm sorry, but they deserve go to out of business.

I had a very stubborn client once who ran a hotel chain. Won't say what or where, but I wasn't surprised when their random "security through obscurity" VNC server got compromised. I wasn't finishing migrating to the new PCI-DSS compliant system we built, either, so there go 5000 credit cards "encrypted" with some sweet rot13-level bullshit in Turbo Pascal I cracked in about 30 minutes with no code access.

Re:Moved a server from one building to another with zero dow... by ed25519FUUU on Wednesday August 05, 2020 @02:22PM (#24063899)
Seems like some dedicated hardware here would make this go faster if there's a business for it. For example, if the bandwidth isn't high, you could setup a wireless mesh from point A to point B and connect via some appliance to the NIC.

Walk the length with the appliance and verify there's no dead spots, then just hook up to a power supply and get things done.

Re:Moved a server from one building to another with zero dow... by Bnshsysjab on Wednesday August 05, 2020 @05:49PM (#24066013)
In the 90’a wifi was even more of a trash fire than it is today.
Re:Moved a server from one building to another with zero dow... by renewiltord on Wednesday August 05, 2020 @02:23PM (#24063908)
Presumably at least part of the value of the job from his POV was the excitement of it. I've definitely done suboptimal jobs where I just enjoyed them a little and they were a break from the replicable tasks.

Since the customer is never going to know the awesomeness of it, it's really just for yourself.

Re:Moved a server from one building to another with zero dow... by dialamac on Wednesday August 05, 2020 @11:45AM (#24062102)
There’s a market for everything. The “shit customer” would have hired someone. Sure things could have gone to shit and they wouldn’t be paid, but they didn’t, and there is a living to be had serving this market segment. Every contracting business has difficulty with AR, stereotyping often isn’t even particularly acccurate. Suffice it to say there is a business to be had in catering primarily to difficult customers.
Re:Moved a server from one building to another with zero dow... by mlyle on Wednesday August 05, 2020 @12:07PM (#24062354)
> stereotyping often isn’t even particularly acccurate.

While I agree that it's difficult to predict exact credit risk based upon customer personality, explicit threats not to pay you like exist in the story -are- a bit of a signal that there may be a risk of nonpayment.

Re:Moved a server from one building to another with zero dow... by kube-system on Wednesday August 05, 2020 @02:12PM (#24063776)
Sure, I think the point was that there is a market for customers with subprime credit. Sometimes it is fairly profitable too.
Re:Moved a server from one building to another with zero dow... by nautilus12 on Wednesday August 05, 2020 @01:39PM (#24063396)
Depends on what you mean by business. The nature of that type of business is that it's not very predictable or repeatable so you get one small chunk of business but ultimately it's not the kind of business (even a consultancy who's job it is to throw hours into the fire) you really want. Scaling that out would be death by 1000 papercuts.
Re:Moved a server from one building to another with zero dow... by dialamac on Sunday August 09, 2020 @11:31PM (#24105217)
No. If anyone is going to serve the subprime market effectively and profitably, it is someone that can scale it out. That’s the whole point.
Re:Moved a server from one building to another with zero dow... by nautilus12 on Wednesday August 12, 2020 @09:45AM (#24132010)
You misunderstand what I am saying. I meant that the nature of this type of business is that it that you can't really scale it out