As someone who recently created yet another startup, I am reminded daily that literally everything I do has to be done for the first time. It's like molasses covering every part of the machine. When a client needs a proposal you have to write one from scratch, when someone wants payment terms or to negotiate a contract adjustment you have to figure out contract language. You don't have expense reimbursement forms, you don't have job descriptions, you don't have anything to use as a starting point or template for anything you do.
Big companies have all that. They have clients that are planning on using them again this year. They have meetings they took two years ago that are about to be a sale. They have a guy on the third floor that has seen this problem they're about to have once and knows how to avoid it. People instinctively trust them, so they don't have to explain themselves as much. People instinctively fear them, so their contracts get upheld. People that used to work for them now work at potential customers and call them up.
And on and on and on. Having a slightly busted software development process is, in fact, peanuts compared to all this.
Or, perhaps another way to say it is that there's a lot of different kinds of "software" besides the digital ones-and-zeroes kind we think about here. A group of humans all organized around a business purpose with tons of experience and momentum is a high functioning machine as well.
A local restaurant becomes #1 by having the best food and/or location, and providing it at a price people are willing to pay.
A car dealership probably succeeds by having the best prices on cars and the best customer service.
Most banks compete on customer service and some amount of product differentiation.
Insurance is some combination of customer service, market share, and correctly pricing things.
To the extent that better software contributes to these core "capabilities", companies will invest in it. Investment means it gets management attention, willingness to spend, willingness to hire annoying top-shelf talent that demands flexible working arrangements and huge salaries.
Many -- maybe most -- business don't have competitive dynamics that software enhances. A hotel is a great example: it's location, price, service, and cost control. The software they use at the front desk has approximately zero importance to whether the place will stay in business, or fold. Owners are right to invest as little as possible in it and move on to something more important, like building a brand.
https://www.jimcollins.com/concepts/the-flywheel.html
I have to disagree with the broad statement that a busted software development process is peanuts. In highly regulated industries like finance, bad software can cost you billions in fines from regulators.
In these industries, they have to invest in technology and good processes if they want to stay in business.
In a sense, this lends credence to the theory that once you’re big and powerful and have lots of clients, you can afford to lose these opportunities. But I still think that you would have a huge competitor advantage without all the molasses.
You're assuming the regulators are low-level enough to identify a flawed system. A lot of regulations are applied at a policy level (aka process level) and auditors (internal or external) are not always able to identify severe gaps between what's documented in a policy and reality of day to day. Good people and technology matter way more than process.
Sure, if we hire perfect people there's no reason to even have a process. In reality, when that breaks down you have to rely on your process to close those gaps. But, like you said, a policy/process is only as good as the paper it's printed on if nobody follows it.
100% agree - My point was more that regulators tend to use policy and process artifacts as proof of security compliance. The idea that more highly regulated/audited IT sectors are inherently better at security is probably false. The audits just aren't stringent/detailed enough and there's aren't enough incentives for executives/boards to do more than the minimum. Which is why the values/opinions/ethics of those decision makers becomes key.
The most secure 'company' I worked for was a large research university because it was a priority from the director and we were constantly getting probed by overseas IP addresses. On the other hand, the worst company I worked at in terms of security was in the payment processing world. We were PCI DSS compliant and audited almost continuously by various interested parties (banks, card brands, third parties, etc.). I don't think any of our people were bad/inept but at the end of the day our policy artifacts made us sound/appear way more compliant than we were. The problem was upper management didn't care (allow us time or $$) about closing the gaps in implementation unless we failed audits or pen testing. We didn't fail but no one in IT thought we were doing enough. We didn't have enough auditing and regularly had inexplicable 'bugs' reported that honestly could have been evidence of network intrusion but we had no proof either way.
The safest decision, 99% (but not 100%) of the time, is to cancel that software project before it goes to production, because if you've been successful up to now, by definition you have software that is Good Enough (if not good). So, in highly regulated industries, changing your software may very well be an unwise decision.
Not least because, if it's the same software the regulators were ok with last year, they probably won't fine you billions for using it this year.
Caveat to all of the above: "probably". But the chance of new software getting you into trouble is still higher.
Most businesses fail because they don't have customers, not because they're inefficient or badly-managed.
They key is right there in your statement: "have to". Good places to work don't build software because they "have to", they do it because it's viewed as an important competitive differentiator and execs WANT to spend on it.
I think for software sales, this type of thing is slowly fading as it becomes more and more commoditization. For all but the largest of businesses, you go to AWS and pay the market price for compute and other services, there's not much need for contracts and negotiations and meetings, the price is the price. Going to a competitor or doing business the old way simply costs much more. Eventually, businesses align on consuming commodities as commodities, the spending slowly moves from CapEX to OpEX.
AWS is the power company now. You agree to their terms, and you pay their bill. Unless you're doing a major industrial facility that requires mega-watts of power, you don't need to interact with their sales staff; same applies to AWS (and ironically, you can probably use power as a proxy to relative deal size here, too).
100 years from now, all the conceivable software is going to exist that could exist, at least for 'business processes.' If your business is selling potatoes as a farmer to distributor, the software you need to participate efficiently in that market will already exist.
The market price isn't what most companies pay. If you pay significant amounts to Amazon (maybe 1 employee salary worth) you're screwing yourself if you don't negotiate lower rates
Edit: Just asked the corporate overlords. It's actually against our finance rules to pay for any recurring service charge without a negotiated contract.
Your company probably also employs a professional shark whose job it is to squeeze the blood out of every stone he is pointed towards. With even a modest number of sizeable contracts, such a person would pay for themselves very easily.
I disagree. Unless we're talking about Artificial General Intelligence or some other general purpose technology, I think bespoke software will always be better--if not more cost effective--than off-the-shelf.
On the other hand, there are entire markets, even quite large ones, that you can't pursue because they're way too small. That's what I'd look for if I were thinking about starting a new business.
End of the day, a big company or government agency is too big for any human to internalize. It's a machine that pumps out process, but the downside of consistent process that change is more difficult.
When I worked at an agency that supported social services, some guy was bitching and complaining about how everyone who wasn't him was stupid, the agency was awful, etc. One of the more respected directors walked by and heard this. She popped in and said something like "There are x million people who depend on us to feed their children. And y million people who expect us to do so in a responsible and equitable way. We've never missed a payment, ever, because of this team."
Another example is the military, which is probably the dumbest possible bureaucratic institution ever devised. Yet is capable of executing the mission with ruthless efficiency, and can survive in the face the adversity and loss that few organizations can.
Smaller organizations are always better at reacting to change, because there are fewer stakeholders. But, they suck at many things, including consistent delivery, paying things on time, and organizational resiliency. Software makes them better, but software is its own form of bureaucracy with its own set of problems.
Think about the difference in terminology between a coach and players. When a coach takes a team through a play, they usually don't tell the players which hand they need to dribble with or which foot to plant first when running, they use much broader strokes and leave the implementation details to the player.
Let's take the example of... perhaps a Jamaican bobsledding startup. In the early days, you don't know what you're doing so everything matters. You scrap to purchase airfare to get to key industry events so you can be where the players are. If you mess up almost anything, it's game over. But you make it there, and you perform like you knew you could. You pick up sponsors. Expenses like airfare are now no longer a problem. What before used to be a mad scramble to organize and scrape together funds and make sure you have a good pitch for the big meeting is now a smoothly oiled proven machine with accounting and assistant cogs. Your jerryrigged home workouts are now streamlined by an Olympic training facility. Your mind is now no longer occupied by what has become minutia to you. You don't really worry about grocery shopping because you have a team nutritionist. You don't care about how efficiently or inefficiently the process of grocery shopping and cooking goes down now, just that it does and is on time.
You no longer make music with an instrument. You conduct an orchestra. Your mind is now thinking in bigger blocks.
Software is just one section of most orchestras.
I still think that doing things the right way and getting rid of those famous inefficiencies could leverage success even further. With regards to software I think that's the difference between large companies with a known engineering culture (like Facebook, Google, Apple, Microsoft) as opposed to those known to be a bit more red-tapey (like IBM, Adobe).
But of course, "doing things the right way" is terribly hard to quantify, and so is "how much success would be possible". If numbers at the end of the year are already positive, good luck trying to convince C-level to introduce fundamental changes to existing processes.
I don’t think it is hard to quantify actually, shorter build times are better than longer ones, branch merges that take seconds are better than the ones that take up to a day, spending people’s time on higher level thought processes vs squandering their time going through pages of manual steps that could just be a script, etc. It’s just a lack of awareness of what the company is doing that turns them into molasses, and puts projects at risk of being canceled. The companies are still successful overall somehow, but individual employees learn and grow at a much slower pace than they’re capable of.
Software is viewed as a cost center at a lot of companies, and in many cases that is the correct decision. The company officers and managers understand this, which is why they act the way they do.
Their goal is not to make the best possible software -- it is to ensure the software is good enough that it does not hold back other areas of the business, and to achieve that within the budget they are given.
Put another way, a company's organization can be seen as its software, except it's running on a kind of wet, carbon-based hardware that does certain things much better than the silicon-based ones, and other things much worse. As the capabilities of the silicon-based computer increase relative to the capabilities of the carbon-based, there are few industries where it would be wise to continue purchasing software like one purchases pencils.
A lot of advances are gradual, and if you are a large company you will have time to react.
Plus, small companies that make big mistakes that do survive, are scarred by the experience and tend not to repeat that particular kind of mistake again. Companies that can make a blunder of the sort described in this story, repeatedly, have to have been big in order to survive long enough to repeat it.
IBM will continue to target their most expensive employees for layoffs while hiring cheaper new grads, but even after those alleged practices, their profit per employee fell by 5k. Seems reasonable to expect they won't be around in a decade, though I'm no financial expert and this is a projection from two datapoints.
You might take it as evidence that, of all the myriad aspects of a business, scale far outweighs all other factors for generating a profit.
You don't have to be "good" (whatever that might mean) to generate profit, production costs just need to be marginally cheaper than your prices.
The fact that a large company is rich does not necessarily mean that the inefficiencies don't exist.
I somewhat agree with you on "this whole software stuff is really peanuts and their mental energy is better spent on other business-related stuff". For most companies this software stuff can be mostly irrelevant, and it's our fault this is the case. As Erik Dietrich says [1], we are "efficiencers" -- we're supposed to make all kinds of processes more efficient, not just be code monkeys and copy code from StackOverflow.
[1] https://www.amazon.com/dp/B0722H41SG/ref=dp-kindle-redirect?...
This comment should be featured higher in a discussion. The "stupid" label is thrown rather fast although more often than not the problem lies in how critics fail to understand the problem and, more importantly, the restrictions applied to the decision-making process. Therefore critics base their view on their personal and very over-simplified and under-informed view of the problem, don't understand how an outcome is a valid and reasonable solution, and instead of assuming that they have limited information and understanding... They instead assume they have perfect information and complete insight into a problem and therefore any outcome they didn't approve or understand can only mean stupidity.
What I see is: most of this new cohort has _no_ idea what our business is and they are overwhelmingly concentrated in middle management.
https://www.ribbonfarm.com/2009/10/07/the-gervais-principle-...
If software is not your core competency, you don't need to be any good at it to make money.
That leads to ignoring simple observations about reality like this one, since this one implies that in most companies software doesn't matter much and the CEO is right to treat it as a cost center. Nobody wants to hear that about what they do with their life.
To be fair, you could run 'sed s/programmers/humans/' on my first sentence and leave its truth value unchanged.
These now big corporations were really good at something...so good that they beat the competitors, grew their market share, expanded their head count, etc.
However, operating at a higher and higher scale (or with more modern technology) brings new challenges that weren't part of the game before. Ones that they are not necessarily good at. Eventually they might become great enough to threaten the company itself (AOL, AT&T, Yahoo, etc.)
The inefficiencies you see are characteristics that accrued due to the level of success and exposure to new challenges. The inefficiencies are not characteristics that got them to that success.
https://en.wikipedia.org/wiki/Theory_of_the_firm
I think a fair summary is to say that there must be some value in having thousands of people who start each day with some vague idea of what the organization needs from them. And this value must be huge, because it has to outweigh all the value destroyed by corporate inertia, dead weight, infighting, etc. If it doesn't, the firm can't survive very long.
Only a wealthy company can spend $400.000.000 on an SAP project, for example, but in reality it would probably be better for the business long term, to build something custom and embrace the fact that IT & tech is important to achieve its goals. If it's not, why spend the money?
Doing it right probably means creating a "software factory" or tech hub within or outside of the company.
If we're talking a company with muscle and will to plow down $100s of millions on a massive tech/automation project, this should be feasible right?
For an example, see Lufthansas tech initiative: https://lh-innovationhub.de
(I'm not in any way affiliated to LH, I just happened to read about it a while back, while trying to convince my then CEO that something like that probably would be better that spending millions on proper crap from 3rd party)
I'm sure there's many other examples.
Traditionally, the problem is that with money comes politics. Everyone wants a piece of this pie, and a lot of non-tech/dev employees are used to turning to vendors.
The thing is, what you want to achieve is a cultural change, not simply an organisational update (which is why, in the LH case, the tech hub is not located at LH HQ. It is not a cultural match, according to the CEO).
Does this change even require money, except salaries for the "right people"?
Big business have done IT & tech a huge disservice by outsourcing and off-shoring things not considered "core business", such as tech & coding. The pendulum is swinging, at least in my experience, and suddenly everyone need dev expertise again. Of course they do, because most likely you will be disrupted by a company without the legacy, that is sprung from tech.
Well, not a disservice to us still around, but it's obvious that theres a lack of expertise to just pick up off the streets.
How long you can keep a lot of people employed in a traditional IT dept. and buy a lot of half-crap vendor solutions that you glue together in half-assed ways I guess depends inertia of you customer base, and also as pointed out below, already accumulated wealth.
The other side is that a large company often gets large contracts. If you sell a software 10 million times it doesn't really matter if you spent 100x as much on it as a small company would. The small company with 10000 customers still is having 10x the revenue per work done.
Could you make more profit? Sure, but lacking market pressure you're rarely concerned with efficiency.
I just noticed my company always books flexible airline tickets. They cost about 2x as much as non flexible ones. I've never missed a flight and most of my colleagues haven't either. It makes sense if you're a busy manager that often doesn't fly. But for most engineers the flexibility costs more than what it's used for.
If they release the leash and let people book "extra" tickets when they miss a flight, next thing you know people will be taking their S.O. along on conferences using "rescheduled" tickets or having the company pay for vacation tickets and so on. An interesting unofficial benefit, not necessarily a bad idea if it can be controlled. However people always push the boundaries and next thing you know...
So it saves money in the long run to have one employee traveling equals precisely one (admittedly expensive) ticket. They can put their oversight labor into preventing more elaborate forms of control fraud such as using sales meeting budget for dating purposes or padding wedding receptions into corporate meeting budgets or all the nonsense that happens with "company" cars.
Also the labor savings of phoning it in by taking a week to approve tickets means it could be very expensive indeed to have a protocol that permits same day travel if you need to reschedule, all those tickets being approved at considerable labor expense same day just in case you need one once a year. Or paying you for a week to sit there waiting for tickets to arrive, etc.
A side dish is small companies have control fraud oversight as a small rounding error, but when you start thinking about an entire department to organize travel, now you need oversight of an entire department, leading to strange inefficiencies.
Have you read "Searching for Stupidity"?
Basically the NASDAQ of 1990 has almost no relation to that of 2000 not because of competence but of stupidity. Big lumbering behemoths shoot themselves in the foot and fall over.
That's because economies of scale work. They make it easier to get customers, they have easier access to credit, they make it cheaper to employ employees, they make it easier to have things go your way if there's a contract or legal dispute... The list goes on.
Economies of scale are why living in a poorly ran-community is better then trying to scratch out a living as a hermit in the woods. It's the same thing in workplaces.
It seemed to work for them.
I am back in finance at a private company, and I am part of our on-campus recruiting efforts, and I frequently tell candidates "listen, take a job with one of our direct competitors, that's fine, but whatever you do don't take an offer at an investment bank if you can avoid it- they will just work you to death largely filling tech debt and you won't get the opportunity to work with the truly best and brightest..."
How do they work? currently at an IB and unsure about continuing, everyone seems apathetic
Hell, in many situations in general it isn't that important, since knowing the right problem to solve and who to market the solution to often matters a lot more than the technical quality of the solution in question. People and companies generally don't judge products and services by code quality or development practices, and unless these cause it to constantly break, don't really care about them at all.
But other factors do matter here. If a company is a known brand, then they can keep getting customers for terrible products for a long, long time, simply because they're the first name people go to. At some point you become so big, you get so much momentum, that you can basically publish/release anything and people will buy it. Apple could sell the most broken phones on the planet, and have 10 million + people rush out and buy every one.
Do companies get away with it forever? No, but the bigger you are, the longer it takes for you to fail after your products/services become crappy.
And yeah, as you say, local optimums matter too. If most companies in your field are inefficient and put out terrible products, then there's not much pressure for you to be much better.
So, there is no contradiction. If anything, it might make more sense that adding people increases waste and stupidity. Remember that startup founders' ideas are failures over 90% of the time. These businesses started or grew with at least one good one. Then, they add lots of often-less-innovative people with their own ideas thrown into the mix. Of course many of them are going to be bad. Human nature, esp egos and image management, takes over from there adding the conflicts seen in the article.
I work at a F250, and I once asked my boss: "Why does the parent company allow our company to exist, when we seem to be steeped in incompetence?" He said to me: "Because we make a shit ton of money."
How that looks depends a lot of where you sit. If you are on the inside you see the lack of coherence up close. On the outside you see the collective power of the organization. There is enough coherence that the multiplier effect is still very positive.
My answer would be that while a startup seen in isolation can be very efficient, if we look at startups as a group there will be a lot of duplicated work. It's easier for a large company to solve a problem once and for all.
And this is the primary advantage a large org will have and compensates for many of their shortfalls.
As an arbitrary example, I think if we compare the number of employees onboarded vs the number of new applications shipped, small companies/startups and enterprises are polar opposites. My experience with startups is that you end up with a new web service for every 2-4 people (wholly anecdotal) you onboard as developers. For that kind of company, it makes sense to optimize shipping new web services; you do it a lot! For an Enterprise, you might onboard 100 developers just to maintain and develop existing systems. Writing new frameworks to make building apps easier doesn't make much sense; many of your developers are working on an existing system that would be expensive to port to a new framework. Meantime, your HR department is drowning trying to onboard 20 people a week.
Wasting money shipping this listing is fine; they did it once. This was released in 2013, and I wouldn't be terribly surprised to find out this, or some small bastardization of this, is still running as a production page for Prudence. The money they wasted, amortized over 6 years, is nothing compared to what I've seen startups waste in productivity while new hires wait for laptops, credentials, access, someone to explain some legacy crap that everyone had to deal with, etc.
Tech is an afterthought for most of these companies. Most of them are natural monopolies. Insurance for instance is a business that works such that the larger you are the lower the risk and cost. Take the ACA: many nonprofit startups tried to join the ACA marketplace to compete with the Big Boys. Essentially all failed.
These companies have so much money that they can afford to do things like 10 million dollar POCs (which I've seen). Often they have literally 5+ different teams doing the same exact thing and will pick the best out of that list.
In some cases (I won't say which) they essentially launder profits through the "tech" parts of the company. They basically optimize for cheap people they bill out to the "real" company at some enormous fee.
How fast this change is coming for a particular company depends on how inert the customer base is, and how much of the customer "journey" is digital.
For example, insurance -- I've worked at one of the largest (if not the largest) insurance companies in the world. Obviously they make some money, or turn over at least, but the customer base is usually pretty static.
What I mean is, you get an insurance, and you rarely change unless something drastic happens.
This allows for a big enough company to completely manhandle IT/tech/dev -- spend $100 of millions on project spanning years, and ending up with a lot of money in the pockets of "the usual suspects" and often software that does not work at all. All legacy cruft still intact, and just another layer of expensive crap that business users hate.
Still, the business of insurance is heavily tech & software dependent. So much, in fact, that systems down means “no business”.
I've become somewhat cynical with regards to the enterprise tech business, but I've seen these patterns repeat at too many places. It's truly sad and disheartening.
Ugh. Reading stuff like this gets me deeply sad; what could have been done with that money and time instead? It makes me want to go escape to live in the hills, maybe even a full-on Wonko the Sane.
You're assuming that an organization can whip out the best proposal possible at a moment's notice, which is highly unlikely. What you perceive as waste is simply development costs, without which the organization could not come up with the most competitive option possible.
Complaining that having different teams working internally on a product is wasteful is just like complaining that a sports team that hiring backup players and running development league teams is wasteful.
A lot of us are being paid handsomely, but are we _really_ adding value to society by any other means that just being consumer machines?
I find it interesting to ponder these things. =)
Research grants also fit that description quite nicely.
Can you explain this differently? I don't understand how you would launder profits through yourself (or why you're laundering them at all-tax reasons?).
Same as licensing your name to "another" company that's actually your company, for (coincidentally I'm sure!) all of their profit, in order to shift money around.
Have umbrella company. Put your tech folks either directly in the umbrella company or in some child company. Make your other child companies (or even ones you don't have directly related to the umbrella corp) "pay" the one with the tech folks for their dev work, some arbitrary amount of money (however much you want to move). Now the umbrella company or tech child company has all the profit from your other companies, which you've done for tax or liability reasons or whatever. Mission accomplished. Whether actual tech work was done may or may not matter, depending on whether you were also trying to accomplish actual work at the same time, or just shifting money.
I think it's more fun to understand how much they can charge for what they sell.
On the one hand big corporations are presented as these incompetent behemoths that are locked in their internal petty office politics, managers are clueless as to what is being or should be developed, and tons of money is wasted, they are wooed by empty marketing etc.
On the other hand these organizations are immensely wealthy and successful.
It may actually be that they are right, and for non-tech-centered companies this whole software stuff is really peanuts and their mental energy is better spent on other business-related stuff. So they will come up with random low-effort comments on all sorts of details because ultimately they (rightly) don't care. What actually matters is to negotiate deals like referral fees, or coming up with other contract ideas that will bring a lot of revenue in.
Or maybe it's just an equilibrium, a local optimum. You don't have to produce "good" stuff (in the developer's sense) without waste, precisely because the other company that you're targeting is also inefficient in similar ways.
I'm really not sure but I think it's important to look beyond just "haha, stupid managers, they can't make up their mind". There must be deeper reasons.