A new general came in and insisted on using SAP. 6 months later, they had to rebuild both warehouses.

EDIT: For some additional context, the system managed to basically eliminate the "bull whip effect" all the way down the supply chain. It's really a fascinating system. Developed by Dr. Bill Kernodle and Dr. Steve Davis from Clemson Apparel Research.

It’s a really broken system that no one seems interested in fixing.

If you could hide the budgeting effects from anyone with the ability to 1. start projects or 2. add scope to projects, then it seems like it'd work out just fine.

(Of course, actually doing that would require splitting "planning" roles into separate, almost-adversarial "defining projects + increasing scopes for projects" and "feasibility-analyzing + cost-optimizing + decreasing scope for projects" roles. But that seems like exactly the sort of thing the DHS would be fond of. War-games for the project managers; and separate levers for both kinds of sitting presidents to pull on to satisfy their bases, where building up one capability doesn't tear down the other.)

That's the generous interpretation. There's also the case when project managers deliberately low-ball their project estimates to take advantage of the phenomenon. I'd put that in the trust problem category.

The same dynamic can occur with hedge funds, where the incentive is to increase variance (because high volatility pays the fund managers a lot more on average) or increase risk (example swing for the fences if underwater).

It is a trust problem because organizations generally aren't worried about losing money they don't spend that year or the next. They're worried about never getting it back. When you use a ratchet approach to budgets it doesn't allow organizations to respond to responsibilities which may be cyclical in nature. This leads to permanent worst case thinking which leads to hording.

edit: on the other hand: If I'm going to quit the job next year and I don't like my successor I'm definitely going to cut corners and budget.

We already know the answer to this, on average at least the system will be gamed to put money in their pockets, to overall detriment of service.

A market economy does not really help, that just means you don't have a budget to cut in the first place.

It’s more the opposite, that many people have a strong financial interest in maintaining the system.

Authored by Joseph W Kernodle of CLEMSON APPAREL RESEARCH.

I was a GA from 2003-2004.

Later I meet a former procurement officer who was reprimanded for excess purchases in the year leading up to the system launch. In the end his team was the only one able to provide equipment for an extended period of time after the new system was introduced. Everyone else had relied in JIT supply chains, but this guy didn't believe that the new system would work and spend a year building up inventory.

I do some Scaled Agile Framework consulting and the implementation a good example, because it can be fantastic but if leadership at the top of the company doesn't get on board the entire process will self destruct or be bastardized.

The professors who built the system knew this about every customer they talked to and it was a big learning experience for me. New leaders will come in and often insist on tools that they already know, rather than take the time to learn about better options.

If you've ever read Skunk Works (fantastic book), you see lots of this in the military procurement process in the US. It's bad enough that you worry how much it's hurting US national security innovations.

I've seen this with developers. Newly joining developers, particularly junior and mid career developers, tend to insist on using the languages, libraries, and techniques they are familiar with. Senior Developers don't seem to do it as much. Either because they are familiar with many things, or because they have been on the other side of these well intentioned suggestions to know when it is actually productive.

I do care about using the right abstractions and high-level interactions, not as much about the language or frameworks. I’ll let others put in their political weight on that, so that I can do it in the areas I believe will really make an impact to the business goals.

I saw it posted below. The system was/is called Balanced Flow.

For example, the solution may not have worked well with other procurement items and they wanted a single platform. Or the navy already had an SAP contract that could be leveraged without spending money on a separate stand-alone army solution.

So it can look like a bad decision that generated local inefficiency while globally it was a better choice. That's not to say SAP actually delivered on those promises.

Its always good to call out corruption, but its also important to realize when a situation is actually crazy complex and while its an awesome story might not be worth formulating a summary judgement from.

However, for an organization like the army that spends $$$ on logistics, I would bet they would continually fund something like this if it showed enough promise. One thing the DoD and Congress are good at is throwing money at R&D efforts.

In this case, it sounds like they should’ve bolted the JIT platform onto SAP

Basic business logic is built into SAP - but because it covers all aspects of business, it’s a massive tangle of rules.

For instance, let’s say you get an invoice from a vendor. Now you have to update your balance with that vendor, and your general ledger. You have to record the new inventory moving in. Perhaps the inventory has a barcode, or a serial number with a warrantee date, or a batch with an expiry date. The inventory movement will be tied to the cost of that invoice - remember to take into account those exchange rates. Also, there’s a variety of ways to account for item cost. You will have had one or more purchase orders - those have to be closed. If the vendor is to be paid in a different currency, you have to do every accounting entry in your currency and theirs, and record the exchange rate differences. Your accounts payable are now in that other currency, which means that has to be updated all the time for accurate bookkeeping. You have to note which employee was the buyer. This buyer has targets and metrics to be tracked. Perhaps there are reports and alerts set up for them. You will have a budget, which may impact this transaction. Of course you also have master data for the vendor, the employee, and the items bought.

There is a lot more to this than the above, and this is for the simplest business case - one that happens hundreds or thousands of times a day depending on the business. Usually there are special considerations to program into the above as well. Now imagine what happens when a complicated situation arises, where you have complex transactions spanning months, many countries, and many other businesses. Sometimes there are legal rules built into the logic. Usually those rules only alply to one country. And I haven’t even mentioned freight or tax or landed costs from secondary invoices.

SAP is a ball of wire because it simply has to be.

If you built a new ERP, you will end up the same.

I developed a custom engine for things like aluminium rails, and thermal sensors, to generate a new product with the production tree in SAP, so it would be able to decrease the right ammount of raw materials from the warehouse.

It does not make sense for Amazon to run there own shipping fleet... until it does.

The other point is that armed forces tend to have a mandate to control the supply chain, this might be a case of that.

The Army isn't a business and should never be run as one.

Probably not. SAP is an everything-to-everyone product. Using Fred Brook's terminology for essential versus accidental, the problem with everything boxes is that even just the essential complexity of such a product is so insanely large that they are sufficient to produce a "horrid" product. Of course they don't have just the essential complexity and add a healthy dose of accidental complexity, but so would any putative "less horrid" replacement by the time it successfully solved the same essential problems as SAP.

Closer to home, "bug tracking" is an example of an everything-to-everyone product. I can't even count the number of times I've seen a "simpler, less horrid" bug tracker get started due to the perceived horribleness of the current bug tracker, but the new bug tracker simply became the old one once it tried to grow into the same niche. This may sound strange to 2022 ears, but I remember when JIRA was being pitched as the simpler solution and developers were pretty keen on it. There will never be a mature bug tracker that is any fun to use, because by the time it's everything to everyone it'll just be the same morass of being everything to everyone again. It turns out that the "less horrid" bug tracker wasn't actually intrinsically less horrid... it just wasn't everything to everyone. Instead it was a bug tracker for developers, so developers love it. But then if developers are going to be allowed to use it everywhere, it has to satisfy all the other stakeholders too, and it inevitably mutates into an everything for everyone product.

(See also: Salesforce. Letting users configure custom DB schemas is a key indicator of being an everything to everyone product. To some extent, programming languages too; there's a lot of people who pine for something simpler (such as the people who think that if we just went to visual languages, or tried to jam everything into "no code") and don't understand that by the time you have a general purpose language that is truly general purpose you've got a large amount of irreducible essential complexity whether you like it or not.)

It can work for a long time, but eventually it does create opportunities for competitors.

You're absolutely right that there's a floor for how simple these products can be, but most of the incumbents aren't anywhere close to that.

However, I do believe a (slightly) better competitor could be created, with better technology, better workflow, etc. If well-capitalized. If it had well-factored plugins. If you had a top-notch team to design all that and then build it.

That's a lot of ifs. Joel says it isn't guaranteed a brand new team will even do better than the old on the essential angle. It's risky and costly enough to prevent it from happening.

https://www.joelonsoftware.com/2000/04/06/things-you-should-...

Experience probably has already taught you some of these ideas, but you hadn’t invested the time to reason about them and putting them together. Then you read an old blog post, from two decades ago, putting all those ideas into words and it simply clicks in your mind.

Thanks for sharing the link.

SAP and Jira are just not great software. They’ve got excellent sales teams, connections, and probably great support. That’s the key to getting user-hostile software into big orgs.

But the most likely scenario, I think, is that built-for-purpose competitors will build products that peel away small pockets of SAP's customer base over time.

The irony of course is that a product like SAP is the end result of similar built-for-purpose products being hoovered up and merged together in the first place.

Everything is a pendulum!

[1] https://www.cbinsights.com/research/craigslist-unbundling/

The requirements change, merge, flow. In theory each department within the enterprise could have their own optimized, silo'd tool ... but then you need "enterprise integration platforms" to glue together all the workflows between them.

https://www.thirdstage-consulting.com/the-biggest-erp-failur...

Larry Ellison had a famous quote from some Oracle ERP event where he chucked every Oracle ERP consultant under the buss by saying: "It doesn't do everything you want, but it does everything your business needs". He was referring to custom workflows business take on which require massive customizations of ERP platforms, increased consulting costs and a painfully hard upgrade process.

Switching costs and risk are the major reasons these systems stay around forever. If you've got something that works. Even if it's sub-optimal, you're never going to move away from it. I worked at a place that used an ERP from a company that went out of business, so they would install DOS emulators for a really ancient version of DOS on every computer for people who needed access to the ERP. This was seen as a better solution than the perceived risk of switching ERPs. This was 10+ years ago, but to my knowledge they're still doing this.

As you say, the switching cost is way too high and dangerous. They moved billions and billions of dollars daily through a highly complex structures and complex laws in different jurisdictions across the USA. The business logic was built over the 40 years when the world was DOS. It's not like they are dumb and don't know linux or Windows exist and all that.

They brought in huge consulting companies all the time to see if they could get it upgraded, but no programming house, no consulting company would touch it. These consulotants turned down hundreds and hundreds of millions of dollars, because 1) it was just too f-ing complex, and 2) if they messed up, the liability would be immense because of the dollars daily transactions, and what the dollars were used for. It would be an absolute shit-show of the first order if things were to go sideways. I'm positive that these consulting companies E & O (errors and ommissions) insurance company would refuse to insure them. Doing it wrong would be catastrophic on a corporate, but also national level. It would not be a matter of some stick-it notes from 3M not getting to the stores on time, or shoes getting to the shoe stores.

So, DOS in 2020.

Apparently Stanford had its own homebrew system, and some dean had decreed that they had to move to Oracle Financials. For literally years, these poor people struggled with it.

I never worked with that or SAP so I can't compare them, but I think both are the Full Employment Act for consultants.

Apps are what organizations/people buy. Databases are just the storage engine.

The difference is that Oracle underwrote their apps with database profits and they never got the apps game.

Most organizations have horribly convoluted organizational structures and processes, and SAP has been shaped by those customers over the years to cater to those clusterfucks.

A clean, sensible ERP system can only work if the organization it serves is also sensibly organized. It is much easier (and less risky) for a company to choose an ERP system that caters to them, warts and all, rather than overhaul the way it conducts business.

As the old saying goes, SAP and its customers deserves each other.

Now what does this say, about LIDL, a discounter who dumped 500 m with SAP?

https://www.computerweekly.com/news/252446965/Lidl-dumps-500...

Basically, if your buisness process match that of SAP, everything is fine, if not, good riddance. Or in the words of a former SAP manager about Lidl: “They don’t have the right level of business engagement”

Meaning they did not adapt to SAP enough.

For every "We had a great internal platform that we custom built" story, there are multiple "We used a small vendor's product, and it was a nightmare."

Why is SAP successful? Because the general purpose tools (or worse, amalgamations of multiple single-purpose tools) that came before were absolute shit, and SAP is instead only difficult to use.

Progress!

(Also, a TAM that is literally every company, ever)

The people who are hurt by it are not the people deciding to buy it.

SAP looks great for Csuite executives when it produces reports. For an engineer trying to enter their hours it's an hours long torture session every two weeks.

They had evidently selected one approved hotel to demonstrate the process and never bothered to go further.

The hotel was some four-star facility 40km from the office and twice the price of the economy tourist-tier hotels 5km away (which also featured free wi-fi and breakfast). So the remote workers continued to stay at those hotels and had to waste effort poking the system to do an override.

Fortunately, they still did it as "pay with your own account and we'll reimburse you" so you weren't stuck waiting for someone to approve a "can I pretty please spend less money and avoid an extra two hours of travel time per day?" request.

At my prior job, we were trying to do marketing for a SAP consultancy firm, and it was just surreal trying to get a grasp on it. How will potential customers know they're ready for SAP, or ready for your services in particular? Where do you reach out to potential buyers? What sort of messaging are you trying to target? I never got anything resembling an answer. I think they basically just wanted us to write the HTML for the terrible email blasts they sent to their existing lead pool.

After a thought, I suspect there's a particular corner of the C*O universe that treats Gartner reports as some sort of Death Note: a supernatural document where once someone's name appears in it, their entire destiny and fate is fully programmed out. I think they pretty much believed they could ride being in the Magic Quadrant for Nasal Hair Removal Advisory Services in 2006 into golden retirement.

Beyond that, just the philosophy of creating a "purchase order" to pay a UX researcher who did 10 hours of work over three months seemed goofy -- purchase order is what I think of when doing something like ordering printer paper, not getting an invoice paid.

How can you pay an invoice for a service / item you haven’t purchased? If you buy a service from a supplier, you raise a po, you pay an invoice based on the po. As a supplier, I always ask for a po.

And yeah, there's a lot of paperwork but maybe you're hiring this gal because she's your sister. Maybe she's really good. Or maybe you're just funneling some money to a family member. As companies scale up they need controls for a lot of things even if they're legit 95% of the time.

And, again, not a software issue. It's a perhaps necessary bigco process issue.

https://www.oracle.com/applications/

(Microsoft uses SAP as an ERP).

(This was also my motivation to start my own financial reporting startup: what if we designed a system in the year 2022 to be able to support big businesses without it being as awful as big business software.)

Furthermore, AFAIK Microsoft started using SAP well before they acquired Axapta.

It seems like it, but I don't think it is possible. Sure you can clean up SAP itself a bit, but most of the problem is custom UIs that companies develop only to the point where they are useful, but then they decide it is cheaper to train people on how to use the clumsy UI they have instead of paying developers to clean it up. They probably are not long, it only takes a few minutes to train most people on the how to find the few things they actually need, and the slow workflow costs them a couple hours per person per year (and most of them are low wage employees), while cleaning up the UI would costs several man-years of programmer effort.

Now they are migrating to the cloud, and loosing customers as they push them off their on prem s0lutions

Clayton Christensen has some nice youtubes about disruptive innovation where he predicts the death of SAP

Customers are growing increasingly frustrated, the company has dipped its toe into political agendas and they regularly get lit up on Twitter as a result, and Bloomberg has targeted them due to questionable moral practices (most recently, silencing rape victims).

SAP is more concerned with protecting their reputation than actually developing quality software. Sooner or later that is going to catch up with them. They are constantly playing damage control. It's only a matter of time before more and more comes out (and there is plenty to come out).

What it turns into is another story though :-).

Personally, I rather liked the approach that Oodoo takes: https://www.odoo.com/

It tries to be pretty modular and even the cloud hosted plans can be mixed and matched, in addition to which the apps can be self-hosted for free: https://www.odoo.com/page/all-apps

Of course, attempting to compare it to something like SAP is a bit like comparing OpenProject with Jira or something along those lines: where one of the solutions is way more popular and recognizable, even if the other could be serviceable in certain scenarios. It's pretty clear which one will be picked by most of the enterprises, though.

What did you find is a mess about Odoo? And why is customizing it a nightmare?

It would have been better to integrate an existing system instead.

My experience when evaluating Odoo was using Studio, which I have found can really mess up your Odoo instance.

Are you working with Odoo code directly?

SAP is the "European" way of doing things, which means there's one right way and you need to align your business to SAP's way of doing things. If you do that, things work very well.

Peoplesoft is the "American" way of doing it, which means that you make your software fit around you. This means a lot of customizations. This makes integrations much more challenging if you're too far off kilter, and things like upgrading is much much harder, because there's too many customizations.

That's the information I had about 10 years old and not sure how much SAP has evolved since then.

Smaller companies these days have a lot more options, like Netsuite is already there for small businesses and Workday is moving in that direction. Startups come and go every day. Rippling appears to be one that is headed in this direction as well.

Upgrade windows aside, it's ok. I imagine that the sweet-spot for Netsuite are businesses that do 5MM to 100MM ARR. Possibly less if there's a heavy need for EDI, volumetric shipping, etc.

I used to work at a company building software that competed directly with SAP products. The IT people in our customers’ organizations were the hardest people to convince to adopt our software. The end users loved our stuff because it allowed anyone at the factory to build forms and share data. But the IT folks would always get in the way.

“Nobody got fired for choosing SAP.”

Competing with them is super hard because of that.

SAP will fall from a thousand cuts of the facade pattern.

Now that's one simple process imagine that on every single step. SAP is worst software I've ever used.

Alternatives do exist. Well, one does - Oracle ERP. Migration is indeed long and close to impossible, and it’s very much out of the frying pan into the fire.

We've spent 3 years refining our processes and automating our ERP system, both the data structures we need and the design of the UX. No thank you SAP

Yes, it does it correctly for tax purposes everywhere and can be used in a way which conforms with how these companies want to do their accountings for shareholders reporting. Also it used by so many companies that every niche thing you want to do has probably already been done.

The people here complaining about the UX for reporting their hours just have no idea of the mind boggling complexity of the problem space. There is a reason everyone use SAP and it’s not because it’s bad.

A few years ago, I was working for a small, family owned industrial company. They were in the process of moving all of their ERP stuff over to SAP. Things were slow with some of the UI/UX stuff I was working on and someone asked me if I wanted to learn how to be an ERP developer.

Before I was shown anything, the SAP guy pulled me to the side and told me, "Fates, I know you could probably write a DB program significantly faster than what you're about to learn, but this is just the way the software was written."

Cue several of their SAP people dropping random stuff in my lap and me taking days and sometimes weeks to discern how to do it in their UI. It was the most maddening thing I've ever tried to learn. I just remember this graph paper like grid and having to line everything up perfectly or the program wouldn't run at all. I felt like I was working with something from the 1970's, it felt that out of date.

I finally told my boss I couldn't do it anymore, it was driving me mad.

On the flip side, I play hockey with a guy who is an SAP sales person and makes hella money selling their software and thinks its the greatest thing ever.

I don't know about SAP, but one of the main reasons we're adopting Retool internally is because our engineering team keeps deprioritizing writing internal operational tools in favor of adding features for out customers, yet our operations are becoming increasingly more complex. Something like Retool lets our end users (our own staff) be able to work with it.

In a lot of ways, Retool is more like an MS Access, only it is a cloud native SAAS and from the get go, work with integrating existing databases. Writing about SAS, I think it's clear Retool has been thinking about how to grow with the organization, possibly where organizations can stay with Retool instead of trying to implement SAP. I'm also curious about how they are going to work with the increasing shift from "data lakes" to "data mesh".

Fresh out of college, with a full two week (!!!) training on the product, I was sent to a customer place, to build some forms for them. Apparently they were not important or big enough for senior people, so I had the privilege (!!!) of building forms using our software for this client. I spent two days building the perfect form, then suddenly all the layout was lost. I couldn’t recover the layout, I call my team. They casually say our layout engine can be finicky sometimes and will refuse to work if you annoy it too much. No one told me though.

Thankfully I found another job soon.

Many of these so called enterprise products are terrible. But they’re so entrenched that it is next to impossible to beat them. They’re also expensive and their business practices dubious.

For many companies it would be easier to do an asset sale of all their IP than to replace the ERP.

Since most ERP implementations are horrendously overdue and over budget, they are usually declared Done before they’re actually finished. That leaves a lot of crud and manual processes in their wake.

In addition, so many of these systems are on prem using hardware, operating systems and databases that haven’t been supported in decades.

I’m getting nightmares just thinking about it!

I'm sure their goal was to eventually replace SAP. Instead, I think they ended up with an 80% solution that requires the customers to invest 20% as much time as a full SAP setup.

For instance, Salesforce spends a lot of effort on sales and customer facing things. SAP does that too, but it also does logistics. Logistics infrastructure is much harder to reuse across industries than sales and customer support.

True. Making something with a less sadistic and convoluted UI is the easy part. Migrating the data is the hard part.

Many companies died from updating ERPs.

They protect and take care of a lot of problems, the issue is they're so flexible that sometimes people build ad-hoc things on top of them and over time it becomes a ridiculous monstrosity similar to any other system in a company with major legacy.

There is no inherent solution to this problem other than disciple (or control, which has it's own productivity issues).

A lot of things developers must build in, and decisions developers make, are built in already for these systems.

more likely situation would be the companies stuck on SAP themselves dying and SAP eventually dying as a result of that. Obviously that's on a long term timescale, but smaller companies not burdened by legacy technologies winning is pretty much the story of modern business

It's probably also too boring for any tech entrepreneur to even try, but disrupting the space would seem to be the white whale of startups. You never know. At one point in time nobody had a computer on their desk either.

Isn't that where Salesforce came from? At least in the CRM space. It is not as horrid, though that is not saying much :P

And yet somehow last time we had this discussion on HN, more of than half of the developers have never heard of SAP.

Sorry friend, that is not a joke - at least this was attempted once! :)

These go hand in hand

I have a family member who has had enough professional success with SAP; when I mention projects there is a specific response he has that relates to this.

tl;dr- If you change your processes to follow the SAP happy path where they don't you are OK. The more you go out of bounds the more problems you have (be they problems of kafkaesque workflows, or paying the bills to customize it while the scope of customization keeps changing.)

I don't know how true that is, but it's there.