To set the stage, this is a blue-collar industry, and what happens, so I've been told, is quite common in blue-collar industries. A lot of good ol' boy networking, people getting promoted who have no business being promoted, and as a consequence, a lot of corruption, ineptitude, and inefficiencies. The software in this space just flat out suck and anyone at a FAANG company would be appalled at how we write software. It's literally longshoremen trying to run software businesses.
On the other hand, as a terminal, it's very hard not to make money, so there's a lot of players, and unfortunately a lot of, at least shady deals.
I'm involved with a company that sells to terminals that don't have home-grown software systems. I don't have that much info on terminals that have their own software staff, however, there just isn't that much breakthroughs in the space. It's a conservative industry with a lot of money already, so there traditionally have been little incentive with high risks.
In addition to the aversion to risk, there's a lack of software product and engineering strategy to invest in optimization properly. One major vendor, I was told by a colleague did invest substantially, failed, and "it almost bought the entire company down." My company has dabbled before. We've had a couple of PhDs on staff. We even had a neighbor search solution like in the article developed. It worked in simulation and testing, and failed miserably in real life. The customer didn't take it, and we never had the vision to actually salvage anything from the effort.
There are a few vendors in this space, but again, the landscape is pretty sad. I know of one that are flat out frauds. They exist because the CEOs used to be in the industry and have credibility. They're still around because they're able to con non-technical terminal managers and string them along for a while. A few more maybe have 1 or 2 competent data scientists, but that's not enough. I've worked with one company that is good and is easily the most qualified, but they have never had the marketing and leadership to make themselves the billions they should be worth. They also simultaneously price themselves out of range for most terminals.
The industry is changing, though, because private equity has gotten involved in many companies. They have taken over some internal initiatives and basically dragged many of these good ol' boys against their will, or forced them out. They've also forced companies to partner with tech giants to address these issues. IMO, this is the only way we will see supply chain optimization revolutions any time soon.
In short: It is not good enough. When it is good enough, customers adopt it.
I don't know how many meetings I've been at where the PE team blatantly states over and over, let us help you, tell us what you want to do, tell us what you need, and we completely flub it. Sometimes our CEO is too paranoid, sometimes too prideful, sometimes too oblivious, but mostly just doesn't have the knowledge on how to run a software company.
PE is going to run out of patience soon, and it won't be good for us. There's going to be headlines of how PE gutted us, but in reality, we gutted ourselves.
BTW the current state of English water companies is a not a good advert for private equity.
You have to realize that these terminals are typically a lot smaller in terms of manpower than you might think. A terminal handling 1 million TEU/year may operate with 2-3 crews in a 16 hour/day schedule for handling ships and trucks, plus some extra people handling the empties. Let's say it's 60 dock workers a day. A typically office might only employ about 12 people in overhead: one for customs/quality stuff, one for invoicing and finance, one HR, a bunch of order entry / customer service people, and 2 or 3 yard/berth/stowage planners.
Unless you look at the biggest of biggest terminals, you have to essentially regard them as SMEs in a very non-sexy business, and they treat tech and fundamental research into optimization accordingly.
All of the problems you can neatly divide and name are interrelated in practice, and trying to have algorithms "magically" figure it out for users is pretty much guaranteed to fail. This business eats software hotshots who think they can swoop in and "just" run some algos, "because it's just a TSP / Constraint Solver / [insert your approach here] right?". This business can definitely use bright minds, but start humbly, and start by talking to real world users please.
You don't appear to have any contact details on your profile, but if you ever would like to exchange thoughts with another company in this business, shoot me an e-mail. I think we're doing some interesting stuff for the <1 million TEU/year segment of terminals, especially if they're heavy into multi modal.
Some planning algorithm like this might work, for sure. But onpy if it is included in whatever software being used for this kind of planning. Otherwise, it is a nice academic excercise at best.
- German engineers balked at tightening features on pre-production vehicles because it would prevent their use for recreational purposes - Billions were spent in overtime in a healthcare setting, seemed trivial to improve scheduling to reduce overtime, but there were numerous union constraints and simply not enough hiring supply
I wonder how realistic this google solution is (Houthi missile constraints?). In my experience it's often more valuable to develop solutions that can easily adjust to unplanned changes rather than provably optimal ones
As an employee at a terminal, you have a) no reason to trust some software people who may have never seen one of them steel boxes up close, b) if this works, you or your colleague gets laid off, and c) if it doesn't work, you're now stuck cleaning up the mess the computer will inevitably make for you, which is way less fun than planning things yourself.
The 5%-10% invoices being incorrect is probably on the low end, I'd probably say it's 5-10% in total invoice VALUE. Because of the sheer amount of containers, you need automated invoice calculation. But if you miss an edge case ("We are allowed to invoice a 5 buck surcharge if the container is filled with at least 100kg of explosive goods, AND was loaded during the night shift because dangerous goods handling requirements now require us to have a safety officer on call, which we normally don't have during the night shift") you are leaving serious amounts of money uninvoiced, and you will never know. One of the reasons not that much effort goes into optimization algorithms is that there is so much low hanging fruit left on getting your invoices out correctly...
Also d.) It works sometimes so you get forced to use it and then it gets ransomwared and you're completely unable to work because they laid off so many people you can't fall back to manual operations.
You're also right about the complications and the need to be adaptable to various situations that arise. Some of that just comes from people who are afraid of getting their hands dirty and really understanding the domain they're working on. Operations Researchers and Industrial Engineers are not much different from Computer Scientists: sometimes the desire for a beautiful and simple solution prevents you from building something useful. Sometimes in order to build something useful, your idealized fast-converging LP turns into a QP/MIP bastardized rat king that only converges on an optimal solution half the time, but still gets a better-than-before solution the rest of the time. And some of us really hate that it has to be that way. The rest of us laugh their way to the bank.
From an academic standpoint it seemed a bit to little academic in presentation though, I would like to see the problem formulation! :)
I couldn’t look up his work exactly because our names are highly generic but if you’re interested, I can forward you his work.
Unlikely. I worked in this business and the idea of mathematical optimisation is always the first one to be suggested by someone who's new to this industry. Global shipping companies are clueless about tech and only do what they absolutely must to comply with the regulations, the rest is ignored. Give them an XML Schema for an EDI doc and they will stuff an Excel sheet into a CDATA. They know their game, they wore out the infinite patience of IBM who wanted to convince them to use blockchain to track shipments.
We recently met with a big tech company who we were told to partner with. They start with an angle that made highly suspicious they basically wanted to build a fully automated terminal. When we told them just one fairly mild story of what union issues we deal with, they all gasped and said, "what?" and quickly dropped it.
I agree with you that objective optimization is largely academic. There are always reasons why it's impossible or impractical to adhere to the most efficient versions of standardized processes. Sometimes these are "stupid" (people) reasons, and frequently they're reasonable justifications that accommodate for any number of externalities (weather, downtime, supply chain disruptions, lumpiness in demand signals and forecasts based on seasonality, etc).
That said, it's almost always smart to start with the most efficient version of a process and then add exception-handling, rather than to create a standardized process based on known exceptions. If you let exceptions become the rule, you'll always be operating less efficiently than optimal.
On the terminal side, all I can say is that I am knee-deep in container optimizations right now and it's a damn nightmare. Every terminal does things extremely differently, even if the terminal is owned by the same company. Even the terminology is often different within a company. You optimize for one terminal, and you have to build 80% of it from scratch for the next terminal. Any solution is incredibly difficult to scale.