One vendor is having trouble finding any chip to design for. The software stack supports a couple of dozen STM32 chip series, but none of them are even remotely available. I've seen a lead time of over 11 months. Before COVID, pretty much every single chip was in stock almost all of the time.

Respinning a board isn't too difficult, but good luck doing that if there's no chip to respin it for.

A year ago you could buy 10k+ units of most common parts -- in stock -- from each of the major distributors.

Some people hoard toilet paper, some people hoard baby formula, some people hoard Shanghai apartments, some people hoard microchips...

Got burned with same STM32s recently. A purchaser been shopping parts, and ordered a given model of MCU by muscle memory. The price has moved one zero overnight, and we ran for quite a sum.

That is not sympathetic, that's very normal.

What you're describing is sympathetic and hard to deal with. Their example was lucky about it and still ended up somehow quoted in the story.

However, their software sucks (okay, so do all of the other options...), their peripherals are not as good as some of the other options (in particular I'm thinking of peripheral clock trees when I say this, among other things), and their availability is the _worst_.

Seriously, unless you're big enough that ST management actually knows your name, expect to have sudden availability issues with the STM32. This has been true for the last decade. It will remain true. (Many of these availability problems start at ST's wafer fab, so they're not shared by the other vendors.) I always advise clients who care about such things to consider other vendors.

Now find a single person who can program bare hardware on a short notice on above Arduino level.

The current chip shortage has claimed 6 of our firmware devs, all hired by companies ready to spend just any money for anybody "who can replace that ____ing STM|NXP|Renesas thing"

People downstream in the industry greatly overestimated their knowledge of the industry.

Lots of tech companies around who had zero prior knowledge of embedded development, now jumping on it, and breaking their teeth.

I haven't heard more fabulous questions like "Are there other microcontrollers than Intel, and AMD in the world?" this year than any time before.

Wait, there's real money in that?

Maybe I shouldn't waste my time just doing yard work.

Sidenote: I just asked my wife about parts shortages at her workplace. Evidently you guys are right. I should get out more.

Firmware engineers are the out-of-sight-out-of-mind component in the product development spectrum.

Although the money has been decent, programming complicated devices has spoiled me for one thing..I have just about essentially zero interest in web technologies, products, languages. I've always gotten to work on the sexier version of embedded work and so feel really blessed, it's not all about the lucre.

Which is the saddest part of it all.

But a thing to remember is that people currently hiring themselves been likely neglecting their RnD before the crisis hit them.

Condolences on the loss of coworkers.

But this seems interesting to software people who haven't been doing embedded, but who can wrangle C well enough to adapt (quickly?) to embedded constraints and interfaces.

It's knowing how low level hardware operates and knowing how to dig yourself out of the various gotchas like for example figuring out that a last minute replacement of the crystal oscillator by the HW team with a cheaper one is now screwing up your boot process or that the reason your products are starting to go haywire after 6 months is an interrupt triggering too many writes to the flash, wearing it out prematurely, etc.

Sometimes being an embedded dev is dead easy. Other times you are dealing with a GCC chain from 2003. 2 layers of some 3rd party of libs that someone bought in 2005 because the VP was good friends with a buddy in some other company that made something similar (which you may may not have the code for). With 64k of flash and RAM. And the vendor ground off the chip numbers and put their own sticker in place. Playing detective, historian, and making good guesses as you may not even have access to the firmware anymore. Then on top of that sometimes you are really lucky and have a debug port and can play the printf logging game to see what is going wrong.

Are you in the US? Because EU is full of embedded devs.

It looks easy to find STM32L0x2 chips in stock. Those are not supported by the QMK firmware, but they are very similar in terms of peripherals and features to the supported (and hard-to-find) STM32F0x2 lines. The main differences are a lower minimum operating voltage and more power management options, IIRC.

What you are suggesting is quite impractical, unless you are a really big business with cash, which can simply direct order components.

Small businesses, even in Shenzhen, a place inundated with supply chain abundance, always have to either keep running from one small wholesaler, to another searching for components, or pay n-times the price working with somebody like Arrow.

In my experience, you can't safeguard yourself against such things as a small company no matter what.

I worked in, and around OEM electronics since 2007, and things like having to redesign a product 4 times a year to accommodate a supply chain disruption were happening even in the best years. Nothing special with the current disruption besides the scale.

This is also the reason why Asian electronics makers have such short product lifetimes. It's usually easier just to sunset a product, and make an improved, and better version with newer components, than to fight against the always evolving supply chain. And I not talking about small companies, ASUS, Acer, MSI, and such all practice this.

I know few people running the Chuwi brand. What they do as a small maker is that the moment the get a good consignment of chipsets, and other parts, they spin a laptop model solely for that batch alone.

Then, they live off it until they get another good parts purchase, when they usually sell their component leftovers, or do few final runs if they can find people wanting to buy them for rebadging.

The entirety of small volume laptop industry spins around chipsets, and screens — hardest things to find for a small company.

I want you people to take a looks on a big difference in how companies in the West, and here handle the crisis: Western brands wail, cry, and wait for component supplies to resume, while Asian brands just keep releasing new products with parts coming into their hands, and making great cash from this shortage.

The caveat is that because MCUs are not standardized you frequently get screwed over so my replacement part list is a collection of versions of the part with different amounts of memory that are designed to be pin-compatible. Sometimes that isn't enough, and often companies aren't big enough to negotiate a guaranteed supply. But if you're not big enough to negotiate a guaranteed supply you just have to deal with buying enough of the parts to keep you in business long enough to create a new revision before you literally run out of stuff to sell...

I sympathize with the other comment berating me at length for being flippant about this, and am genuinely sorry for being flippant especially in light of their experience with the opposite issue (buying stuff and getting screwed by customers canceling orders). But I was talking about a dog washing startup. A dog washing startup vocally complaining to the news about needing to respin one board because of a supply chain interruption.

Yes, there is FCC and other testing you have to do if you significantly modify a board, so I'm not sitting here oblivious to the challenges involved in a respin. My response was to them being a poster child for the people suffering from this issue. Those of you who cannot find any component that will do the job have my absolute sympathy and I apologize for not being more verbose.

This is just not true. But when it happens, there is usually an announcement well ahead of time (e.g. 1 year), as well as a last time buy option.

Historically, there has never been a need to buy the chips of all the boards of a revision. If you are in the position of being able to buy all the components of all the boards you'll ever make for a certain revision, you're probably just above hobbyist level.

I think this churn in parts is hurting innovation since it makes things very difficult for small companies just starting out. It also wastes enormous amounts of human effort redesigning boards and rewriting software. People who could be inventing new products are tied up rehashing the existing ones. The cost to the world must be enormous. I do not understand how anyone could think this is a good thing and a reasonable way to run things. It is completely insane.

I've seen this happen with everyone from ST to Microchip to Atmel (when it existed) to Qualcomm to Analog Devices (though interestingly never Texas Instruments so far); this is just something that happens...

Thank you for the clarification, my initial comment was flippant and needed like four paragraphs of caveats.

Let’s take something totally ordinary like an STM32 MCU, and let’s ignore for a moment that there are now some clones on the market, with questionable compatibility at times.

Even though they’re unique parts, there is no way anybody sane would stock up on those for the lifetime of a product revision.

It was never needed and it’s ridiculous to plan for a black swan event like the one we’re experiencing now.

And here’s why: even if you had planned for a sudden shortage of an STM32, you’d still be screwed on some generic components. Because I’ve seen people get stuck recently because they couldn’t source certain generic diodes.

It makes no financial sense to always plan for the worst possible case. The whole reason distributors exist is because they are the buffer that moderates spikes in supply and demand.

The system has worked very well for decades. It’s much better to be right or wrong along with everybody else than to be wrong 99% of the time (and waste margins compared to the competition), and being able to say “I told you so” to the rest once.

And that 99% is not hyperbole.

I have this growing gut feeling we are at the peak of technology... at least for a while.

I’m not a hoarder, I’ve kept my focus limited and goals clear.

But you’re an assumptive a*hat.

Interestingly I saw an analysis a while back (citation not provided because I'm about to go sleep and have an early start tomorrow) that just the change in demand from office to home could account for a lot of the increased toilet paper demand. People were still shitting the same amount but they were using home-grade rolls rather than industrial giant office rolls, of which there was a glut!

I still say “stock up” though at least for what you can actually plan to use. Just like the above with toilet paper, supply-chains are going funky.

Companies hate holding on to inventory. Most companies now don’t hold on to any extra stock needed to manufacture their products.

This is great for your finances in normal times. But don’t complain when there is a shortage and you literally have zero slack for delays and your factory sits idle — that’s the well-known drawback.

Not holding inventory is great for a lot of reasons, but it is a calculated risk...Toyota learned from experience that if they want to do it, they need to understand risk all the way up their supply chain. They largely aren't having this problem. They de-risked certain things because they really treat TPS and everything associated with it as a philosophy not a set of heuristics that should just be implemented blindly. Manufacturing something is all tradeoffs...there is (almost) nothing with a universal upside. Sure I want to hold less inventory, but JIT is actually about manufacturing time, not inventory. If I have NO inventory and NO ability to get inventory my manufacturing time goes up while I wait with my thumb in the fertilizer pipe.

I've worked with a couple of manufacturing plants (and consultants...) that treat 'JIT' inventory management as something that can simply be pushed off to vendors and then the upside of less inventory enjoyed. They have specs and forms and certification and paperwork...but nobody looks at it. Those are the companies struggling now. They outsourced without fully understanding the risk of the outsourcing. Usually this doesn't bite the world, it bites one or two companies that relied on a certain part (someone misses an EOL notice) or a certain vendor (who goes bust because the owner's grandson ran the thing into the ground). It's like my students who make choices in the first week of the semester that seem minor...and then are frustrated when it effects their grade at the end of the semester (sorry, too much grading this week)

[0] https://www.autoblog.com/2021/03/09/toyota-how-it-avoided-se...

Wouldn't toyota model essentially set a required buffer size in kanban, essentially ensuring a certain level of stock decided upon by risk management?

The pull flow would then ensure that the stock fluctuates around that value instead of being driven to 0, because taking a "card" out of stock might trigger purchasing of more stock?

The question isn't how stock is tracked or risked, its who is responsible for it.

For example this report from March 2020 was highly influential and predicted 2.2 million deaths in the US in an 'unmitigated epidemic' scenario:

https://www.imperial.ac.uk/media/imperial-college/medicine/s...

When in reality the official COVID death count in the USA was 580,000. And meanwhile the best public health response seems to have been... to do nothing, given that Sweden has a lower COVID death toll per capita than countries which did lockdown and use masks, and is overall merely #27 in the world for COVID deaths per capita:

https://www.statista.com/statistics/1104709/coronavirus-deat...

With deaths there in 2020 being only 6% higher than 2018 (and deaths in 2019 were 4% lower than 2018, suggesting mortality displacement explains much of that increase):

https://www.statista.com/statistics/525353/sweden-number-of-...

These sort of mega-flus come and go once a decade (look at 2009 Swine Flu, 1993 Flu, 1988 Flu, 1976 Flu, the Asian flus of the 50s and 60s - all basically forgotten):

https://swprs.org/wp-content/uploads/2020/10/sweden-monthly-...

But there were political factors in 2020 (a major re-election year in the USA) which drove the completely unusual and unjustified lockdown response. Its unfair to blame JIT manufacturing.

You just said yourself that the 2.2 million was the "do nothing" scenario, and as we've seen in India it could've easily been that. You also just said said we did many things — "unusual and unjustified" things, in your opinion — therefore mitigating that worst-case projection. I personally lost the point you were trying in these self-contradictions, but I am curious where you were going.

For that we need to look at all the data. People have done that. The predictions were always completely wrong in every case. Here's an article that examines the track record of that group at ICL:

https://www.aier.org/article/imperial-college-predicted-cata...

Re: "do nothing". Both ICL and a separate Swedish team applied the Ferguson model to Sweden and got a prediction of around 90,000 COVID deaths. True total: 98,000 deaths of all causes in total, and that was after 2019 was the year with the lowest mortality on record. Age adjusted, Swedish excess mortality matches that in 2012.

That's one famous example but again, all-data analysis shows the same thing. Lockdowns don't seem to affect the course of the disease:

https://thelancet.com/journals/eclinm/article/PIIS2589-5370(...

Epidemiology has developed a culture of bad science, presenting unvalidated models as indisputable fact and worst of all, pathological lying. Ferguson was recently caught doing it again:

https://www.telegraph.co.uk/news/2021/05/04/uks-covid-reopen...

As pandemic revisionism goes, it was really quite extraordinary. Asked on BBC Radio Four’s Today Programme about the latest data showing Britain is enjoying an eight-month low in coronavirus deaths and infections, Professor Neil Ferguson said on Tuesday: “The data is very encouraging, and very much in line with what we expected.” As it was on the radio it was impossible to tell whether this was said with a straight face, but we must assume it was ...

... Prof Ferguson’s team [had] warned: “A return to higher transmissibility levels after non-pharmaceutical interventions are lifted will also lead to a third wave of hospitalisations comparable in magnitude to the current wave” and called for mask wearing and hand hygiene to continue after full lifting. Under the February modelling which informed the roadmap, hospitalisations should be starting to tick up around now, but there is no evidence of that happening, with cases down 11 per cent in the past week ...

... The team warned there was no way out of a devastating third wave, with modellers arguing that even the slowest release in August would result in “substantial additional deaths” of around 56,900 by June 2022 ... One of the main errors in much of the modelling seems to have arisen from a misjudgement of how much schools would impact the reproduction number

The predictions of epidemiologists, not just Ferguson, were wrong by orders of magnitude and wrong in a way that has caused global supply chain disruptions. We should care more about the effects they had on restricting access to healthcare, because those actually killed people rather than just disrupting electronics supplies. But this story is about the latter. That's why it is wrong to vote down someone speaking the truth about the root cause: exceptionally bad science and our societies apparent inability to detect it or reckon with it.

I'm very interested to understand what part did the US 2020 election played in the (still continuing in some parts) lockdowns in Europe...

This guy is like Nostradamus compared to the people at NASA doing project life estimates for the rovers.

This one is obviously caused by setting "bare minimum" to "actual expected average time".

Mission exceeding minimum plan is much better that very expensive hardware failing before supposed end of mission.

We have already re-spun PCB designs to work in alternate ICs for some of the ones that we know will become unavailable; dual-footprint in case the originals come back. This took time and effort to design and verify and all that work could have gone into the design of new products, but it had to be done. This has put us some number of months behind our original planned dates for our new products.

The problem is that we can see shortages coming up which we cannot work around. We cannot buy enough parts, the lead-times are astonishing. At some point we are going to have to get lucky or halt production.

You cannot support right to repair if you support the idea that chips go obsolete.

* the model was setup while the auto makers were substantially smaller. GM / Ford couldn't hope to control the entire supply chain in the 1930s like they can now. These pre-existing businesses are able to protect their interests.

* Early cars had massive defect rates compared to modern consumer electronics. You HAD to be able to have a large number of people able to repair and maintain cars.

* Due to the value of cars, a higher proportion of damage is likely to be considered repairable over a CE device.

* Many auto parts were simpler when the model setup. Many parts did not have extensive IP protections, and were relatively easily clone-able. (i.e. you can make a mould of a part and cast other copies). Not to mention, outside a few halo proprietary technologies, companies were less litigious and protective of IP.

* Commercial operators such as Taxi and corporate fleets mean that there was a market for these 3rd party products, enough to cover R&D costs of cloning an existing part.

I think we need to enforce a compulsory licensing of IP for all parts, chips, etc after a period of non-availability or obsolescence. Price should be set at a point where 3rd party companies come in and produce generic versions, paying royalties to the owner. Allow these 3rd parties to reverse engineer the products and sell their clones if there is truly a market need.

What will likely happen is that the companies would rather license their designs to a chosen partner, in the hope of having more control over it

Now, they want to do it even quicker.

DRMed autoparts are a new craze. A tsunami of them is coming in 2021+ cars.

They all now want to do the the John Deere trick. They gave the industry a very bad example to follow.

I can't go into more detail than that.

Well, I can think of at least one easy way to avoid that problem. If you document your hardware and put a leash on your lawyers, you'll find that your customers and their independent service organizations will be glad to take on this responsibility for you.

https://www.digikey.com/en/products/detail/xilinx-inc/XCVU13...

... as soon as my board hits production. If they give me any grief at DigiKey, I'll have my VP of Procurement fly up there in the company G5 to give them a piece of my mind.

...

> if you aren't buying enough chips to build all the boards you want to build of a revision

Friendly advice:

If you are going to be a consultant to industry, don't post comments like this.

As someone who has been manufacturing tech products for over thirty years, my first reaction to your comment was "this guy doesn't have a clue". Then I looked at your site and was absolutely floored. My guess is you have lived in what I like to call the "SBIR distortion field", which is a domain that is very, very far from the realities of, say, a dog washing company. Not just because of usually just having to make one or a few of something (rather than 10,000), but also because of the financial dynamics of these programs --I have experience in that domain as well.

Your vision of how this dog washing-machine company should operate does not align with the realities of a business outside of the "SBIR distortion field". Companies don't have cash reserves to fill the warehouse to the brim with components and product, weather a storm, keep the business afloat and everyone employed simultaneously. On top of that, manufacturing at any non-trivial scale is such a cash intensive endeavor that cash must be managed very carefully. If you buy too much inventory you can end-up in financial dire straits.

The phase lag between spending money to manufacture a product and getting a return on that investment can be in the order of months, and that assumes a "linear" market. If you include R&D in that equation, it's even worse, years.

I experienced this personally back in 2008. I did precisely what you suggested above and filled the warehouse with some two million dollars in components and assemblies to get ready for sales of our new product. We had demand. In fact, the purchase of the components and assemblies was triggered by receiving a purchase order for five million dollars of this product. And that was just one customer. I didn't know better. I thought it was perfectly sensible to place large PO's for critical components that would cover us for at least a year and tool-up. We even bought a bunch of brand new CNC equipment to bring manufacturing of heat sinks and other mechanical components in-house in order to reduce our cost basis. In fact, interestingly enough given some of what you have on your site, I made the single largest purchase to date (at that time) from Osram's high power LED division. No company in the world had ordered that many high power LEDs from them.

And then the music stopped.

The economy came to a grinding halt.

Sales went to ZERO.

The five million dollar purchase order? They went insolvent when their bank cut-down and eventually cancelled their line of credit. Other orders from major companies were put on hold (we had a PO but were told they were not going to accept deliveries, so, don't ship). We went from having tens of millions of dollars in orders for that product and that year to, effectively, zero.

What was the end result? It was very rough. All of our cash was in the warehouse, on shelves, as components and assemblies we could not sell. We couldn't even get a loan to weather the storm. Nobody was buying anything, not at scale anyhow. We had to sell some of our component inventory for ten or twenty cents on the dollar just to bring in cash. It was worthless.

I had to take a second mortgage on my home and use credit cards to make payroll (big mistakes, both of them). We survived for two years on bread crumbs. And then I had to shut down the company. It too me years to even be able to talk about this episode of my life to anyone. It was horrible.

The two millions dollars I spent on "buying enough chips to build all the boards", as you put it (it was more than chips, but the example fits) was the single biggest mistake I have made in my business career. And this one cost me a business I built over a decade, starting in my garage with $5,000 to receiving a $30MM acquisition offer just as the economy took a shit (the offer was rescinded).

So, please, pretty please, with sugar on top, if you want to be a consultant, don't say anything unless you really understand it. In this case, you clearly do not. To someone like me --who has actually lived through many ups and downs in life and business-- such comments result in what I will call "less-than-favorable conclusions" about the author. This isn't good for a consultant, unless the consulting is in a domain that does not necessarily align with reality outside of something like the SBIR/academic domains.

It took years to recover, both mentally and financially. I eventually launched a new business, also in tech. Today we are facing having to manufacture 10K to 20K units per month of a new product. When we started design we picked readily available components and went on to design the product over about twelve months (real product design for scale manufacturing takes time).

Today, as we approach production requirements, we are being quoted anywhere from 40 to 50 weeks for some of the components. In other words, we can't even buy them. We are having to consider having multiple alternative designs to see if we can manufacture functionally equivalent versions of the hardware using different chip sets. This means all of our regulatory and safety testing --another thing you ignored-- (FCC, CE, TUV, UL, environmental, thermal, lifetime, etc.) has to be redone, not once, but likely four to six times (depending on how many versions we end-up with). It's a nightmare.

And, no, buying a million chips a year ago wasn't the solution. The cash drain would have resulted in people losing their jobs and possibly even going out of business again as sales levels last years went down some 80%.

You buy as close to just-in-time as you possibly can. This practice has gained acceptance over the years for a reason. Sadly, I happen to have learned the lesson the hard way. If you have to weather a storm it is far better to have cash in the bank than a warehouse full of worthless components that you can't turn into cash precisely because of the storm.

"A man holding a cat by the tail learns something he can learn in no other way". --Mark Twain

So true.

Second, that was a great post and I want to read your (I’m sure imaginary) blog. It was like a mini business education in modern manufacturing.

Finally, I am amazed you’re able to write this with no bitterness. Hats off to you.

You don't go through something like that without the emotion staying with you. Yet, if you are going to move on you have to be able to put it in a drawer and only look at it every so often just to make sure you are not going to do something dumb again. As time passes you have less time to make mistakes like that.

You've highlighted exactly the value and difference experience makes. People born at the crest of the wave can only take for granted their position until they have lived enough to reflect.

Every order we received was like precious molecules of much needed oxygen. We got this order from one of our resellers (we had about fifty all over the world at the time) for about half a million dollars in product. This needs to be in the right context: I had just taken out nearly all of the equity in my home to keep the business going and took out a bunch of cash on all of my credit cards, personal and business. I had already been to the hospital once due to stress and dehydration (I managed to do that twice in a year). A half million dollar order felt like a billion dollars.

We had product. We shipped it and awaited payment in thirty days. That's the other reality, it just takes time to convert components to money.

Almost precisely thirty days into this cycle FedEx freight shows-up with a shipment. Our reseller returned 100% of the order we shipped a month earlier. All of it.

I called the owner of the company and unloaded on him. At the end of the call I ended-up having to thank him.

You see, they were going down in flames, just as most of us were. He was at the point where the banks forced him into bankruptcy. He knew that within days people were going to descend on him to take inventory (and possession) of everything under their roof.

He sent us our hardware back because he actually care for us and did not want the bank to grab hardware he had not paid for. Like I said, I had to say "thanks" and wish him luck.

I can't remember if we ever got another order of that size between that point in time and when we closed our doors.

For example, we sell into markets like automotive, aerospace, and medical. Being a startup we have basically zero leverage in how to go about conducting business with large well entrenched enterprises with business development dynamics that were calcified decades ago. Part of managing my business is accepting and working with the risk profile of having to keep the company solvent long enough to actually engage these customers in the ways they're able to be engaged. I'm not going to be in a position to make demands that they conduct business significantly differently with us relative to their hundreds of other vendors regardless of if it would ultimately benefit both of us to do so. There's an amount of inertia in any status quo that needs to be overcome, and the problem with that is that the party with the most motivation to displace that inertia is also the one with the least power to do so. That reality gets baked into our capitalization and operations strategy.

I'd love to be able to demand that automotive OEMs actually cover the cost of engaging in a PoC with them which isn't going to have any real payoff for months or years, but every single other supplier they have eats that cost just like we do, and betting my company on the incredibly low probability that I'm going to displace the pandering that they expect from their supply-chain all by ourselves would be crazy.

There's also the reality that every business is on a 30, 60 or 90 day phase lag from delivery to getting paid, and so they have no choice but to enforce those rules up the supply chain. If you don't you need piles of cash upfront months before you generate any revenue, at scale that is really tough to manage and there's a very real cost to money.

The simplest example of this I can offer is that if you have to borrow ten million dollars to pay all your suppliers upfront and this money cost you 1% per month (making the numbers simple for the sake of an example), you are going to incur a 5% cost of money if you have to wait five months to get paid (again, keeping numbers simple).

I have a friend in the production business who made commercials for a major animation studio. He told me it typically took them about six months to collect. They would invest massive amounts of money on equipment and personnel to shoot, edit and deliver a commercial and their payment would not come for six months after delivering the end product. The entire cycle would easily have taken a year.

This war was fought decades ago. Just-in-time production won, and it was a decisive victory. This chip shortage is rough, but nowhere near as rough as it would be if we weren't doing things the way we are right now. Everything that has happened has happened for a reason. Attempting to disrupt this will put you in way over your head in ways you couldn't imagine.

Not so easy. This is particularly true as you start to get into higher dollar amounts. Also, it tends to be far more common with international orders than with domestic business. I can say that nearly 100% of our international business was prepaid. Sadly, during the 2008 downturn, all business came to a halt. There were very we places where you could find income that could sustain the prior state of business.

In the case of the the five million dollar contract I mentioned, we did get a $500K deposit with the order. Well, the $500K was spent on components pretty much as soon as it hit the bank, within a week. It's very hard to escape something like what happened in 2008 if all your cash in in a warehouse filled with parts and product you just can't sell.

Huge mistake, they should have bought Johnson instead.

I am glad you found the courage to share a bleak chapter in your life, and for being unflinchingly honest. I hope it was cathartic ─ your lived experience will serve as an extremely valuable lesson for those of us, who might encounter such circumstances.

Yes, it is cathartic to some extent. And yet I don't really enjoy talking about it. It still hurts.

That sharing is so damn rare, and I think a lot of people end up in a really bad place because we don't do a great job at teaching the failure states of business.

So again, thank you. Life willing, you sound like someone I'd be thrilled to do business with.

This is one of those things that makes hardware businesses so darn hard and something software-only startup folks just don't understand. The marginal cost difference and phasing of money you need to support, say 10K SaaS clients vs. shipping 10K non-trivial hardware products can be massive.

In my case the company was 100% bootstrapped. In retrospect I should have gone for investment as soon as we started to take flight. Frankly, I was too busy gasping for air (money) and absolutely overloaded with work to even consider it. Any investor type I spoke to was going to suck time and resources I simply did not have. So we kept going. Had it not been for the 2008 economic downturn we would have had an amazing exit.

> That sharing is so damn rare

Frankly, the experience was at the limit of darkness for me and sharing was nearly impossible for years. In December of 2009 I wrote a friend an email where, among other things, I said "I now understand, in no uncertain terms, why people jump off buildings or walk in front of trains during hard times". He was knocking on my front door within 15 minutes, after breaking the sound barrier driving from his office to mine.

No, I wasn't thinkin of ending my life. Not even close. It's just that the darkness I was facing at that moment in time produced a clarity of understanding I had never had before. I felt that I had full understanding of how someone could make that kind of a decisions. I was simply communicating the revelation I had. I can see how bad it must have sounded.

But the Ontario added a luxury tax and sales tanked. It looks like they had over three million dollars in excess inventory. And soon the company went bankrupt, to this day I think the company would had pulled thru if it did not have the extra money in the bank instead of inventory they could not use.

Every time I read such comments I reflect upon how terrible of a job we are doing in teaching young people about finance, business and entrepreneurship. In the US most high school graduates have no skills to offer other than, perhaps, stacking boxes and making coffee (after some training). I truly don't understand how people are happy with this in a so-called "developed" nation. We launch young adults into the world and they know so little about it. That's where such comments come from. How could you blame them? The entire country failed them by not ensuring their education includes perspectives and knowledge that allows them to deliver value to potential employers.

Well, i cant speak for others, but when i talk about company greed i think of practices like using fake cheese to cut costs but hide it from the customer.

Or when they break their product by update and tell you to buy the bigger one. (Like Synology recently) Or to intentionally slow down the product by Update (like Apple) and lie to your customers about doing it.

Or drying up the well from villages(Like Nestle).

Or demanding tax returns from taxes you never paid (Cum-Ex).

Those companies give a shit on society for each dollar they can get no matter what.

that is greedy and in my opinion very wrong.

Every time i read a comment about how someone defends that i reflect upon how short sighted or antisocial a person must be. Those people have probably no skill beside locust style investment and stealing lolipops from little babies. We launch young Business Administrators into the society and they know so little about how to actually be productive. That's where such comments come from. How could you blame them? The entire Society failed them by not ensuring their education includes perspectives and knowledge that allows them to deliver actual value to economy or society.

I tend to react badly to broad-brush painting of business and those who run them as greedy and evil. There is a cultural undertone that seems to think this way. I can't understand how this happens other than to think that people who think this way (broad characterization) simply don't engage in any critical thinking at all and don't understand business.

The vast majority of businesses, large and small, are comprised of honest hard-working people who have no ill intent of any kind. The percentage has to be in the high 9's, like 99.999%. If this were not the case it would be very evident.

Large corporations can behave badly due to the power they can wield. A simple example of this are companies that have entire floors full of attorneys and can muscle little guys into submission by simply being able to outspend them in legal jousting. I have been at the receiving end of this and it is nasty. You are entirely powerless and can't do a thing about you. The asymmetry is beyond evident and, yes, very much unfair.

This, in my opinion, is a structural failure of our legal system. I am not a lawyer, so I can't really dissect this down to details. I just think that "equal under the law" depend on how much money you have to tilt that equality. I don't know what could be changed in order to achieve balance.

A simple example of this could be patents. A good utility patent can cost in the order of $25K to $50K to secure and take years. A large company has the resources to write hundreds of patents per year. Small to medium businesses are generally more innovative and creative than large organizations with lots of inertia. However, they are often starved for cash, which means they have to choose between innovating, paying the bills, keeping people employed and existing or dumping cash into patent after patent. This creates a situation where small to medium businesses end-up living in a very real legal mine field of patents that could, at any time, take them out. Beyond that, patent litigation --any litigation-- is so expensive that almost anyone has to cave immediately.

Yes, we could do a lot better in ensuring fairness and socially responsible behavior across the board. Not sure how we get there when our political system's fitness function is completely disconnected from delivering anything of true value to society. As long as politicians are evaluated through a fitness function with that consists of votes and not much else, we are not going to have leadership who cares about doing anything other than lying and pandering for votes. How do we fix anything when these are the kinds of people running the nation?

There are companies like Sun Microsystems, and then there are companies like Oracle.

There are companies like Gandi, and then there are companies like GoDaddy.

There are companies like Fastmail, and then there are companies like Google.

I prefer to look up to the first group.

One of the things I have grown fond of forcing myself to say or think as a way to center myself is that reality can't be reduced to a single variable. Reality is a complex multivariate problem. As such, a reduction to a single variable, is both irrational and unrealistic. No matter the issue, there's a lot more under the surface than a single variable, issue, property, behavior, etc. A lot more. The sophisticated thinker will recognize this and try their best to avoid the kinds of sweeping generalizations that are, sadly, so common these days. Entire groups of people negatively painted with a broad brush is probably the most recognizable for of this effect today.

Look at it a different way: Back then I thought what you are proposing was sensible enough that I spent two million dollars to execute precisely that strategy. I ended-up losing a business that I built over ten years because of that decision at a time when it was the worst decision one could make.

In other words, if I called you a fool I would be calling my younger self an even bigger fool. I actually believed it enough to effectively destroy my company and affect my life for years. I am not calling you a fool. I am sharing a lesson I learned the hard way and simply warning readers not to assume they understand reality without the benefit of experience. Sadly some of this stuff we only learn after the fact, not before. I can't blame you at all for not understanding it.

EDIT: If there's emotion in my tone, please forgive me, ten years later and this still hurts. The experience put me in the hospital more than once and nearly cost us everything, we were horribly close from losing our home and everything we built over decades.

I did intend the comment to be about the dog washing startup that I assumed to be a fairly small business. Not buying the (presumed to be in the 10,000 quantity range) MCUs they needed ahead of time, knowing that they will be the single linchpin chip that there will be no pin-compatible replacement for, is what I found to be ridiculous.

I hope you find that perspective to clarify my intent some.

No need to apologize at all. This is conversation. We all have much to learn.

Today, what you suggested is precisely what I do. I try to make sure there are at least three pin/function-compatible chips that can swap in for any given device. Preferably from different manufacturers. I also talk to distributors to get a sense of volume. I prefer to buy devices and components that are being manufactured and stocked in larger quantities. A silly example of this is that it is much easier to find a 47 uH inductor in stock than a 50 uH part. One has easy substitutes, the other can turn into a nightmare.

As for buying 10K microprocessors, again, that can be a tough decision to make. On the financial front, you could be talking about a $50K to $200K expenditure before you sell any product. In terms of logistics, if I have $200K in microprocessors in stock and I can't buy RS422 drivers I can't build a product. Which means that the decision of locking-up cash in the warehouse can quickly turn into a nearly all-or-nothing proposition. In other words, if you are going to stockpile microprocessors you might have to stockpile another $500K in parts in order to ensure that the investment isn't worthless if there's a shortage.

And then there's the issue of what you do with your nice pile of components if nobody is buying anything. As 2020 has proven, if you are in the wrong category, you could literally sit there for a year without selling much. That's what really hurts when you locked-up a pile of cash in the warehouse. We have a client who's business went down 80% last year. They had to shrink from 50 employees to three. They had to further shrink from a 100K square foot facility to a 22,000 sq ft building. And business is slowly crawling up. Had they made a huge cash investment early last year they would have been out of business by now.

In the electronics manufacturing business you have at least three tiers of manufacturers.

One is the super small shop that just sends everything out to contract manufacturers, along with parts they purchase themselves.

The next is the small-to-medium shop that graduated to having the CM provide parts. In other words, you design your product and fully trust your contract manufacturer to handle the supply chain. CM's will work with distributors to stock components and build boards. There is no way CM's are going to stock components clients don't need just to be sure they have a supply for a year's worth of boards. The only components CM's might stock in large quantities are parts others are using that are low cost. A simple example of this might be resistors.

The next level is a case where a manufacturer has enters into a contract with the distributor and the CM to have "bonded" inventory. They commit to buying a certain quantity of product --no matter what-- and, in exchange, the distributor and CM will inventory enough product to meet the demands of that contract. For example, you might commit to manufacturing 10K LED bulbs per month and need to ensure a supply of, say, half a million LEDs. You sign a contract and this happens. The advantage of this approach is that you are billed as product is delivered rather than for the entire half million LEDs you bonded. Of course, you are buying 10K bulbs per month. It's a machine, once it is set in motion you have to meet your obligation.

The next level might be manufacturers that do their own in-house assembly. I've lived in all of the above categories. The in-house assembly case can give you a lot of control and even lower your COGS, but you are now paying for everything pretty much upfront.

Once you start adding other component classes (mechanical, optical, etc.) things get even more complicated.

Each of these models has a financial formula associated with it. I have no idea where CCSI (the dog washing machine guys sit). My guess is it isn't a high volume business. I would further guess they make boards in batches of 100 or so (I could be wrong). When you don't know a pandemic is coming and the world is going to come to a halt, buying enough to make 100 boards a month is the right decision. If someone suggested they should buy enough to make boards for the entire year it would not sound like good advise unless the cost basis of those boards was such that it materially affected profitability in a significant way.

Business has become so competitive and fast that everyone pretty much ends-up adopting a JIT (Just in Time) manufacturing methodology. Anything else is suicide.

Here's another take: Do I invest money parking components in a warehouse for a year --just in case-- or do I put it into marketing, R&D and new product development? I think I can say that, under normal circumstances, it would be irresponsible (as learned the hard way) to park it in the warehouse. No crystal balls.

As someone else in this thread mentioned, I too wish there were more documented stories of business failures. That's where the real lessons for all of us lie.

Usually telling someone to "get a blog, dude" is an insult, but in this case it's a real shame that the experience you're relating is going to be buried and forgotten in an ephemeral HN thread. Have you considered sharing your experiences and thoughts on an actual blog, perhaps in conjunction with whatever business you're engaged in nowadays? That's one page I'd bookmark for sure.

It kind of ties into Steve Jobs comments on consultants versus people who have to live with the consequences of their decisions.

Any details on this? Media/tradejournal links?

I wouldn't have believed you six months ago, but hey, everything's changed. Still, copper-clad board shortages... I mean if that really is for real we're headed back to the stone age real quick. I hope it's just a rumor.

but sure

https://www.linkedin.com/pulse/crisis-printed-circuit-board-...

https://elmatica.com/portfolio/copper-foil-shortage-persists...

Scores of recent articles at your fingertips.

Also: I wasn't berating anybody.

He bought in two units to fix to have in stock since he already had a unit running his greenhouse.

I asked him why use Vic-20s when there were a number of controllers already on the market. He said the problem he never knew when the present unit may fail and the Vic-20 did the job of an $10,000 controller which he did not know how long it would take to get the present controller to work on his system and the present one works fine.

It has been decades since I did the work and I would not be surprised if the system is still working today because he was prepared.

If it's a big news story people are focused on it. And humans mistakenly attribute events to what the are focused on.

Trump was a great scapegoat for local, city, regional, state politicians.

All kinds of things can be blamed on climate change, systematic racism, etc. Basically anything that a leader can't fix but is in the news is a great scapegoat.

Not to say that these aren't real issue. They are, but once you hear how they are over attributed as causal you can't unsee it.

Companies always piggy back on the current crisis to make changes or blame issues on that crisis that have nothing at all to do with the actual crisis...

come to think about it, governments also do this.

This was my thought watching the video in the article. It went on and on about supply shortages, China, and the Biden administration’s talks to prevent shortage as well as Chinese competition, and then near the end of the video, two little details slip by so fast you could miss them: auto makers cancelled their own orders, now they want their place in line back, and the biggest chip maker (TSMC) is in Taiwan, but it’s really close to China, so it still counts.

Yes, preventing supply shortages and paying attention to national security are good things. But neither of those would address the specific “shortage” behind this article & video.