News for hackers. Stuff that matters.
Re:Shopify to lay off 10% of workers in broad shake-up by stephc_int13 on Tuesday July 26, 2022 @09:50AM (#32238239)
There is something I don't understand about the way those big companies work.

From what I can see they are providing software and services.

Even the biggest software teams I have seen were rarely larger than 50-80 people.

And they were working on large complex codebase for AAA games.

Of course you need support, admin, HR, and sales, and R&D on new projects, but still.

How can they manage to productively use the time of 10000+ people?

What are they doing?

Re:Shopify to lay off 10% of workers in broad shake-up by pc86 on Tuesday July 26, 2022 @09:57AM (#32238333)
AAA gaming is probably the worst possible example you can look at for what a well-run, sane team would look like. So let's say if you only want your people working 40ish hours a week and not 70-90, you'll 2x or 2.5x that count. Shopify also has a lot of dev rel folks even if they don't call them that - people writing engineering blog posts, working on tutorials, documentation, etc. They're typically not the same people actually working on core code day to day. My guess is all the infrastructure and security people at Shopify are non-devs. You'll also have all the devops people who are managing all the pipelines and everything. My understanding is Shopify deploys to prod many many times a day (hundreds?). The pipelines to automate that, and automate or at least manage the rollbacks, would be at least as complicated as the code itself. And across all these different areas you have that amorphous "Staff/Principal Engineer" level where you're doing a lot of reviewing, pseudo-management, that sort of thing. Not to mention managers, directors, VPs, SVPs, etc.

But you ask how they can productively use the time of 10k people and I think this post is sort of evidence that they couldn't.

Re:Shopify to lay off 10% of workers in broad shake-up by stephc_int13 on Tuesday July 26, 2022 @10:07AM (#32238458)
I understand the need for support around a complex and constantly evolving codebase, but this is the scale of it that I can't grasp.

Software is supposed to scale almost for free, they don't need people to actually make the transactions and push the bits by hand, this is all automated.

My guess is they have a lot of sales and an enormous amount of overhead.

According to Price's Law about 100 people are doing half of the work, but of course it is difficult to guess which ones and they still need the other half to be there.

Re:Shopify to lay off 10% of workers in broad shake-up by neon_electro on Tuesday July 26, 2022 @01:36PM (#32241278)
>Software is supposed to scale almost for free,

Sure, that may be true, but startups don't suffer for lacking of scaling ability.

Software does not scale with changing requirements for free. Full stop.

Re:Shopify to lay off 10% of workers in broad shake-up by golergka on Tuesday July 26, 2022 @01:04PM (#32240923)
The more your software touches real money and merchandise, the more time you have to spend to make sure you're not breaking the law in any country you're operating in. It's not fun, it's very error-prone and it's very important to get right.
Re:Shopify to lay off 10% of workers in broad shake-up by wollsmoth on Tuesday July 26, 2022 @10:32AM (#32238808)
Hard to say without knowing what they're working on. Maybe there's a lot to do with integrating other software/services. Maybe they've got a good devops org to help smooth the process for everyone. Sometimes having a few more people than you need on many levels makes the org a lot more resilient to individuals leaving so the org can keep on rolling without a hitch.
Re:Shopify to lay off 10% of workers in broad shake-up by kill_nate_kill on Tuesday July 26, 2022 @10:43AM (#32238976)
If your system doesn't have any slack in it, how can it respond to change? Having a slightly underloaded system is definitely the way to go, it leaves you with some gas to change direction if you see that you're barreling towards a cliff.
Re:Shopify to lay off 10% of workers in broad shake-up by marcosdumay on Tuesday July 26, 2022 @12:31PM (#32240457)
Well, the transition from 2k to 10k people isn't something done to add slack to create a flexible system. For that you at most double the number of people, you don't multiply it by 5.

They must have changed something either on their product or on their procedures on those 5 years.

Re:Shopify to lay off 10% of workers in broad shake-up by wollsmoth on Tuesday July 26, 2022 @01:23PM (#32241136)
My guess is they decided to start writing more tools internally instead of relying on external vendors to respond to their needs. Again, hard to say. How much did eng grow vs sales/support.
Re:Shopify to lay off 10% of workers in broad shake-up by oblio on Wednesday July 27, 2022 @04:50AM (#32248240)
> My guess is they have a lot of sales and an enormous amount of overhead.

That's not overhead, it's business.

Software people think that they're crucial. They are in the sense that the thing has to be built.

But if nothing is bought, the business goes under and the software is scrapped.

"The overhead" makes the business work, makes people buy it and continue using it, helps them when they don't know which buttons to push, which integrations to use, etc.

Re:Shopify to lay off 10% of workers in broad shake-up by conorcleary on Tuesday July 26, 2022 @10:25AM (#32238696)
Customer support and new-build implementation tied to marketing is a large chunk of the people hours.
Re:Shopify to lay off 10% of workers in broad shake-up by acatton on Tuesday July 26, 2022 @10:40AM (#32238925)
> Even the biggest software teams I have seen were rarely larger than 50-80 people. And they were working on large complex codebase for AAA games. […] How can they manage to productively use the time of 10000+ people?

I work for a 100k-employees company. According to the internet, a third of it are engineers.[1] I think I can explain.

First of all, I think the AAA game example is bad. Most of my acquaintances working in the gaming industry work long hours for a crappy salary compared to their skill, because most of them are passionate.

Most business software is written by overpaid engineers who see the whole thing as a 9to5 job. So you can already divide the amount of hour worked per employees by at least 1.5 if not 2.

Even then, if you take something which seems as simple as Shopify the system can be made complex for managers and directors to justify headcount.

It's an e-commerce website, right? So you can split 100 engineers over 5 teams. 10 "production engineers" (or SRE/DevOps/SysAdmins) and 10 software engineers for each team: - Product search team; - Payment team; - Authentication team; - Admin UI team; - CRM team.

Once you have 100 engineers, you can multiply your headcount by 10 by just splitting each of these teams into five more. If you take the Payment team, you can create a credit card team, a paypal team, an alipay team, … For product search, you can create a team to manage just the suggestions of the search bar, another team which optimizes term matching with products, another team which does the list of related products on the side bar, …

I just found jobs for 1,000 engineers, right at the top of my head! Now you have so many engineers and so many customers, you also need internal tools. You need an internal tool to manage customer refunds, customer support, etc… You need HR tools for vacations, performance review, …

Of course all of this comes with waste, but who cares as long as you're offsetting it with your revenue? ¯\_(ツ)_/¯

Anyway, this is why I will never be a business person :) .

[1] https://earthweb.com/how-many-software-engineers-does-google...

Re:Shopify to lay off 10% of workers in broad shake-up by rekrsiv on Tuesday July 26, 2022 @02:23PM (#32241919)
I like this perspective a lot.

I just got laid off from a ~1.5k-employees private company that applied the same method to split up a large monolithic cash cow (crafted from scratch by a small clique of hackers who met early in life) into microservice and microfrontend internal products because it looked like a good long term plan in Q2 of 2020 and we thought we could scale that forever and start selling our dog food as a service.

Something went wrong, and we hit a ceiling very early. The amount of waste that was generated by the rigid management ecosystem that was created to sustain such a large scale up (for tracking purposes) became so strict that any and all attempts to use the scientific method to solve anything by the then-outnumbered engineering staff became impossible to justify to any non-engineering roles.

If the task didn't fit in a 2 week sprint, it couldn't be planned.

If it can't be planned to be shipped from scratch in a 2 week sprint, from analysis by both your BE and FE devs to both of them shipping at the same time by end of the sprint, it couldn't be done.

If you needed to solve for "what do people wish they could buy?" instead of optimizing for "what are the most humans currently using?", it couldn't be done.

We constantly regretted not sticking to our engineering principles.

The system collapsed. A lot of nice-to-have-but-working-at-a-distance positions were eliminated, as well as a few individual contributors that management hadn't realized were important to their core functions but were hired too recently and didn't have time to adjust to the point system.

I wish I could tell you what happened next, but the layoff happened. Most of us thought it would happen because management would finally realize the (human) system was designed poorly and without a good feedback loop while growth happened. But it turns out they might have only reverted to a previous commit of the organization structure and are still intent on trying again with the same rulebook.

Slower but infinite growth is still the objective, excessive tracking is still the norm.

Their marketing had been doing poorly in recent years, and the layoff wasn't even mentioned in the news, but it seemed like it affected a lot of devs in my neck of the woods despite the near complete lack of media coverage.

Does this happen often? Is this what scaling up always looks like?

Re:Shopify to lay off 10% of workers in broad shake-up by specialist on Sunday July 31, 2022 @08:19PM (#32300250)
Paraphrasing, from memory:

Happy marriages are boring; They all look the same.

Whereas each unhappy marriage is uniquely unhappy.

Re:Shopify to lay off 10% of workers in broad shake-up by hoten on Tuesday July 26, 2022 @03:56PM (#32243156)
Not even to mention the impact Conway's Law suggests! As a company grows in number of specialized teams, you'd expect to see magnifying difficulty in having efficient communication, eventually impacting what the product looks like (or at least, how robust the code is and how easy it is to augment). Seems like it could be a bad feedback loop - code got more complex indirectly because of number of teams, so naturally you hire more people to maintain the same output.

See this video by Casey M. https://youtu.be/5IUj1EZwpJY

Re:Shopify to lay off 10% of workers in broad shake-up by SoftTalker on Tuesday July 26, 2022 @11:05AM (#32239346)
> as you're offsetting it with your revenue

This is the problem. IDK about Shopify specificially, but a lot of these companies don't have revenue, or it's far from offsetting their expenses. That was OK as long as more venture funding was readily available; today that's not the case.

Re:Shopify to lay off 10% of workers in broad shake-up by acatton on Tuesday July 26, 2022 @11:30AM (#32239700)
I believe that you're biased by HN, since a lot of the discussion here revolve around over-valued VC-money-roided-up startups burning millions of dollar per months.

If you're looking at big tech, telco, banks and finance, there are a lot of companies where headcount waste is offset by revenue. Many of which are even making profits. According to wikipedia, shopify had a net income of ~$3B for ~$5B of revenue in 2021.

Re:Shopify to lay off 10% of workers in broad shake-up by beambot on Tuesday July 26, 2022 @02:29PM (#32242012)
SHOP: $4.8B in trailing 12-month revenue, growing 70%+ YoY for last few years and $250M in positive free cashflow.

They're likely predicting eCommerce growth slowing a bit (combination of post-COVID & reduced consumer spending), so pushing for better fiscal discipline & improving net margins going forward.

https://finance.yahoo.com/quote/SHOP/financials?p=SHOP

Re:Shopify to lay off 10% of workers in broad shake-up by petercooper on Tuesday July 26, 2022 @05:11PM (#32244074)
I just found jobs for 1,000 engineers, right at the top of my head! Now you have so many engineers and so many customers, you also need internal tools. You need an internal tool to manage customer refunds, customer support, etc… You need HR tools for vacations, performance review, …

I think the complexity demonstrated in your post is a reasonable explainer for why companies like Salesforce and SAS are so highly valued - it stops a lot of those teams needing to exist at companies for which proprietary tech isn't a competitive advantage.

Re:Shopify to lay off 10% of workers in broad shake-up by nostrademons on Tuesday July 26, 2022 @11:14AM (#32239498)
Your example is pretty illuminating. AAA games are one-and-done: you ship, and then the codebase is basically frozen, and you make money off expansions. The other area where team sizes top out at 50-100 people is desktop software, where (barring future versions) it's also one-and-done. You sell a software package, money comes in, that's it.

The way the calculus works for Internet-based SaaS is: say you have revenues of $4B (like Shopify), and an engineer costs $400K. If the engineer works on a project that improves sales by 0.01%, it's profitable. If a team of 10 engineers works on a project that improves sales by 0.1%, or 100 engineers improving sales by 1%, it's profitable. How many 0.1% opportunities do you see in the product? How many 1% opportunities? What if you add risk? An engineer might fail to see any effect on 4 of their projects, but if they improve sales by 0.1% on the 5th, they've still made the company twice as much as they cost.

It's hill-climbing. There are usually lots of 0.1% opportunities available with a big worldwide SaaS. Things like localization and entering new markets will generate way more than that; even things like moving around CTAs and changing colors will usually generate more in profit than the engineer costs.

Also add to that diminishing marginal productivity of each engineer. In a team of 5 moving a button around takes an afternoon. In a team of 1000 it often takes a couple quarters, because you need to get sign-offs from all the teams that the code touches, run it through QA, and gather metrics to show you didn't regress anything. The engineer is still profitable, because the effect of their change multiplied across millions of users is way more than their salary costs. But they're not as profitable, because they work so slowly.

Re:Shopify to lay off 10% of workers in broad shake-up by stephc_int13 on Tuesday July 26, 2022 @11:50AM (#32239938)
I've also seen this kind of operations in mobile games.

At Supercell, they have some old titles generating around 400K of revenue per day, with a minimal team working on new features, fixing bugs, looking after numbers and trends, managing ads... All of that with less than 15 people.

Of course this is not the same scale, but they have a LOT of daily users, and at their peak the company was about 200 people.

And I've been there, I can tell you that there is ample slack, most of them are barely working.

Re:Shopify to lay off 10% of workers in broad shake-up by majani on Tuesday July 26, 2022 @04:25PM (#32243506)
Mobile gaming has been extremely commoditized though it's still possible to get a billion dollar company out of the sea of copycats
Re:Shopify to lay off 10% of workers in broad shake-up by rvba on Tuesday July 26, 2022 @01:13PM (#32241014)
I understand what you mean. But then why cant the big tech hire support staff (looking at you Google).

Probably lots of money could be made with better support.

Microsoft seems to do it thr correct way - I worked at a "boring" company that had a lot of licenses and we had highest tier support - we found a bug in Excel (!) and they fixed it.

If you are a small fish you wont get any support and often not give money. I wonder if this doesnt add up.

Re:Shopify to lay off 10% of workers in broad shake-up by specialist on Sunday July 31, 2022 @08:25PM (#32300298)
Hostility to (human) customer support is baked into Google's cultural DNA. (That Larry Page bit about just automating away support.)
Re:Shopify to lay off 10% of workers in broad shake-up by 40acres on Tuesday July 26, 2022 @09:56AM (#32238319)
I agree with you for the most part. The number of people who could run Shopify at the current scale is definitely more than 80 people but probably less than 10000.

They took a big bet on global growth and this would've exploded HC related to sales, marketing, compliance, recruiting, etc. They pulled hiring forward on an expectation that the market advanced 5 years during COVID and now have to cull that new growth. Lots of late stage private companies and recent IPOs are going through the same thing.

Re:Shopify to lay off 10% of workers in broad shake-up by hef19898 on Tuesday July 26, 2022 @10:09AM (#32238478)
And in that light ten percent isn't even that much. It really sucks for those being laid off, Shopify seems to take their responsibility for their people serious so (if they follow through with what they promised). And yes, pulling your timetable and hiring forward was bet, but a less risky one than it turned out to be in hindsight.

More general, we had the 2008 financial crisis after one not too big investment bank went under. Covid did, despite all the real world damage from lost lives to the economy, not doing anything like it. I always wondered why, aparently more free money, on top of all the free money we had since 2008, helped. I just hope that the postponement of hitting a wall doesn't mean we ginna hit it much faster. We'll see...

Re:Shopify to lay off 10% of workers in broad shake-up by xwolfi on Tuesday July 26, 2022 @10:08AM (#32238461)
Not much. They're now starting to fire a bunch to make the rest quit.

I'm in a well run, immensely profitable, 100+ yo company, so big a country needs us to run, literaly, and a few others would be destroyed if we suddenly closed shop and ran with the money.

We are 60 000 and feel too fat. I cannot understand what frigging shopify is doing with 10k people. We make 5bn a year profit these days, is shopify close to 1?

Edit: waoh, they indeed were at 3 bn last year, 300mil the year before, -100 mil before, without reading the details. It's not regular profit but if it was, it'd just confirm we're too fat, not them :p

Re:Shopify to lay off 10% of workers in broad shake-up by echelon on Tuesday July 26, 2022 @11:41AM (#32239826)
> What are they doing?

Microservices.

I haven't worked there, but I've worked at incredibly similar companies with the same scale.

Every single business function split into fine grained services, with teams of size 5-20 (larger teams have split services, oncall, etc.)

Auth, API gateway. Layers of devops teams. Traffic, monitoring, data store, many different analytics support teams. Kubernetes maintenance.

The payments platform is probably huge. The app teams too.

Infosec of various flavors. Vendor integrations...

Think about the sprawling functionality of a company at scale.

Anyone else is sales or support. And other business functions, like various legal departments.

10,000 is the right size for an org like Shopify.

Re:Shopify to lay off 10% of workers in broad shake-up by ryanbrunner on Tuesday July 26, 2022 @12:34PM (#32240515)
Needing to have a big team so that you can refactor into microservices, which you need because you have such a big team is a very "the bureaucracy is expanding to meet the needs of the bureaucracy" sort of thing.
Re:Shopify to lay off 10% of workers in broad shake-up by echelon on Tuesday July 26, 2022 @12:40PM (#32240611)
It's a business expanding to fill the market sort of thing.

The people on those teams undoubtedly have plenty to work on.

Re:Shopify to lay off 10% of workers in broad shake-up by ecshafer on Tuesday July 26, 2022 @01:03PM (#32240915)
FWIW we don't really use Microservices at Shopify.
Re:Shopify to lay off 10% of workers in broad shake-up by ibawt on Tuesday July 26, 2022 @04:47PM (#32243762)
cardserver, storefront renderer, remember-me... the list goes on.
Re:Shopify to lay off 10% of workers in broad shake-up by perlgeek on Tuesday July 26, 2022 @10:05AM (#32238425)
> How can they manage to productively use the time of 10000+ people?

Sales and support basically scale arbitrarily large, as long as the market is there.

Re:Shopify to lay off 10% of workers in broad shake-up by dehrmann on Tuesday July 26, 2022 @12:31PM (#32240459)
> What are they doing?

Scale is harder to manage as the userbase grows. A lot of popular services could run on one machine if you can live with 3 nines of reliability and never scale past 100,000 users.

When you reach that scale, you also start doing lots of product work that's either exploratory and unlikely to matter, or micro optimizations trying to move the needle 1%. If you hit a win on one of these, because of your scale, the win can be huge.

Re:Shopify to lay off 10% of workers in broad shake-up by jokethrowaway on Thursday July 28, 2022 @04:43AM (#32261812)
Wasting time.

I've seen large companies and it's astonishing how little work and how much process they have for minuscole changes.

The only productive way of managing this is do what Google or Amazon do: split in sections and teams and get them to do something independently, with the hope that someone will struck gold. It's a bit like a venture incubator and - sometimes good things come out of it, most of the time dead products come out.

The company where I'm currently in is ~200 engs and a few companies ago with 50 engs was easily 10x more productive and had a way bigger codebase.

Management is either too useless to care / too attached to saying they have N underlings and the business makes so much money they can afford to spend x% on tech workers, just to be safe.

Re:Shopify to lay off 10% of workers in broad shake-up by moneywoes on Tuesday July 26, 2022 @10:11AM (#32238501)
I feel it’s a matter a growing faster than your competitors
Re:Shopify to lay off 10% of workers in broad shake-up by randomUserHere on Tuesday July 26, 2022 @10:23AM (#32238669)
So many times I see this kind of comments were people totally ignore the complexities that comes with large scale projects, huge complex legacy codebases that need to be maintained. What they see is a simple website that can be build by 5 people startup team in a day.
Re:Shopify to lay off 10% of workers in broad shake-up by taf2 on Tuesday July 26, 2022 @09:53AM (#32238277)
Lots of teams - working on lots of different products that all connect together.
Re:Shopify to lay off 10% of workers in broad shake-up by ActionHank on Tuesday July 26, 2022 @10:05AM (#32238423)
Multiple products targeting multiple users, both internal and external.
Re:Shopify to lay off 10% of workers in broad shake-up by stephc_int13 on Tuesday July 26, 2022 @10:18AM (#32238590)
Alright.

Let's say that each product dev team is about 80 engineers. Let's double that number for support, HR, admin, etc. Let's add a team of 10 people for sales ( not needed for internal products but hey)

Now let's round it up to 200 heads.

That would make 50 large software products, is that right?

Re:Shopify to lay off 10% of workers in broad shake-up by ggreenbe on Tuesday July 26, 2022 @11:09AM (#32239421)
It's more than 50... I mean just look at the number of API endpoints across multiple APIs both internal and external. It's easily hundreds then add sys ops, dev ops, dev content writing (docs/guides), frontend, design, pm, support, etc.

I feel like most of you all are unaware of the complexity and scope of Shopify...

Re:Shopify to lay off 10% of workers in broad shake-up by jokethrowaway on Thursday July 28, 2022 @05:21AM (#32262114)
I've seen companies where a team was managing 10 services and companies where a team was managing one endpoint

The former had some stressed principal engineers, the latter was terribly inefficient and people had 1% productivity.

There is a happy medium, but nobody can manage a large headcounts without splitting in a lot of tiny products.

The core product of shopify is not something that needs 10k engineers for maintenance, even if they can afford that.

Re:Shopify to lay off 10% of workers in broad shake-up by woofcat on Tuesday July 26, 2022 @11:08AM (#32239391)
On a large chunk of Enterprise Software you'd have teams of people working on specific features / subsets of a larger product.

I.e. a team that is working on only Facebook Messenger back-end, a team working on Facebook Messenger Front-end, a team working on Friend Recommendation, etc.

Each of these teams is let's say 10 Dev, 2-4 QA, 1 Product Documentation, then all the normal overhead of HR, etc.

Re:Shopify to lay off 10% of workers in broad shake-up by foobiekr on Tuesday July 26, 2022 @02:06PM (#32241685)
See the previous discussion of manager incentives to hire here:

https://news.ycombinator.com/item?id=32149937#32171559

Re:Shopify to lay off 10% of workers in broad shake-up by cosmiccatnap on Tuesday July 26, 2022 @10:08AM (#32238473)
Marketing...