News for hackers. Stuff that matters.
Re:Porting Firefox to Apple Silicon by loufe on Wednesday January 20, 2021 @12:20PM (#25849041)
We talk often on this forum of how innaccessible giants like Google and Amazon are for the little guy. I thus found this point particularly interesting:

>"Attempts to contact the vendor through regular support channels were unsuccessful so we ended up searching LinkedIn and managed to find an engineer working on the core antivirus detection. They immediately understood the seriousness of the problem and took prompt action to get a fix shipped, thus preventing quite the disaster for the users of this product. It’s notable that without this last-ditch effort we would have been effectively blocked from releasing a native Apple Silicon version for an indefinite period."

Re:Porting Firefox to Apple Silicon by duxup on Wednesday January 20, 2021 @12:37PM (#25849287)
I was a back channel route to engineering at a company I worked for a while. A few of the engineers trusted me and a couple select customers / sales guys knew they could come to me to run a bug by me 'in theory' and if we had enough information I could unofficially run it by engineering without going through the song and dance of the typical support channels.

I could get a quick ya or nay form them on some things and it was so much faster for everyone involved.

If it was a ya, I knew we had something, still more work to do but the case would skyrocket through the usual channels and engineering was engaged and ready.

If it was a nay, the usual channels it went and everyone was ok with that.

The engineers would give me a few minutes knowing I wasn't going to bring them poorly thought out garbage, I would limit the rate of these special situations, and special customers / sales guys could get the job done way faster.

It was a well known process by those who knew about it... but not everyone knew.

Re:Porting Firefox to Apple Silicon by zerkten on Wednesday January 20, 2021 @01:45PM (#25850234)
This exists in almost every company by design. Engineering teams wouldn't make any progress towards their mission if they are constantly dealing with outside interruptions, but at the same time there are things which should be qualified.

Customer support is a cost center and the focus is on mitigating the cost of providing that support. If you fail to do this you can burn through a lot of cash quickly. What management needs to realize is that this is also an important interface point which requires attention. This doesn't happen at all, or is inconsistent.

It's important for at least the following to happen:

1. Bad issues that engineering will fix don't get stuck in support.

2. Product management review and respond to feature requests, or enable support to respond to customers.

3. Support have a reasonable level of technical and communication skill, and are empowered to answer for the company.

4. The organization works through rather than around support.

What I've always found interesting, is that all of these are often failing in some way at the same time in an organization of any size.

Your role as the back channel is helping to provide some coherence here. However, things can go bad if you left. Inevitably, this is the fault of the company, but when I've found myself in this position I've tried to "promote" people in support to take the lead on this role. Further, formalizing the special request process to be minimally tracked helps visibility with my manager and others. Eventually managers ask why you have become a gopher.

Improving the workflow often involves helping support build relationships with engineering. Management can buy in if support attrition is high (it often is if there is a limited career ladder for support) and it can also improve their perception when people are focused on trimming support cost.

Re:Porting Firefox to Apple Silicon by jkaptur on Wednesday January 20, 2021 @02:08PM (#25850526)
The binary distinction between "cost centers" and "profit centers" has always seemed arbitrary to me (especially since, as an engineer, I've been in both without my work being substantially different).

To be frank, it seems like an organizational way to say "we don't find this work to be valuable or interesting, and we'd like to do the bare minimum of it - in fact, we'd like to unleash smart people to explore new frontiers of just how minimal the bare minimum could possibly be."

It seems like this leads to incredibly predictable problems: brain drain, demoralized workers, the bare minimum being aimed for and not actually being achieved, etc.

I feel like a better organization has no "cost centers" - every single role at the company contributes to the mission and to the bottom line. If they didn't, that position wouldn't exist.

What am I missing?

Re:Porting Firefox to Apple Silicon by kortilla on Wednesday January 20, 2021 @09:21PM (#25854557)
There are parts of the business that cannot boost revenue. “Investing” in them doesn’t really make sense beyond nominal amounts because the max return they can provide is eliminating themselves.

Unless you’re into fraud, “accounting” and “accounts payable” are examples of cost centers. You don’t hire a bunch of innovative people to boost it because it’s not going to ever increase your revenue.

The distinction is made from a strategic perspective because scaling up “cost centers” should be avoided at all costs and scaling up “profit centers” is something you want to do as much as you can.

It has no overlap with “interesting work”. Very often the boring parts of an industry are the profit centers (e.g. in academia the profit comes from packing students into classrooms, not research).

Re:Porting Firefox to Apple Silicon by Closi on Thursday January 21, 2021 @02:40AM (#25856495)
I also disagree with the “cost centre” view, I think it’s often used too simplistically and doesn’t account for the fact that all areas should add value (otherwise you wouldn’t have them by design).

Some examples (that I have seen in reality):

Finance departments are cost centres, until you give them enough resource and they find you a more efficient tax structure. Cut finance departments start to struggle with things like credit control which affects your revenue.

Distribution Centres are usually seen as a cost centre, until you drop spend and it impacts COGS or customer lead times, or inventory in shops raises because of less frequent deliveries and you get out of stocks.

IT is a cost centre, but when funding is reduced change across the whole business slows and other areas are impacted (eg the customer web experience).

In reality the distinction of “some areas generate profit” and “some areas just cost” isn’t true in the end. All areas contribute to profit - some just do so indirectly.

I think the idea of Michael Porters “value chain” is better, where everything contributes to customer value (including indirect functions). The argument this makes is if you see some areas as just cost centres (e.g. fulfilment centres) then you can miss your ability to maximise customer value (e.g. offering faster delivery options).

Even sales people don’t usually generate profit on their own because without the other business areas they would be selling hot air.

Re:Porting Firefox to Apple Silicon by snoshy on Thursday January 21, 2021 @03:01AM (#25856586)
I think this is exactly right. However, the question still stands: how does a large tech-focused corporation encourage engineers to pay attention to such requests?

Expecting enough of them to just volunteer their time doesn't appear to be a sustainable answer.

Re:Porting Firefox to Apple Silicon by Closi on Thursday January 21, 2021 @03:10AM (#25856632)
Oh, I agree with that - I just disagree with the cost centre view (most of my career has been in logistics which is traditionally a 'cost centre' while at the same time people complain about the service they get! In logistics there is a relationship between service and cost).

I spent a few years in IT as a Product Manager (or a similar role), and I viewed my primary role as protecting my team from the barrage of shit that I got, so that they could focus. This involved making sure I was politically the first point of contact and reducing back-channels (some are fine, but not ones that change functionality, involve significant work or are too distracting), placating the people requesting functionality or fixes by understanding how serious the dependency/issue was, triaging it and either placing it on the roadmap or saying no. We also had an engineering manager that could be the contact for specific bugs who could then triage and pass it on.

Re:Porting Firefox to Apple Silicon by kortilla on Thursday January 21, 2021 @05:21PM (#25865275)
They are still cost centers. Being a “cost center” does not mean money can be cut from the department and the business won’t suffer. It means that pouring extra money and scaling up the department does not generate more revenue.
Re:Porting Firefox to Apple Silicon by toyg on Thursday January 21, 2021 @09:03AM (#25858864)
> Unless you’re into fraud, “accounting” and “accounts payable” are examples of cost centers. You don’t hire a bunch of innovative people to boost it because it’s not going to ever increase your revenue.

You're so, so spectacularly wrong on this, I am honestly gasping for air.

Accountants are the only people who know if your company is alive or a walking dead. How do you expect to run a company if you don't know reliably and with precision how much money it actually has/makes/spends? Money is the lifeblood of a company! Don't you want to be constantly improving the way you make, spend, and report it to investors and the public?

The biggest companies in the world typically end up with CEOs that come either from sales or from accounting. That is not an accident. Business is about money, and you want smart and innovative people to look after it. Conservative CFOs can be the death knell of a company, among other things.

Re:Porting Firefox to Apple Silicon by kortilla on Thursday January 21, 2021 @05:19PM (#25865231)
> You're so, so spectacularly wrong on this, I am honestly gasping for air.

Calm the fuck down. It’s a conversation.

> Accountants are the only people who know if your company is alive or a walking dead. How do you expect to run a company if you don't know reliably and with precision how much money it actually has/makes/spends? Money is the lifeblood of a company! Don't you want to be constantly improving the way you make, spend, and report it to investors and the public?

You entirely missed the point. At no point did I say accounting was not important. I pointed out though that investing more and more into accounting does not boost returns. If that were true, every company could just hire thousands of accountants to boost their profits. This is what separates a cost center from a profit center. Your department provides value in the same way that running water does. It’s critical and you don’t want to skimp on it, but it’s just a part of the business that isn’t helping grow the total market capture.

> The biggest companies in the world typically end up with CEOs that come either from sales or from accounting.

Why would you include sales together with accounting? Sales is precisely the opposite of accounting in this regard because it’s very easy to tie sales directly to revenue. So easy their compensation is literally based on it.

Not so hot take: CEOs that come from accounting and not a customer-oriented profit center are the worst. They know what the numbers look like but are fundamentally disconnected from why customers give money to the business. Seeing the minutiae of the ins and outs of money gives a super false sense of understanding the business. Accounting CEOs are terrible in any industry that requires innovation or getting ahead of trends.

Re:Porting Firefox to Apple Silicon by zepto on Thursday January 21, 2021 @10:56AM (#25860187)
I generally agree with you about not dismissing the value of accounting.

However this:

“Accountants are the only people who know if your company is alive or a walking dead.“

I disagree with.

A walking dead company is only walking dead until one of its initiatives pays off.

The accountants will only know this after the fact, whereas numerous other functions may know it to varying degrees of confidence before the fact.

Re:Porting Firefox to Apple Silicon by toyg on Thursday January 21, 2021 @12:06PM (#25861202)
> The accountants will only know this after the fact

The accountants know when your debits have to be repaid, how likely they are to be repaid or refinanced by then, and what the penalty for not doing so will be. I'd argue that, in most cases, nobody employed in the development/production chain will have that information, possibly not even the CEO.

Re:Porting Firefox to Apple Silicon by zepto on Thursday January 21, 2021 @01:00PM (#25861867)
Sure, but accountants will not know the likelyhood of success of a new product or service, how the features will affect sales, the likelyhood of making large sales, the state of strategic relationships, etc. etc.

Yes, they may have models and estimates, but the real information will be in the hands of those involved directly.

Even things like refinancing and the options for doing so can be affected by things like letters of intent from potential large clients, industry validation, etc.

Just knowing numbers and dates isn’t enough.

I’m not saying accounting isn’t important, but it just isn’t the only source of truth.

Re:Porting Firefox to Apple Silicon by toyg on Wednesday January 20, 2021 @06:24PM (#25853250)
You're missing the lack of creativity and courage in the managerial class.

Support can very much be a profit center. Support personnel is relatively cheap; if their services are priced correctly, they can easily become a stream of recurring income - and everybody knows that "recurring income is best income".

However, this requires efficiency and creativity at the managerial level. It's easier to see support as a burden and just work on shrinking its costs, instead of maximizing its revenues by formulating good price plans. The former is an internal effort that is fairly easy to implement in short timeframes and will easily win brownie points with direct superiors (who doesn't like to cut costs?); the latter requires actual pricing skills and market knowledge, and might take a while to get results. The mediocre manager will always prefer the former.

Re:Porting Firefox to Apple Silicon by vegetablepotpie on Thursday January 21, 2021 @10:07AM (#25859545)
I feel like this is a double edged sword. Corporations have already embraced this idea of converting cost centers into profit centers and it makes life more difficult for consumers. Every time I call support at my ISP, because my internet is down, they work on my problem and tell me that my plan is slow and that I can upgrade my package for an increase in my bill.

This sets up perverse incentives, that as far as I can tell, are theoretical, but have potential to become more prevalent. Because of the cost center as a profit center idea, my ISP can generate more revenue by providing less value to me. If failing infrastructure causes me to call support more often, and more support calls increase the likelihood of more revenue, why should the ISP invest in better infrastructure?

The key to having a successful business is to carefully align the incentives of specialities in an organization to make the most competitive offerings to the market. If there are competitors, and customers can switch to them, and the competition is more compelling, then I would go to other ISPs.

Re:Porting Firefox to Apple Silicon by toyg on Thursday January 21, 2021 @12:01PM (#25861124)
I agree that what is good for an individual company is not necessarily good for consumers or the market as a whole, and that there is always a balance to find, but that's another issue. After all, the ideal scenario for a company is to have customers pay for support without actually using it most of the time.
Re:Porting Firefox to Apple Silicon by ddingus on Wednesday January 20, 2021 @02:35PM (#25850896)
Not much.

"Cost center" can be transformed into something else given both an understanding that support can and should contribute to future sales, and an organization capable of putting that understanding to work.

I have seen a similar scenario in manufacturing where various setup, prep, quality tasks are seen as cost centers and minimized.

Doing this kind of thing has ripple costs. Always.

In a perfect world, we make software, or hardware, and it just works and people grok it.

In the one we live in, these are fantasies and we can choose to understand, recognize the value, or not and get the benefits or not.

The users, customers, move from role to role, and support often determines their willingness to use the product again. That is straight up powerful marketing by referral.

Support often is the first to understand a user, customer needs an option too, or add on, replacement, preventative maintenance. Done right, these leads into lean, consistent sales.

"Cost center" to me has always been a bit silly in this way. There is opportunity to add value throughout the chain of people, process, machines, systems that are all necessary to properly conceive, realize and deliver something to others.

One thing often missed along with failing to understand value is failing to ask to be compensated for it.

Doing things in a robust, high value for the dollar way is not the cheapest way, in terms of raw product price, and depending, size of margin.

But, we do get what we pay for too, and the lowest price often comes with externalities paid by both the enterprise and its customers too.

Sometimes I see this all framed as a luxury. That is just as much of an error, and does come with unnecessary costs and or poor alignment with actual value.

Re:Porting Firefox to Apple Silicon by Gene_Parmesan on Wednesday January 20, 2021 @06:56PM (#25853520)
Our support team is not technical in any way (they are support for our entire moderately sized membership-based nonprofit), but they are consistently a source of extremely high quality feedback, for the reasons you mentioned.

If there are serious UX issues, your designer might not uncover them, but support will hear about it. If there are edge case performance issues, your dev team might not uncover them but support will hear about it. Very few people know more about how real users interact with your products than support.

Re:Porting Firefox to Apple Silicon by ddingus on Wednesday January 20, 2021 @09:32PM (#25854620)
Word

I should have included that. Glad you did.

Re:Porting Firefox to Apple Silicon by cat199 on Wednesday January 20, 2021 @03:02PM (#25851213)
> What am I missing?

not much, or everything -

it's basically an accounting term on how you are tracking an expense and so it is very insightful as to how the effort of your project,group,department etc. is perceived by upper mgmt

so "we don't find this work to be valuable or interesting, and we'd like to do the bare minimum of it - in fact, we'd like to unleash smart people to explore new frontiers of just how minimal the bare minimum could possibly be."

is pretty spot on, if the effort has been (mostly arbitrarily) categorized as such..

when i learned the accounting theory behind it, it suddenly illuminated managment attitudes in current/previous jobs - literally in some orgs overly reliant on this perspective there is literally nothing certain efforts can do through official channels to be viewed as 'valuable' ..

Re:Porting Firefox to Apple Silicon by mr_toad on Wednesday January 20, 2021 @08:05PM (#25854075)
Originally everything was a cost centre, but some rebranded as profit centres, and the PHBs swallowed it.
Re:Porting Firefox to Apple Silicon by julienfr112 on Wednesday January 20, 2021 @05:59PM (#25853077)
You should have shared you wisdom with Boeing in 2010 ...
Re:Porting Firefox to Apple Silicon by noizejoy on Wednesday January 20, 2021 @02:33PM (#25850864)
x% of support requests are of questionable nature - to mention just a few categories:

* people expecting to use a sophisticated tool (for doing complex business processes requiring special know-how) without paying for and spending time on adequate training)

* people unwilling to RTFM, google, youtube, etc.

* people whining when a general purpose tool doesn’t fit their exact workflow to a tee

Re:Porting Firefox to Apple Silicon by ddingus on Wednesday January 20, 2021 @02:36PM (#25850912)
Those are all sales and service opportunities, BTW.

Back in my support role for higher end software, I flat out hit numbers comparable to sales and generated a ton of great leads.

Fact is, people do what they do and they have their reasons.

Judging them and acting on that judgement by marginalizing an important and necessary part of the process has a higher net cost to the world, and often the enterprise, than just doing those things reasonably does.

Net happiness goes up too. True for the enterprise and users, people at large.

Re:Porting Firefox to Apple Silicon by noizejoy on Wednesday January 20, 2021 @02:38PM (#25850933)
Not all sales and service opportunities have positive ROI
Re:Porting Firefox to Apple Silicon by ddingus on Wednesday January 20, 2021 @02:41PM (#25850982)
There are no free lunches. Expecting otherwise is a very good sign the enterprise is penny wise, pound foolish.
Re:Porting Firefox to Apple Silicon by noizejoy on Wednesday January 20, 2021 @07:20PM (#25853716)
Not every business environment is captured by grand sweeping general statements. Reality is a lot more nuanced than that.
Re:Porting Firefox to Apple Silicon by ddingus on Wednesday January 20, 2021 @09:31PM (#25854612)
Precisely.

And the fact is, enterprises seeing to make every support transaction a positive ROI are, in fact and in deed, penny wise and pound foolish.

They will see an opportunity cost due to missed sales opportunity.

They will see greater load due to people using an inferior process and poorly empowered people, repeatedly.

They will see a diminished overall market perception.

Their products will provide less value due to a greater misalignment with both exiating and potential users needs, which drive perception of value, which drives more dollars.

Personally, having been on all sides of this matter, I rank what we are discussing at the very top when considering who I will buy from and or work with.

Flat out, when I see enterprises putting seriously crazy amounts of money in the bank, I accept zero excuses in this regard.

It is not necessary. Lives are short, money hard to come by. Best get solid value for the dollar.

Re:Porting Firefox to Apple Silicon by noizejoy on Wednesday January 20, 2021 @09:34PM (#25854635)
I generally agree with your arguments when the customers are medium to large businesses - and I assume that’s where your experience is. In the consumer and small business space the dynamics are very different.
Re:Porting Firefox to Apple Silicon by ddingus on Wednesday January 20, 2021 @09:38PM (#25854664)
Been there. They aren't. Some of the mechanics vary, but good service does not.

Right now, I am in the small business space and rock solid support is how we are killing it.

Been there, very large, small, medium, consumer, b2b...

Don't tell me it can't or should not be done when billions land in accounts free and clear.

It can. Should.

I spend with those who get that first and foremost.

Re:Porting Firefox to Apple Silicon by noizejoy on Wednesday January 20, 2021 @09:47PM (#25854713)
Maybe you should quit your current career arc and help companies change their attitude and thus unearth extra billions in profits?

And I don’t mean that cynically - since if it’s as easy and guaranteed profitable as you say, why wouldn’t you be able to convince numerous CEOs to unearth all of those extra billions?

Re:Porting Firefox to Apple Silicon by ddingus on Thursday January 21, 2021 @12:30AM (#25855791)
It is all value judgements.

Again, if the priority is to always have a positive ROI, and that metric is computed every quarter, without due and inclusive consideration for externalities?

All the things I discussed here are going to get watered down. And it is always the same priority on max dollars now, max recurring dollars now, and WGIF about the future, others.

Where that happens, so do the things I just said. Not my mess to clean up.

Some enterprises get it. They get my time, attention, dollars and referrals first.

Beyond that? Got better things to do.

Clearly you value things differently. That does not make anything I said wrong.

Take care. You get last word on this.

Re:Porting Firefox to Apple Silicon by magicalhippo on Wednesday January 20, 2021 @04:10PM (#25852036)
> Customer support is a cost center

For us, great customer support is one of our stronger sales arguments. In fact we've not had to push hard on sales due to our customers calling former colleagues who moved to a competitor to tell them "you have got to get this software". Having great support has been key to this experience.

Most of our support people have been recruited from our customers, so they know not just our software well but the processes and regulations around it, allowing them to quickly understand the issue at hand and offer relevant help.

So while it might look like a cost center on paper, I'm quite certain it's a massive net gain overall.

Of course as you say, we work hard to mitigate the cost of providing that support, like routinely looking at implementing changes that'll reduce repeat support issues. Maybe as simple as reworking a dialog text, to adding more automation.

Re:Porting Firefox to Apple Silicon by duxup on Wednesday January 20, 2021 @02:48PM (#25851058)
Yup that's accurate. I was a regular old support drone who had some connections that allowed for some special paths to engineering.

We had 'official' faster escalation paths but those inevitably are determined by $$$ and there's always more ways to measure 'important customer' than can be defined / shown in $$$.

Management was totally aware of it all and supportive.

But eventually I got tired of the land of 'support' and moved on for a variety of reasons, mostly because time and again I saw support treated like the usual 'cost center' and I didn't want to be a part of that.

Re:Porting Firefox to Apple Silicon by cosmie on Wednesday January 20, 2021 @04:34PM (#25852296)
> What management needs to realize is that this is also an important interface point which requires attention. This doesn't happen at all, or is inconsistent.

> What I've always found interesting, is that all of these are often failing in some way at the same time in an organization of any size.

The formalization of it is frequently the cause of it failing or being inconsistent. Once it's a workflow that's explicitly acknowledged and condoned by management, it will start to lose its effectiveness. As an official express lane between customers and engineering, every account/sales person will become aware of it and overload it, either in the general course of supporting their client portfolio as much as possible, or even worse, by making this internal express route known to clients, as they can get incremental revenue by branding it as a "VIP Support" service or to make at-risk clients feel special. Which will eventually end up in actual client contracts in some form or another, opening the door to client abuse (or misuse) as well as causing legit cases that would have gone through this implicit channel to get routed to and trapped in normal support because the client at hand didn't pay up for the express lane.

You've also replaced a channel built off of relationships and mutual trust/respect into one based on official responsibilities and inertia, and all the hazards that entails. Such as political/managerial turf wars that add friction to the process, as well as cost minimization efforts that deskill the role over time and profit maximization efforts that overwhelm the capacity of the role, alienating the engineering team and undermining the entire intent.

... not to say it's impossible. But that's generally why you'll see it failing in some capacity any time you witness it, because it's almost impossible to maintain equilibrium the moment you officiate it.

An alternative that tends to be more lasting is for management to _actively facilitate organic growth_ of these sorts of things. Enable and encourage and provide opportunities for inter-departmental relationships and lines of communications to form. That way there is no single "back channel", and organic lines of communication between different parts of the org are robust against the loss of a single node.

Re:Porting Firefox to Apple Silicon by coliveira on Wednesday January 20, 2021 @02:07PM (#25850515)
The biggest problem we have in software engineering is the lack of support staff. You don't think a civil engineer has to deal with minutiae of paperwork, but software engineers for some strange reason think it is ok to be inundated by clerical work all the time. The industry eventually must evolve to create software assistants capable of running code, triaging bugs, etc.
Re:Porting Firefox to Apple Silicon by techsupporter on Wednesday January 20, 2021 @02:14PM (#25850600)
> The industry eventually must evolve to create software assistants capable of running code, triaging bugs, etc.

In my opinion, we had this. They were your senior support staff or were operations; some companies combined this into a formal role called "Support" or "Service" "Operations."

But then we as an industry decided that operations is bad[0] and if you write the code then you can obviously test the code, deploy the code, maintain the code, and support the code. Then every Hip And Cool Start-Up adopted the model of "sysadmins and support staff are bad because we've had bad experiences in the past so we will also have our devs talk directly to customers until they get tired of doing that and we just replace it with a contact form encumbered by CAPTCHA and a no-reply e-mail address."

As someone who has greatly enjoyed, been very good, and very well paid (so my employers agreed that I was good at it), at support and operations roles only to see them disappear into the inky void of Everyone Codes All Of The Time, I am both biased and frustrated.

0 - Because money, I suspect.

Re:Porting Firefox to Apple Silicon by tylerfontaine on Wednesday January 20, 2021 @09:00PM (#25854425)
There are still lots of good support organizations out there, and not every place looks at support as a burden. It's a pretty critical piece in the "new" 'as a service' world, helping folks use complex systems that they don't control, etc.

I've been heavy in the data space, and get to do some fascinating work with folks, helping design data models, implement analysis, and other things in a wide variety of verticals. It's support, so sometimes there's some more tedious things too - there's no avoiding that. :)

But, and perhaps I'm biased, it's still a great career path, even if it's not as flash as "code all the time" work.

Re:Porting Firefox to Apple Silicon by coliveira on Wednesday January 20, 2021 @08:23PM (#25854198)
I think this is an indictment of the lack of organizational skills in the area of software engineering. It is still an industry run by the sit of their pants. There is no clear separation of responsibilities and everyone wants to do everything (and usually badly).
Re:Porting Firefox to Apple Silicon by duxup on Wednesday January 20, 2021 @02:43PM (#25851001)
It's funny you mention that. I was a support drone when I was in the situation I described above.

But support being support ... it is eventually devalued and I chose to learn to code to move out of those types of roles.

When I moved on (through a somewhat handy acquisition and layoff and etc) some engineers reached out to me to join the support team there.... but I was done with support (and other factors).

Re:Porting Firefox to Apple Silicon by InfiniteRand on Wednesday January 20, 2021 @11:32PM (#25855420)
In my company, there is a blurriness between support and technical sales, and while that can be a little chaotic, one benefit of that approach is support is looked at as a profit center to a degree. This is because the support people keep an eye out on upselling, maintaining subscriptions, and promoting consulting work. We're not obnoxious about it, but there's some awareness that part of the role of support is to promote the long-term growth of the customer relationship and sales.

I think if there was a stricter division between support and technical sales, there would be more of a temptation to focus on burning through support requests as quickly as possible. The flip side of this is that it is easy for us to get bogged down in a complex half-support half-sales opportunity situation and that can sometimes cause other support requests to fall through the cracks.

Re:Porting Firefox to Apple Silicon by indymike on Wednesday January 20, 2021 @02:48PM (#25851057)
There is a fine line between running interference and sales prevention.
Re:Porting Firefox to Apple Silicon by tines on Wednesday January 20, 2021 @05:36PM (#25852893)
Just for reference, the pair is spelled "yea or nay" rather than "ya or nay".
Re:Porting Firefox to Apple Silicon by anonymouse008 on Wednesday January 20, 2021 @01:34PM (#25850094)
That sounds like a dream career -- any advice getting there when you've hacked on graveyard startups most your life?
Re:Porting Firefox to Apple Silicon by cosmie on Wednesday January 20, 2021 @07:20PM (#25853724)
Stuff like what the parent described tend to be "on top" of your day job, rather than your actual day job.

Parent likely established a working relationship with individual(s) in engineering organically over time, and at some point leveraged that relationship to ping them about a customer issue that crossed their path and seemed to be an engineering concern vs user-error. That didn't cause waves and went well, and got repeated enough to become an established but unofficial "thing" parent was capable of, and they became known by a few sales/account folks as the go-to person when they felt a situation may warrant that unofficial route.

You can make a career off of doing this sort of thing, but I'd caution against it. If a company is hiring for specific scenario of "fast lane between client and engineering", the actual job is "support with special escalation privileges, servicing clients splurged for the premium package". You get all the soul-crushing hell of working a normal support role, but with the added benefit of solely servicing clients that expect you to hand them the world because they paid extra for it. Which is far closer to a nightmare than a dream; particularly considering someone working one of these official roles likely has the skills to pivot out of the support org and into the engineering org in some capacity, and drastically increase their salary potential while simultaneously improving their quality of (work) life.

In a more general sense though, pretty much any career benefits from doing what the parent described. It's effectively just flexing your soft skills and establishing relationships with people outside of your immediate sphere/department. Which has a tendency to make it easier for you to get things done, and garnering a reputation to that effect.

Re:Porting Firefox to Apple Silicon by duxup on Wednesday January 20, 2021 @02:41PM (#25850981)
It wasn't an official job.

I was a regular support drone as far as anyone knew.

I just had some connections that came about because I could be discrete and the engineers understood that I didn't bring them garbage too early (without enough information) or without good reason.

Re:Porting Firefox to Apple Silicon by macksd on Wednesday January 20, 2021 @06:40PM (#25853389)
And this is why it's important to recognize and retain people who get the right things done. One does not simply backfill a support position and replace someone who has built up the internal AND external reputation and relationships required for issues like this to get fast-tracked and fixed in a way that makes everyone happy.
Re:Porting Firefox to Apple Silicon by dingaling on Wednesday January 20, 2021 @03:48PM (#25851790)
Soft skills are key; getting to know key people, and their responsibilities and capabilities and personalities.

But above all: be a good listener. Listen to what they say, think about it and build it on next time you talk with them. If they see you showing interest and learning about their domain, you'll get a direct line to them. And don't always bring them problems, be sure to stroke their ego too by asking what they're working on.

It doesn't happen overnight, it requires perserverence and a dollop of luck. You won't walk into a job like that, it takes years of building a reputation for yourself.

Re:Porting Firefox to Apple Silicon by meibo on Wednesday January 20, 2021 @12:44PM (#25849393)
The antivirus industry is the biggest player of the modern adware/malware crisis.

The dark patterns used in software like AVG and avast, both making every system I see them on so slow that they might as well be unusable, are all focused on getting more installs, be it to force people into getting whatever "premium" subscription or harvesting data(e.g. attaching themselves to every sent email like a virus).

There are very few that I could actually recommend, like Malwarebytes - for most users, Windows Defender will be more than enough nowadays. I haven't used a mac in a while, do you actually need AV on them today?

Re:Porting Firefox to Apple Silicon by sjg007 on Wednesday January 20, 2021 @01:01PM (#25849618)
I mean there is mac malware and some of it is quite sophisticated. The compiled apple script bitcoin miner being one of the more ingenious ones. But those were distributed through pirated applications. There are probably a few zero days as well that maybe an AVE package could help stop if a signature is rapidly distributed. Ignoring the fact that the AV engine itself is a target too.

Most people using only the app store helps cut that down.

Re:Porting Firefox to Apple Silicon by wlesieutre on Wednesday January 20, 2021 @01:13PM (#25849792)
The one in recent memory for me is the KeRanger ransomware that was distributed in the official Transmission installer.

https://unit42.paloaltonetworks.com/new-os-x-ransomware-kera...

>Transmission representative John Clay told Reuters via email that the ransomware was added to disk-image of its software after the project's server was compromised in a cyber attack.

>"We're not commenting on the avenue of attack, other than to say that it was our main server that was compromised," he said. "The normal disk image (was) replaced by the compromised one."

Re:Porting Firefox to Apple Silicon by setpatchaddress on Wednesday January 20, 2021 @02:11PM (#25850574)
Just to answer the original question: that's an example of something Apple handled -- no external AV required.
Re:Porting Firefox to Apple Silicon by wlesieutre on Wednesday January 20, 2021 @02:23PM (#25850728)
Yes, although it snuck by Gatekeeper to begin with by being signed by another developer account.

Would make it past Apple's new notarization scheme these days?

Re:Porting Firefox to Apple Silicon by lizknope on Wednesday January 20, 2021 @03:05PM (#25851253)
My work laptop runs Windows. It runs Outlook, Chrome, Slack, and Exceed to connect to a remote Linux server where I do all of my work. At random times throughout the day the fan will get really loud. When I run the process viewer tool during this time I see things using 100% CPU. It's a Core i5-8350U with 8GB RAM and during these times it gets almost unusable. I've googled a few of them and they always seem to be antivirus things.

At home I have over 8 Linux machines and the only times their fans get louder are when I am actually running a video encoding program or something CPU intensive like that. Some of them are slower with only 4GB RAM and they are always responsive.

Re:Porting Firefox to Apple Silicon by floatboth on Wednesday January 20, 2021 @07:59PM (#25854025)
The built-in Windows Defender is absolutely painful on low end machines. Yours should be absolutely fine, but on a 2010 macbook with an ancient core2duo and 2gb ram, it's very noticeable.
Re:Porting Firefox to Apple Silicon by zaphirplane on Thursday January 21, 2021 @06:25AM (#25857784)
Out of interest why is the performance of a basic a/v be affected by “older” cpu.

Isn’t it basically finger printing files and intercepting IO and so the resources it uses just depends on the activity of the device not the age of the CPU

Re:Porting Firefox to Apple Silicon by mr_toad on Wednesday January 20, 2021 @08:53PM (#25854382)
I have an i7 with 16GB of RAM and when Defender and update both decide to run at the same time it cripples the machine for half an hour.
Re:Porting Firefox to Apple Silicon by fomine3 on Wednesday January 20, 2021 @11:50PM (#25855545)
"i7 CPU" is totally zero informative for now. It's vary from 2core 1GHz (ancient i7 620UM) to 18 core.
Re:Porting Firefox to Apple Silicon by kiwijamo on Wednesday January 20, 2021 @09:37PM (#25854657)
That is strange. Until fairly recently my daily driver was a low end ex-lease HP machine from 2013 or so with an i3 with 4gigs of RAM. Never saw much issue with Defender or the updates. Not unusual for updates of any kind to be all over and done with in <5mins from clicking update to a fully usable desktop. Win10 has improved a heck of a lot over previous versions even on old hardware. My work-supplied Mac on the other hand, even with much better specs than my lowly home machine, usually takes at least 30mins to do updates (with most of the time spent during the reboot which means I can't use it at all). One of many reasons why I moved away from Apple for my own hardware.
Re:Porting Firefox to Apple Silicon by sam_goody on Thursday January 21, 2021 @05:17AM (#25857383)
> There are very few that I could actually recommend, like Malwarebytes

Malwarebytes installs a program with elevated privileges that starts on boot and always runs in the background, and regularly sends data home - despite that it is an ON DEMAND scanner.

I have written to the company to understand this virus-like behavior, and have gotten no response.

Do you have a reason to trust them?

Re:Porting Firefox to Apple Silicon by meibo on Thursday January 21, 2021 @07:30AM (#25858182)
Commercial Malwarebytes isn't on-demand, it's an actual anti-exploit/rootkit solution. I assume the free version, which only has the on-demand features, comes with these components anyways? Might explain that behavior.

My reason to trust them is that they seem to be generally respected still, I've been using them for a long time and they've yet to start annoying me with dark patterns and upsells - of course that's not a super great indicator.

Re:Porting Firefox to Apple Silicon by 3395810 on Thursday January 21, 2021 @09:44AM (#25859304)
The free version of MBAM is an old-demand scanner but the paid version has "active" protections.
Re:Porting Firefox to Apple Silicon by bostik on Thursday January 21, 2021 @05:03AM (#25857274)
> * The antivirus industry is the biggest player of the modern adware/malware crisis.*

This is so true it hurts. Veracode releases an annual report ("State of Software Security"), part as marketing material, part as an industry insight leaflet. The worst offenders for software security and defect rate are, year after year, security products.

As an infosec veteran, it's obvious to me that the "industry" at large is not obeying the rules they set for others. The shoemaker's children have no feet.

Re:Porting Firefox to Apple Silicon by Merad on Thursday January 21, 2021 @09:17AM (#25859009)
> There are very few that I could actually recommend, like Malwarebytes

This used to be the case, but the commercial/enterprise cloud version of MBAM (required by my company) is godawful. It seems to call out to its cloud back end every time an executable launches, and it murders performance. It's most obvious in terminals when it causes a simple command that should run in < 1 second to take 4-5 seconds.

Re:Porting Firefox to Apple Silicon by climb_stealth on Thursday January 21, 2021 @06:30AM (#25857821)
If anyone ever tries to install Sophos on your Mac, do your best to avoid it. Corporate IT seems to like it. It slows everything down. It's a nightmare to get rid off again.

The problem is for a lot of jobs you don't get a choice. The employer enforces it, no dark patterns necessary. And then you end up with a computer that is 70% busy doing AV-stuff and leaving 30% for actual work.

Re:Porting Firefox to Apple Silicon by e40 on Wednesday January 20, 2021 @03:56PM (#25851872)
This really bugs me about Apple. We had the DTK (pre-release hardware for the M1). We got into trouble with an upgrade from beta2 to beta10 and the machine was bricked. Everywhere we turned, we were told to use the developer forums. We did. There were no responses there and I didn't see anyone but customer helping customers.

So, since we're an Apple developer, we decided we would use one of our DTS (developer technical support) tickets. Nope. Pre-release anything is not supported.

So, we ended up waiting for release, bought a new M1 mini and then started our porting effort. Then, we ran into problems and used one of our DTS incidents and we got some help. However, we lost months.

Re:Porting Firefox to Apple Silicon by saagarjha on Wednesday January 20, 2021 @06:10PM (#25853164)
I don't have a bricked DTK, but the last seven builds or so kernel panic every few hours of use. It's kind of sad to see where the program is now…
Re:Porting Firefox to Apple Silicon by jarjoura on Wednesday January 20, 2021 @05:20PM (#25852727)
beta 10 also bricked my DTK unit as well. Total paper weight
Re:Porting Firefox to Apple Silicon by selykg on Wednesday January 20, 2021 @12:36PM (#25849280)
My experience working with antivirus vendors is... not good.

Product I used to work on had frequent false positives from antivirus software marking certain files as having some malware or whatever in it.

It's super unpleasant trying to get those changes pushed out. Glad that they were able to get some resolution quickly, usually that isn't the case, at least in my experience.

Re:Porting Firefox to Apple Silicon by bluedino on Wednesday January 20, 2021 @12:21PM (#25849058)
If Mozilla can’t get a hold of anyone...

20 years ago Google would have sent someone to Mozilla HQ for a week to work on stuff

Re:Porting Firefox to Apple Silicon by vlovich123 on Wednesday January 20, 2021 @12:49PM (#25849448)
This is the problem with jumping to conclusions from a vague excerpt. The vendor is some random anti-virus vendor and if you know what trying to get a handle there means....

> More of a concern was user reports that some antivirus software was flagging all our Universal Binaries as malware, and corrupting the Firefox installation the moment the update arrived.

> The software was using machine learning techniques and presumably observed that our combined Universal Binaries didn’t quite look like any other legitimate software it had ever seen before.

> Attempts to contact the vendor through regular support channels were unsuccessful so we ended up searching LinkedIn and managed to find an engineer working on the core antivirus detection.

Re:Porting Firefox to Apple Silicon by rebelwebmaster on Wednesday January 20, 2021 @01:50PM (#25850293)
It was Cylance made by Blackberry.
Re:Porting Firefox to Apple Silicon by mlyle on Wednesday January 20, 2021 @03:44PM (#25851743)
I had an awful experience with Cylance and some open source software I maintained, too-- false positive detections, and they wouldn't fix it.
Re:Porting Firefox to Apple Silicon by cozzyd on Wednesday January 20, 2021 @04:47PM (#25852435)
Their website is... blech.
Re:Porting Firefox to Apple Silicon by OnlyMortal on Wednesday January 20, 2021 @12:27PM (#25849120)
Apple did this when transitioning from CodeWarrior to the GNU chain. Apple had to apply patches to the C++ compiler for the company I was at.
Re:Porting Firefox to Apple Silicon by zerkten on Wednesday January 20, 2021 @01:54PM (#25850360)
This is surprisingly common for large companies. It often isn't formalized because no one wants to dedicate engineers to going onsite with customers.

Often the expert is too valuable to give up, or is a poor choice for customer engagement. If you have a consulting team, they may lack the experience needed, or reputation, that the customer wants. As soon as you send the expert onsite, you will have a challenge not sending them at a later time. This scares off engineering managers from lending their engineers because inevitably they have to fight off the requests.

The better workflow is one where you can send a less expensive resource (however you measure it) onsite and have them work remotely with the expert. If you can stick with that you often end up with the onsite person leveling up their skills and the ability to re-engage in a scalable manner. Any engagement needs prep before the onsite, a plan for escalation when onsite, and a disengagement plan.

Re:Porting Firefox to Apple Silicon by OnlyMortal on Wednesday January 20, 2021 @04:51PM (#25852473)
It was Agfa in Antwerp. Their guy was very polite but I had to explain to him what was going wrong. I think he was more tech support than anything else.

Edit: Apple also had to change the linker for the sake of Macromedia’s monolithic applications. Another story though.

Re:Porting Firefox to Apple Silicon by dpkonofa on Wednesday January 20, 2021 @10:03PM (#25854835)
I'm not sure I follow. Why would an anti-virus product accidentally flagging their Universal Binary block them from releasing a native Apple Silicon version? Of all the Macs in use, I'd have to imagine that only a small percentage are using that specific anti-virus software much less any anti-virus software at all. It might cause them some headaches with specific people who are using that software but why extend that all the way to blocking the release indefinitely?
Re:Porting Firefox to Apple Silicon by robin_reala on Wednesday January 20, 2021 @01:48PM (#25850267)
I guess this was Cylance: I ran into this one on my work laptop with Firefox Nightly and managed to get it escalated internally from us too.
Re:Porting Firefox to Apple Silicon by jmull on Wednesday January 20, 2021 @12:34PM (#25849248)
They are being very careful not to identify the anti-virus vendor.

So it's hard to tell if the size of the vendor is the issue here.

Re:Porting Firefox to Apple Silicon by drewg123 on Wednesday January 20, 2021 @09:02PM (#25854440)
I was doing Mac drivers about a decade ago, and some of the dumbest and most ignorant questions on the kernel and drivers mailing lists came from A/V folks. Things like "my machine locks up when our software does X", where it was clear that X was blocking the entire kernel waiting for a userpace thread. Ugh.

I've resolved to never, ever run A/V software on any machine I control based on the quality of those devs.

Re:Porting Firefox to Apple Silicon by slim on Thursday January 21, 2021 @01:48AM (#25856229)
because A/V folks are hackers, not engineers. hackers do impressive things without reading the fucking manual. that's why they can break seemingly unbreakable things. but on the surface, to the engineer, they seem uncompetent.
Re:Porting Firefox to Apple Silicon by 2Gkashmiri on Thursday January 21, 2021 @02:25AM (#25856419)
how do you "secure" machines if they are windows ones and dumb people plugin flash drives and click on big shiny download ram buttons. genuinely curious.
Re:Porting Firefox to Apple Silicon by icebraining on Thursday January 21, 2021 @07:06AM (#25858036)
One way to do it is to whitelist all binaries in the system, and sandbox all applications (to prevent chances of a malicious PDF/image/etc abusing a buggy application).
Re:Porting Firefox to Apple Silicon by 2Gkashmiri on Thursday January 21, 2021 @12:45PM (#25861670)
can you do that on windows? every single exe, every process?
Re:Porting Firefox to Apple Silicon by icebraining on Friday January 22, 2021 @04:58AM (#25870141)
Yeah, the security policies let you do that. I think the current mechanism is called AppLocker.

Note that there may be still ways to bypass it if you're an attacker sitting at the computer, rather than a hapless user.

Re:Porting Firefox to Apple Silicon by oauea on Thursday January 21, 2021 @06:31AM (#25857822)
Surprising they even went through with that. Personally I'd have said fuck it, blamed Apple and moved on with my life.

If Apple wants to create incompatible hardware, let them put the effort & money into fixing the software, if they want the software on their platform.

Re:Porting Firefox to Apple Silicon by m_st on Thursday January 21, 2021 @08:31AM (#25858586)
Though I get your point, in this particular case it was the antivirus company that was the problem. Not Apple.
Re:Porting Firefox to Apple Silicon by TwoBit on Wednesday January 20, 2021 @05:24PM (#25852774)
AV vendors and software are cancer.
Re:Porting Firefox to Apple Silicon by m_st on Thursday January 21, 2021 @08:36AM (#25858634)
I once did the same with Microsoft Windows 8 (or 8.1) keyboard layout changes breaking old software. You'd think they don't suddenly change from '.' to ',' as decimal separator. But that's what they did. And so did Apple more years ago by the way, even resulting in a calculator that couldn't be used anymore with the numeric keypad.

Anyway, found a guy working on keyboard layout stuff at Microsoft through LinkedIn as the other support channels were non-responsive. Unfortunately he just confirmed the change if I remember correctly. But at least we knew what was coming.

Re:Porting Firefox to Apple Silicon by tyingq on Wednesday January 20, 2021 @05:22PM (#25852743)
Oh wow. So in this case, a foundation with roughly half a billion in revenue per year is still somewhat of a little guy, at least for the standard process. I'm curious which antivirus vendor it was. Mozilla did eject a few extensions written by antivirus vendors in 2019, probably for good reasons.
Re:Porting Firefox to Apple Silicon by GekkePrutser on Wednesday January 20, 2021 @11:13PM (#25855297)
Even working for a big multinational with a big tailored support contact, support from our major IT services vendor is abysmal. It's not just the little guys that have trouble.
Re:Porting Firefox to Apple Silicon by ksec on Wednesday January 20, 2021 @03:00PM (#25851198)
>It’s notable that without this last-ditch effort we would have been effectively blocked from releasing a native Apple Silicon version for an indefinite period."

And they are Mozilla. Imagine Indies.

The Modern Day Apple requires you to get some Mainstream Media publish about How Apple block Open Sources Software to be running on M1 before Apple saw the PR damage and start acting on it.

Re:Porting Firefox to Apple Silicon by mlyle on Wednesday January 20, 2021 @03:45PM (#25851752)
This was about Cylance being jerks, not Apple. I've fought Cylance quite a bit on Windows for flagging open source software as malware, too (and they fail to respond / fix).
Re:Porting Firefox to Apple Silicon by account42 on Thursday January 21, 2021 @03:54AM (#25856884)
So what do you do if an antivirus vendor is uncooperative? Can you sue them for defamation?
Re:Porting Firefox to Apple Silicon by mlyle on Friday January 22, 2021 @02:08PM (#25874979)
Give up on your open source project, in my case.
Re:Porting Firefox to Apple Silicon by ttflee on Thursday January 21, 2021 @03:56AM (#25856908)
The issue itself sounds like a prologue for an A.I. dystopian story.
Re:Porting Firefox to Apple Silicon by gigatexal on Wednesday January 20, 2021 @04:11PM (#25852052)
It's not like Mozilla is a nobody software outfit. For this to be what they had to do go get someone's attention at Apple is terrible.
Re:Porting Firefox to Apple Silicon by lambada on Wednesday January 20, 2021 @05:27PM (#25852797)
It wasn't Apple they were trying to contact, it was a random anti-virus vendor who were flagging all new Universal Binaries as malware.
Re:Porting Firefox to Apple Silicon by gigatexal on Wednesday January 20, 2021 @10:29PM (#25855023)
Doh! My bad
Re:Porting Firefox to Apple Silicon by dubcanada on Wednesday January 20, 2021 @12:27PM (#25849130)
Obviously there is more to it then what I am going to say, and who knows with remote workers where people are.

But Apple and Mozilla headquarters are 5 miles apart (roughly). Couldn't you just walk/drive/scoot/fly/what ever over and talk to someone?

Re:Porting Firefox to Apple Silicon by djrogers on Wednesday January 20, 2021 @12:43PM (#25849383)
Setting aside that the problem with AV had nothing to do with Apple, for the most part nobody was working at either of those offices over the summer (2020, remember?). Also, which of Apple's 130+ Silicon Valley offices are you going to go to, and who do you ask for when you get there?
Re:Porting Firefox to Apple Silicon by dak1 on Wednesday January 20, 2021 @12:31PM (#25849182)
It sounds like this was an issue with an independent antivirus vendor, not Apple.
Re:Porting Firefox to Apple Silicon by milkytron on Wednesday January 20, 2021 @12:38PM (#25849304)
I'd imagine that security and the front desk are going to prevent anyone from entering to meet with specific teams or individuals without an appointment.

If you knew someone and had scheduled time with them, then yeah I'm sure you could hoverboard your way over.

Re:Porting Firefox to Apple Silicon by dblohm7 on Wednesday January 20, 2021 @01:14PM (#25849807)
Also, pandemic, remote workers, etc. gcp and the engineers who worked on this do not live in SV.