News for hackers. Stuff that matters.
Re:Apple has new Mac app coming to make running local AI mod... by asdff on Tuesday November 10, 2026 @07:06PM (#50448089)
Why is there a desire to have this stuff on the edge on the engineers lap instead of just building some arbitrarily scalable cluster for I'm guessing cheaper than apple's usual premium? Habitual practice from the engineers who would compile on the device getting hot in their lap? Lack of widespread hpc experience compared to running docker?
Re:Apple has new Mac app coming to make running local AI mod... by bix6 on Tuesday November 10, 2026 @07:33PM (#50448305)
People don’t want to send all their private company data to the model providers?
Re:Apple has new Mac app coming to make running local AI mod... by bronco21016 on Tuesday November 10, 2026 @07:37PM (#50448339)
Because why not? If you can get unlimited inference with roughly Opus 5.5 intelligence for $10k that easily beats current API pricing. Like not even a question.
Re:Apple has new Mac app coming to make running local AI mod... by codebje on Tuesday November 10, 2026 @07:43PM (#50448397)
Distrust in allowing an unreliable vendor to control the fate of your business?
Re:Apple has new Mac app coming to make running local AI mod... by aurmc on Tuesday November 10, 2026 @07:45PM (#50448413)
Many users would prefer to use a Sonnet-level (or better) LLM without sharing their data with others.

And with a device like this, you pay one single time to buy the product and then never have to think about paying for tokens ever again. For ordinary users that's silly, but for users or businesses chewing through lots of tokens, this means they can create their own turnkey replacement for "the cloud" where the only cost after purchasing the machine is the electricity to run it (which with these machines is very little). Mix that with the privacy benefits and suddenly it makes a lot of sense.

Re:Apple has new Mac app coming to make running local AI mod... by asdff on Tuesday November 10, 2026 @09:53PM (#50449253)
Seems everyone in this thread is assuming I am saying go to private third party compute. No. Build your own cluster on metal you own on property you own, and have your team use that and just some thin clients to write with. Your compute goes way further. Your cluster can be going 100% of the time with jobs queued up by your team. All that memory you buy for your team in macbooks is idle and unavailable when your employee closes the lid and tosses it into the bag into the car on the way home. It could be used 24/7. On hardware you own. For cheaper to build I'm sure than apple hardware tax. On easier OS to manage and no doubt deal with anything involving any sort of heavy compute work than OS X. Are we really going to waste resources with docker when we could be using a node completely? Come on.

Like I said, seems HPC is lost art. All four sub replies missed that in my original comment.

Re:Apple has new Mac app coming to make running local AI mod... by monegator on Wednesday November 11, 2026 @01:50AM (#50450403)
yes. People do not read.

I agree with you, and i would like to set up a server at work for local inference but we simply do not have the knowledge, it's much easier to get a couple of AI Max machines that are good enough for me and the boss which i convinced on the good points on running clankers locally and be done with it (though my plan is to use the AI MAX machines to figure out what the infrastructure and workflow will look like, and at some point ask for the investment)

Re:Apple has new Mac app coming to make running local AI mod... by rick1290 on Wednesday November 11, 2026 @02:02AM (#50450481)
this is neat. how has it been going with the ai max machines? also looking to do the same as id like to get ahead.
Re:Apple has new Mac app coming to make running local AI mod... by monegator on Wednesday November 11, 2026 @03:49AM (#50451090)
I don't know yet! The boxes have been sitting there for a while. It's been a crazy month
Re:Apple has new Mac app coming to make running local AI mod... by monegator on Saturday November 14, 2026 @10:59AM (#50498053)
ok, a colleague started playing with the AI MAX 395, qwen 3.8 flash next unsloth running at 40tk/s

damn, too bad that the laptops with the 470 are capped at 64GB

Re:Apple has new Mac app coming to make running local AI mod... by throwuxiytayq on Wednesday November 11, 2026 @02:30AM (#50450674)
> but we simply do not have the knowledge

How is this literally even possible in 2026? In a thread about LLMs, no less?

Re:Apple has new Mac app coming to make running local AI mod... by monegator on Wednesday November 11, 2026 @03:47AM (#50451073)
assimilating knowledge still take time, i do not take whatever LLMs regurgitate at face value.

Then there's testing the approaches and see whatever sticks best.

Re:Apple has new Mac app coming to make running local AI mod... by dansquizsoft on Wednesday November 11, 2026 @06:32AM (#50452178)
Yeah this is a wild take…
Re:Apple has new Mac app coming to make running local AI mod... by throwaway277432 on Wednesday November 11, 2026 @01:53AM (#50450423)
It's investors making these decisions. They would rather have predictable cloud spend than trying any experiments with high cap-ex. Has been this way since AWS is a thing.

Not saying they're right.

Re:Apple has new Mac app coming to make running local AI mod... by eddythompson80 on Wednesday November 11, 2026 @02:11AM (#50450544)
Your main argument is about better allocation for that RAM, which only makes sense in the conditions of a shortage. If RAM is plentiful, like it’s bound to be once the datacenters craze is over, that argument becomes “why does every apartment need a washer/dryer when we can have shared washers/dryers in a common area that are being used 100% of the time”. The answer is “because we can and it’s not that huge of a difference” plus you can have both. Like you can have a server cluster for large batch jobs, and personal ones for personal use.

For individuals though, it’s the same reason plenty of people pay for the 2tb or 4tb SSD for their laptops or desktops even though NAS has been a thing for 40 years. Maintaining a “server” of any sort is a completely separate endeavor for most. Just like a NAS is a no-brainer for many and non-starter for many more.

Re:Apple has new Mac app coming to make running local AI mod... by flaunf221 on Wednesday November 11, 2026 @04:04AM (#50451169)
RAM was dirt cheap before and yet until M3 Apple made laptops starting with 8 GB of RAM. Why do you think Apple will suddenly become generous later?

The problem with LLMs outside of server is that people typically have many devices. So you can run LLM on your laptop. You want to make a query from your phone. Start your laptop? You want to do something from your smart home? Start your laptop? What about your partner? Will they need to buy another top of the line laptop for themselves? Your kids?

Alternative to server is of course that everything will have over 100 GB of memory, eat battery like crazy, be dumber and slower than single server in your household.

4 TB ssd was like $300 until recently. Even if RAM prices were to go to previous values, I don't think AI capability will be any comparable. Besides you also need proper compute, unless you want your prefill to be snail slow.

Re:Apple has new Mac app coming to make running local AI mod... by eddythompson80 on Wednesday November 11, 2026 @04:30AM (#50451336)
It has nothing to do with generosity. Apple (and other laptop and PC makers) just have different segments of the market that they catered for. Plenty of people use laptops just to browse the web. They didn’t need a lot of RAM. Those who needed the RAM (usually branded as pro-users or “creators” or engineers) always had the option to get more RAM both from Apple and others at a premium. It’s just that ~32GB was the upper amount for almost any use case. Why would Apple put 256GB or 512GB in every laptop back when there was no use case for it?

Every scenario you described applies the same to a NAS. What if you need a file from your phone? Open your laptop? What if your partner needs a file? Open your laptop? The answer to all of them is either yes, or they have it on their laptop/phone too.

Apple (and other laptop makers) weren’t charging $300 for 4tb even before the shortage. They realized that not everyone needed that and they way overcharged for that but still less than today’s prices. People who needed it bought it even though it would have been a lot cheaper for them to build a NAS. A current-gen M5 is perfectly capable of doing decent tk/s given enough RAM. In 2 or 3 years it would be even more capable, it just needs RAM.

Plenty of people will use clou hosted models like they do with storage today. Plenty of people will build home servers for AI. Just like plenty of people have home NASs. But the number of people who just want a large SSD on their laptop make the previous group a rounding error and those people would rather have a beefy laptop or desktop with enough RAM in it. If anyone can solve the Phone<->Desktop communication seamlessly it’s Apple after all.

Unlike storage however, a lot of people are now aware of how much data collection tech and AI companies are doing and the idea that AI models are training on their personal photos and prompts is deeply unappealing to many. With just storage, what could Google or Apple do? Certainly they wouldn’t just print your photos and put them in an ad or on their website. But with AI models, it’s a different equation

Re:Apple has new Mac app coming to make running local AI mod... by flaunf221 on Wednesday November 11, 2026 @05:10AM (#50451643)
> Every scenario you described applies the same to a NAS.

No. Files are your property. Inference is a service.

There's also the fact that working with files over network is often ill-advised and borderline impossible for some workloads. Even when that NAS is connected over wire and not wireless. Meanwhile even if you're on so-so connection, LLMs work great.

Re:Apple has new Mac app coming to make running local AI mod... by eddythompson80 on Wednesday November 11, 2026 @05:16AM (#50451707)
And storage can’t be a service?
Re:Apple has new Mac app coming to make running local AI mod... by flaunf221 on Wednesday November 11, 2026 @06:02AM (#50451989)
A. Like I said - unless you invented magical communication channel that will give latency and speed and and reliability of local NVMe drive to your remote NAS, no, remote storage can't replace local storage in full.

NAS is mostly good for backups, torrenting linux isos and serving them with Plex or Jellyfin. You are not installing software on your NAS instead of your machine. Or doing video editing. Or playing demanding games. I have file server at home and I still have ~8 TB of storage in my desktop, of which at leat 3 TB I can't move out.

B. If I have 1 TB of my work files, my wife has 1 TB of her work files, that's 2 TB of total storage. If both us need LLM, it doesn't mean we need twice the LLM as one of us.

Re:Apple has new Mac app coming to make running local AI mod... by Gigachad on Wednesday November 11, 2026 @01:51AM (#50450411)
I don’t want it in my laptop, I want it in my mini server rack at home. I’m not running any of my personal data through an external service that is certainly collecting it as training data.
Re:Apple has new Mac app coming to make running local AI mod... by usrnm on Wednesday November 11, 2026 @02:44AM (#50450760)
I want it in neither my laptop nor my home server, which I don't even have, I want it in my phone or something even smaller. I understand that it's currently not possible, but this is the direction I want the technology to be moving into.
Re:Apple has new Mac app coming to make running local AI mod... by throwaway_new_1 on Wednesday November 11, 2026 @02:59AM (#50450828)
Highly likely it will happen. But at that point the phone will likely just be a slab of display and not much of an interface. Everything is done through an LLM and showed on the display. Not sure if touch screen will even be required at that point.
Re:Apple has new Mac app coming to make running local AI mod... by asdff on Wednesday November 11, 2026 @12:52PM (#50456260)
Great, because that's exactly what I suggested you do when I said "hpc" and not "anthropic subscription"
Re:Apple has new Mac app coming to make running local AI mod... by _ph_ on Wednesday November 11, 2026 @03:33AM (#50451000)
First of all, it seems to be possible to build a scalable cluster from TB5 Macs. They don't have to be on individual desks, they could be hosted in a data center.

But on the other side, equipping data centers was never Apples main business. Their approach to markets is to design products for the individuals. They also usually have very long time plans. Look at where the Apple Silicon today is and how long a path it took. What about the day, where at least a medium sized Mac becomes a great platform for running large models locally? Very easy to use, no setup, no privacy concerns. Also never worry about "tokens". You can then use AI as you use other local resources of the hardware you own today.