News for hackers. Stuff that matters.
Re:Apple's On-Device and Server Foundation Models by cube2222 on Monday June 10, 2024 @06:44PM (#40640114)
Halfway down the article contains some great charts with comparisons to other relevant models, like Mistral-7B for the on-device models, and both gpt-3.5 and 4 for the server-side models.

They include data about the ratio of which outputs human graders preferred (for server side it’s better than 3.5, worse than 4).

BUT, the interesting chart to me is „Human Evaluation of Output Harmfulness” which is much, much ”better„ than the other models. Both on-device and server-side.

I wonder if that’s part of wanting to have gpt as the „level 3”. Making their own models much more cautious, and using OpenAI’s models in a way that makes it clear „it was ChatGPT that said this, not us”.

Instruction following accuracy seems to be really good as well.

Re:Apple's On-Device and Server Foundation Models by crooked-v on Monday June 10, 2024 @07:17PM (#40640419)
I want to know what they consider "harmful". Is it going to refuse to operate for sex workers, murder mystery writers, or people who use knives?
Re:Apple's On-Device and Server Foundation Models by its_ethan on Monday June 10, 2024 @07:42PM (#40640620)
The caption for the image gives a little more insight into "harmful" and one of the things it mentions is factuality - which is interesting, but doesn't reveal a whole lot unless they were to break it out by "type of harmful".
Re:Apple's On-Device and Server Foundation Models by hotdogscout on Monday June 10, 2024 @10:30PM (#40641658)
I bet it's the usual double standards the AI one percenters cater to.

No sex because apparently it's harmful yet never explained why.

No homophobia/transphobia if you're Christian but if you're Muslim it's fine.

Re:Apple's On-Device and Server Foundation Models by arthur_sav on Tuesday June 11, 2024 @01:44AM (#40642824)
They'll inject whatever ideology / dogma is "the current thing" into this.
Re:Apple's On-Device and Server Foundation Models by HeatrayEnjoyer on Tuesday June 11, 2024 @03:02AM (#40643385)
Isn't "current thing" a phrase used by right winger anti-vax anti-LGBT types?
Re:Apple's On-Device and Server Foundation Models by mvandermeulen on Tuesday June 11, 2024 @03:25AM (#40643535)
Do they have exclusive rights?
Re:Apple's On-Device and Server Foundation Models by soygem on Tuesday June 11, 2024 @03:36AM (#40643590)
Yes, what's your point?
Re:Apple's On-Device and Server Foundation Models by HeatrayEnjoyer on Saturday June 15, 2024 @02:02AM (#40687800)
Repeating right wing rhetoric is corrosive and a red flag that the user is at best repeating it out of ignorance, but likely knows very well what they are doing. We don't go pouring HCL on things we care about, nor do we accept the presence of those who do.
Re:Apple's On-Device and Server Foundation Models by rvnx on Tuesday June 11, 2024 @04:31AM (#40643896)
"Think Different"
Re:Apple's On-Device and Server Foundation Models by m463 on Tuesday June 11, 2024 @02:36AM (#40643205)
Bet it depends on the country.

In the USA, you won't be able to ask about sex, but you can probably ask about tank man.

Re:Apple's On-Device and Server Foundation Models by jeroenhd on Tuesday June 11, 2024 @02:39AM (#40643219)
I would've thought the same until Microsoft started hiding tank man results in Bing. I'm not so sure if companies will start training different models for every oppressive regime.
Re:Apple's On-Device and Server Foundation Models by t-writescode on Tuesday June 11, 2024 @03:06AM (#40643415)
Oh?

https://www.bing.com/images/search?q=tank%20man

Looks visible, to me.

Tiananmen Square even shows Tank Man on the first page, 13th and 15th entry, for me. Admittedly, I expected it more quickly on Tiananmen Square; but that might be because I, as a person, forgot that it's also a literal square with more stuff going on at it than a single moment in history.

Re:Apple's On-Device and Server Foundation Models by lelandfe on Tuesday June 11, 2024 @08:45AM (#40645586)
2021, a classic, “we made a whoopsie” response.

https://theguardian.com/technology/2021/jun/04/microsoft-bin...

Re:Apple's On-Device and Server Foundation Models by Aerbil313 on Tuesday June 11, 2024 @09:01AM (#40645735)
None of the use cases they presented in WWDC using Apple Intelligence was creative writing. There is one, that uses ChatGPT explicitly:

> And with Compose in Writing Tools, you can create and illustrate original content from scratch.

https://www.apple.com/apple-intelligence/

Re:Apple's On-Device and Server Foundation Models by tonynator on Monday June 10, 2024 @09:24PM (#40641270)
So it's not going to be better than other models, but it will be more censored. I guess that might be a selling point for their customer base?
Re:Apple's On-Device and Server Foundation Models by dghlsakjg on Monday June 10, 2024 @09:39PM (#40641362)
iPhone share is ~59% of smartphones in the US.

Their customer base is effectively all demographics.

Re:Apple's On-Device and Server Foundation Models by tonynator on Monday June 10, 2024 @10:12PM (#40641558)
Those who dislike censorship and enjoy hacking avoid iPhones for obvious reasons.
Re:Apple's On-Device and Server Foundation Models by overstay8930 on Monday June 10, 2024 @10:19PM (#40641604)
People who understand cybersecurity hygiene use iPhones for obvious reasons
Re:Apple's On-Device and Server Foundation Models by realusername on Tuesday June 11, 2024 @02:39AM (#40643227)
Those cybersecurity experts are using GrapheneOS and certainly not an iPhone where they can't even check if all is going well...
Re:Apple's On-Device and Server Foundation Models by jitl on Tuesday June 11, 2024 @09:19AM (#40645865)
I’ve worked with a few people from NCC Group, Matasano, security staff at Airbnb and OpenAI, all carry and recommend iPhones for security footing. Depending on threat model, “lockdown mode” on iOS has a lot of what is useful in Grapheme like turning off built in connectivity services & disabling JIT and other code paths in the webview.
Re:Apple's On-Device and Server Foundation Models by realusername on Tuesday June 11, 2024 @10:20AM (#40646475)
To each their own, I certainly would not recommend an iPhone in this scenario, especially even more at a top tech company.
Re:Apple's On-Device and Server Foundation Models by Hugsun on Tuesday June 11, 2024 @03:02AM (#40643388)
The reasons are very not obvious to me. Could you elaborate?
Re:Apple's On-Device and Server Foundation Models by astrange on Tuesday June 11, 2024 @03:01PM (#40650115)
iPhone security is very good, better than, say, your desktop even though you don't carry your desktop everywhere you go. (Some Android security is also very good, depends on the hardware though.)
Re:Apple's On-Device and Server Foundation Models by overstay8930 on Tuesday June 11, 2024 @11:27PM (#40654157)
There is no other ecosystem that will provide easy E2EE in the cloud, that’s reason enough for the vast majority of users who are just sick of breaches because of pure sysadmin incompetence.

There are plenty of examples in every Apple service or even their accessories (i.e. wireless keyboard encryption). FaceTime is hardened from even theoretical attacks that could probably only be performed by 5 eyes like transcribing E2EE calls based on bandwidth use (FaceTime has built mitigations around this attack vector).

Re:Apple's On-Device and Server Foundation Models by binkethy on Tuesday June 11, 2024 @04:00AM (#40643735)
It would seem that integrating a backdoor funnel to OpenAI is a bit of a security issue to those who care about such things.

Yay, we can all train corporate models for free involuntarily.

I guess it's time to check out Lineage OS and Postmarket OS. It was always a matter of time.

Re:Apple's On-Device and Server Foundation Models by scarface_74 on Tuesday June 11, 2024 @06:16AM (#40644499)
No one is going to train an AI on random user generated data. The data is going to be horrible and it’s going to be full of PII that’s too risky to expose.
Re:Apple's On-Device and Server Foundation Models by Aerbil313 on Tuesday June 11, 2024 @09:05AM (#40645770)
People who understand cybersecurity who are not operating within a US-allied country use ... I don't know what to be honest. What to do in such a situation, where Apple is a US-based company obligated by law to comply with requests from three letter agencies and Android is a buggy mess which probably is backdoored by every major power?
Re:Apple's On-Device and Server Foundation Models by astrange on Tuesday June 11, 2024 @03:00PM (#40650102)
Your threat model shouldn't be about three letter agencies unless you're running a terrorist group. It should be about porn spammers or something.
Re:Apple's On-Device and Server Foundation Models by Aerbil313 on Tuesday June 11, 2024 @06:30PM (#40652395)
Do you mean that US is the measure of justice and goodness in the world and anyone who opposes her is a terrorist? I don't mean to detract the conversation but this is really victim blaming. There are valid situations where the US DoD is a threat.

Your username reminded me of Julian Assange, for a second I mixed them up.

Re:Apple's On-Device and Server Foundation Models by astrange on Tuesday June 11, 2024 @06:42PM (#40652503)
No, I assume if you're working for a foreign military then you wouldn't be posting about it on here.
Re:Apple's On-Device and Server Foundation Models by Aerbil313 on Tuesday June 11, 2024 @07:07PM (#40652702)
I assume if I was working for a foreign power I'd already know the answer to my original question... unless there's no good answer.
Re:Apple's On-Device and Server Foundation Models by Der_Einzige on Tuesday June 11, 2024 @10:28PM (#40653877)
I can assure you that if you’re doing AI research, glowies likely spy on you, but it’s the “good” kind of spying where you might get a job offer some day instead of the kind where they Guantanamo bay you…
Re:Apple's On-Device and Server Foundation Models by Aerbil313 on Friday June 21, 2024 @11:46AM (#40750765)
Unless of course you are not based in a US-allied country, as stated very explicitly in my comment.

Ah the neverending American centricism on this site.

Re:Apple's On-Device and Server Foundation Models by overstay8930 on Tuesday June 11, 2024 @11:18PM (#40654109)
Sorry to break it to you but you cannot hide from the US government if you use modern hardware or connect to the internet in any way.
Re:Apple's On-Device and Server Foundation Models by Aerbil313 on Thursday June 13, 2024 @01:36PM (#40672364)
How likely is it that seemingly well-intentioned companies/open source projects like Mullvad, Wireguard, Qubes OS, Monero etc. are infiltrated by the US government?

Or are you talking about the presence of Intel ME and the like in modern hardware?

Re:Apple's On-Device and Server Foundation Models by overstay8930 on Saturday June 15, 2024 @11:12PM (#40694414)
I’m talking about the simple fact that Five Eyes have so many resources put together that you will never hide forever, unless you are willing to drastically change your life.

The good news is you are not important enough for them to care about you unless you are an Iranian general with nuclear access or some shit. Even these ransomware groups aren’t even on the radar of the people who are actually being targeted, I’m talking about stopping the yakuza trying to sell nukes to terrorists level of threat.

Re:Apple's On-Device and Server Foundation Models by Aerbil313 on Friday June 21, 2024 @12:00PM (#40750890)
I appreciate you, and thanks for the "You're not that important" reminder for the average reader of this comment. However I must state I personally really am in need of an answer. Hypothetically, what if I am that important? What if I am not yet a target but think there's a decent chance that I will be targeted by nation-states, the most capable being US one day? You seem pretty knowledgeable. In the age of ML-assisted mass data scanning and ISP data collection & correlation, what would the lifestyle changes I must undergo to remain truly pseudonymous on the internet look like?
Re:Apple's On-Device and Server Foundation Models by duxup on Monday June 10, 2024 @10:19PM (#40641599)
I feel that way, have an iPhone.
Re:Apple's On-Device and Server Foundation Models by ihumanable on Tuesday June 11, 2024 @02:06PM (#40649508)
Yea, same.

I have literally no desire to hack and fuck around with my personal cell phone, doing so would take away time from the hacking I actually want to do.

Re:Apple's On-Device and Server Foundation Models by tonynator on Wednesday June 12, 2024 @01:32PM (#40660730)
One option you can't hack on, the other you can but don't have to at all. Why choose the former? Are you so sure you'll never want to do anything on your phone that's unapproved by your corporate overlords?
Re:Apple's On-Device and Server Foundation Models by gfourfour on Tuesday June 11, 2024 @01:39AM (#40642796)
How does an iPhone contribute to censorship?
Re:Apple's On-Device and Server Foundation Models by tonynator on Wednesday June 12, 2024 @01:33PM (#40660743)
Re:Apple's On-Device and Server Foundation Models by causal on Tuesday June 11, 2024 @01:51AM (#40642883)
Refusing to answer any question would result in a perfect score for the first chart since it says nothing of specificity