News for hackers. Stuff that matters.

mrfox321

Re:Kimi-K3 Releases on HuggingFace 7/27 on Monday July 27, 2026 @08:28AM (#49068704)
No. Distillation trains on a teach model's logits or output tokens.
... the toppings contains potassium benzoate
Re:Google can now read your WhatsApp messages on Tuesday July 08, 2025 @02:41PM (#44502794)
I know right?

They have always been behind. Why would this time be any different?

Re:Q-learning is not yet scalable on Sunday June 15, 2025 @09:26AM (#44282185)
Read the paper.

They control for the data being in-distribution

Their dataset also has examples of the problem being solved.

Re:OpenAI to buy AI startup from Jony Ive on Wednesday May 21, 2025 @01:50PM (#44054151)
Sounds like excuses
Re:Some Thoughts on Autoregressive Models on Friday March 07, 2025 @05:51AM (#43289112)
They didn't show this, they just increased the length where accuracy breaks down.
What's your leg circumference at, now?

Mine is also smaller, due to patella tendinopathy.

Re:What Happens When a Whole Generation Never Grows Up? on Thursday January 02, 2025 @02:10PM (#42577630)
Who do you think the shareholders are?

What demographic are they?

Re:ByteDance's Recommendation System on Friday December 20, 2024 @10:28AM (#42471909)
It's because they originally built their recommendation system to recommend friends and their content. Here, the social graph makes complete sense as the foundation for their simple search algorithm. But as they expanded their recommendation capabilities, the features stuck around. It's the same reason why tech debt accumulates. Data sticks around in the same way code does. But data is even higher friction, since it's a superset of the code.
Re:ByteDance's Recommendation System on Friday December 20, 2024 @10:14AM (#42471769)
I mean, it's their serving infra, but not their features. See my reply in another parent comment.
Re:ByteDance's Recommendation System on Friday December 20, 2024 @10:12AM (#42471728)
It's simpler intuition but more complex from a data / ml perspective.

Their algorithm is really built around their features. Specifically, temporal representations of user interest:

https://ieeexplore.ieee.org/document/9458799/

The features used by their algorithm tells you what a user is interested, historically.

Contrast this to Meta, which uses the social graph as their features. Imagine features like the number of times a user likes another author's / cluster's content.

Tiktok will serve you $TOPIC because you have $INTERACTED with $TOPIC historically.

Meta will serve you $TOPIC because you have $INTERACTED with $PEOPLE who post $TOPIC, historically.

Meta only coincidentally gives you what you like.

Tiktok knows what you like.

This is the difference. This is why IG is losing.

Re:ByteDance's Recommendation System on Friday December 20, 2024 @10:07AM (#42471671)
IG users and Tiktok users were / are quite similar. Especially when Tiktok wasn't yet eating IGs lunch.
Re:ByteDance's Recommendation System on Friday December 20, 2024 @09:08AM (#42471174)
IG has plenty of data (I did ML at IG). Don't be naive.
Re:ByteDance's Recommendation System on Friday December 20, 2024 @08:44AM (#42470991)
Not true. Why do IG reels and YouTube shorts suck, then?

They clearly built something superior. And it can't seem to be matched by the biggest tech companies.

At a big co I worked at, the lack of consistency between trading systems caused money to (dis)appear (into)out of thin air.

Prior to one of these hiccups, I hypothesized, given how shitty the codebase was, that they must be tracking this stuff poorly.

This led to an argument with my boss, who assumed things magically worked.

Days later, we received an email announcing an audit one one of these accounting discrepancies.

JPMC proposed using crypto, internally, to consistently manage cash flow.

Not sure if it went anywhere.

Second order methods, and their approximations, can be used in weight updating, too.
20 to 15 is insane.

Are these low 15s or high 15s? Regardless, that's fantastic progress.

Do you respond better to intervals or longer aerobic efforts?

I ask because Ive always wondered if training should be specialized for fast/slow twitch runners.

Specifically, do you focus on strengths or weaknesses.

Let me follow you on Strava :p

Re:An illustrated proof of the CAP theorem (2018) on Tuesday October 08, 2024 @08:07AM (#41776430)
simultaneity is relative.

That's all.

Re:Too much efficiency makes everything worse (2022) on Sunday September 29, 2024 @09:46AM (#41687384)
More importantly, he invented diffusion models:

http://proceedings.mlr.press/v37/sohl-dickstein15.pdf

Re:How America's universities became debt factories on Saturday September 14, 2024 @01:02PM (#41541052)
China does this, for better or for worse.
Similar, but specialized to softmax(QK)V computation
Re:How Olympic Timing Works on Sunday August 11, 2024 @08:15AM (#41215646)
These people want to win. It's not just society wanting to define winners.
Michael Phelps is an alien..
When you need the best possible model, full stop.

E.g. finance

In a sufficiently competitive space, good enough doesn't cut it.

Re:Show HN: How we got fine-tuning Mistral-7B to not suck on Thursday February 08, 2024 @08:00AM (#39301426)
I guess op may be envisioning an end-to-end solution that can train a model in the context of an external document store.

I.e. One day we want to be able to backprop through the database.

Search systems face equivalent problems. The hierarchy of ML retrieval systems are separately optimized (trained). Maybe this helps regularize things, but, given enough compute / complexity, it is theoretically possible to differentiate through more of the stack.

the police in America
Re:Veritasium: The SAT Question Everyone Got Wrong [video] on Tuesday December 26, 2023 @10:14PM (#38778706)
For the same reason why he brought up his iq tests in the first place.

Because he self-identifies with some definition of iq.

Re:An Interactive Guide to the Fourier Transform on Friday December 15, 2023 @01:01PM (#38656728)
Nit to your nit, which is incorrect w.r.t. parent reply:

A Fourier transform is not a projection, it's a change of basis represented by a unitary transformation.

Re:An Interactive Guide to the Fourier Transform on Friday December 15, 2023 @12:58PM (#38656702)
A square orbit would require infinite acceleration. Specifically, a dirac delta.
Re:Nearly 40% of U.S. homes are mortgage-free, census shows on Tuesday December 12, 2023 @09:35AM (#38612640)
This is my belief as well.

In California, the haves will have houses. The have nots will have ADUs.

Our world is structuring itself to (more) obviously reveal the wealth disparity.

Older »