News for hackers. Stuff that matters.

A comment on Show HN: Real-time system that tracks how news spreads across 200k websites.

Re:Show HN: Real-time system that tracks how news spreads ac... by YmiYugy on Sunday November 30, 2025 @08:14AM (#46096377)
The idea is pretty cool, but it doesn't work super well. 1. I imagine most major news outlets don't have RSS feeds these days. 2. A lot of stuff originates from news agencies, so they don't spread from website to website, but radiate out from the agency. 3. Most of the included sources are pretty small. To draw meaningful conclusions we would need infos like popularity, political leaning, nation of origin, etc. 4. The similarity check doesn't appear to do translation. So when news spreads from one country to another we loose the thread.
[ Parent | on HN ]
Re:Show HN: Real-time system that tracks how news spreads ac... by badestrand on Sunday November 30, 2025 @08:29AM (#46096482)
The devil really is always in the details.
[ Parent | on HN ]
Re:Show HN: Real-time system that tracks how news spreads ac... by Joel_Mckay on Sunday November 30, 2025 @11:18AM (#46097893)
Being consistent in message framing even when its not in the best interest of the public should not reasonably be considered "news" =3

https://en.wikipedia.org/wiki/Sinclair_Broadcast_Group

https://www.youtube.com/watch?v=GvtNyOzGogc

[ Parent | on HN ]
Re:Show HN: Real-time system that tracks how news spreads ac... by andai on Sunday November 30, 2025 @12:43PM (#46098724)
>the similarity check doesn't appear to do translation

This surprises me. The system is based on embeddings. AFAIK embeddings cluster the same concept in different languages in roughly the same place? Maybe it depends on the model (or maybe it's not exact and the clustering cutoff loses it).

[ Parent | on HN ]
Re:Show HN: Real-time system that tracks how news spreads ac... by antiochIst on Monday December 01, 2025 @12:45PM (#46110420)
I'm basically throwing away non english articles for now... I'll pry get them in later, but I want to get english right first before trying to move to other languages...

The embeddings themselves will (pry) cluster ok in different languages (but I have not tested this yet)

[ Parent | on HN ]
Re:Show HN: Real-time system that tracks how news spreads ac... by fcarraldo on Sunday November 30, 2025 @12:47PM (#46098765)
> I imagine most major news outlets don't have RSS feeds these days

I’m not aware of any that don’t. RSS is alive and well.

[ Parent | on HN ]
Re:Show HN: Real-time system that tracks how news spreads ac... by dleeftink on Sunday November 30, 2025 @01:08PM (#46098940)
Also, not all information spreads through public channels, and might not even be/become publicly known. But that doesn't mean news refraction based on textual similarity isn't worthwhile to pursue, as it can reveal a lot about the self-organising principles by which the media operate.
[ Parent | on HN ]
Re:Show HN: Real-time system that tracks how news spreads ac... by Animats on Sunday November 30, 2025 @04:26PM (#46100589)
Yes. For example, this story about Ukraine [1] is credited to WNYT as first, but the story itself credits the Associated Press. This problem is worth solving, because it's something search engines should be doing.

[1] https://wnyt.com/ap-top-news/rubio-says-us-ukraine-talks-on-...

[ Parent | on HN ]
Re:Show HN: Real-time system that tracks how news spreads ac... by antiochIst on Monday December 01, 2025 @12:44PM (#46110407)
yea, what im currently doing is pretty simple check on published at date from the rss feed (with some small validation checks)... but its causing issues bc it can be wrong and mess up everything...

I think checking source in story is next step...

[ Parent | on HN ]
Re:Show HN: Real-time system that tracks how news spreads ac... by justin66 on Wednesday December 03, 2025 @06:19AM (#46133181)
Treating the Associated Press as a special case might be worthwhile. Its stories will appear in hundreds of places, some with a little alteration and some fully intact.
[ Parent | on HN ]
Re:Show HN: Real-time system that tracks how news spreads ac... by antiochIst on Monday December 01, 2025 @12:42PM (#46110388)
Yea not all major have rss feeds, but it seems like the majority still do.

No translation yet.

I think the biggest problem is im relying on published date from the news source itself too much and its wrong sometimes... not super often, but if 1 out of 100 sources get its wrong then it can steal credit for being source article when its not.

[ Parent | on HN ]