News for hackers. Stuff that matters.

A comment on Claude Sonnet 4 now supports 1M tokens of context. Parent comment

Re:Claude Sonnet 4 now supports 1M tokens of context by giancarlostoro on Tuesday August 12, 2025 @01:40PM (#44879527)
Here's a paper from MIT that covers how this could be resolved in an interesting fashion:

https://hanlab.mit.edu/blog/streamingllm

The AI field is reusing existing CS concepts for AI that we never had hardware for, and now these people are learning how applied Software Engineering can make their theoretical models more efficient. It's kind of funny, I've seen this in tech over and over. People discover new thing, then optimize using known thing.

[ Parent | on HN ]
Re:Claude Sonnet 4 now supports 1M tokens of context by mamp on Tuesday August 12, 2025 @02:33PM (#44880143)
Unfortunately, I think the context rot paper [1] found that the performance degradation when context increased still occurred in models using attention sinks.

1. https://research.trychroma.com/context-rot

[ Parent | on HN ]
Re:Claude Sonnet 4 now supports 1M tokens of context by giancarlostoro on Wednesday August 13, 2025 @11:14AM (#44889541)
Saw that paper have not had a chance to read it yet, are there other techniques that help then? I assume theres a few different ones used.
[ Parent | on HN ]
Re:Claude Sonnet 4 now supports 1M tokens of context by kridsdale3 on Tuesday August 12, 2025 @06:03PM (#44882313)
The fact that this is happening is where the tremendous opportunity to make money as an experienced Software Engineer currently lies.

For instance, a year or two ago, the AI people discovered "cache". Imagine how many millions the people who implemented it earned for that one.

[ Parent | on HN ]
Re:Claude Sonnet 4 now supports 1M tokens of context by giancarlostoro on Tuesday August 12, 2025 @09:28PM (#44883726)
I've been thinking the same, and its things that you don't need some crazy ML degree to know how to do... A lot of the algorithms are known... for a while now... Milk it while you can.
[ Parent | on HN ]
Re:Claude Sonnet 4 now supports 1M tokens of context by nxobject on Friday August 15, 2025 @07:31PM (#44918481)
What we need are "idea dice" or "concept dice" for CS – each side could have a vague architectural nudge like "parallelize", "interpret", "precompute", "predict and unwind", "declarative"...
[ Parent | on HN ]