News for hackers. Stuff that matters.
Re:Transformer Attention is off by one by smallnamespace on Monday July 24, 2023 @06:28PM (#36855137)
Papers are mostly read by other researchers, where the added background is actively bad because it obscures the real meat of the paper to the main audience.

If you just wanted a digestible intro then you would usually buy a textbook.

I think the argument that every research paper ought to be a mashup of a textbook + the actual research to be a bit silly from a “people should specialize at what they’re good at” standpoint.

Put in another context, I also don’t want every recipe to reintroduce what it means to “fry” or “braise” or “marinate”. We have Google for that.

Re:Transformer Attention is off by one by RationPhantoms on Monday July 24, 2023 @09:18PM (#36856573)
I've long wanted an informational slider to bits of text. Something where you can zoom in and out to the level of desired complexity. LLM's might be able to fill in some of those gaps. You could turn any paper into a introduction of the subject it's a part of.
Re:Transformer Attention is off by one by tonypace on Monday July 24, 2023 @11:31PM (#36857528)
This sounds like a good use case for local llm models. Browser plugin, precooked prompts for different levels of detail, maybe a lora to give the model some idea of expected output. I bet some of the 13b models could do a useful job on this even if they were imperfect.
Re:Transformer Attention is off by one by jdougan on Tuesday July 25, 2023 @07:55PM (#36871346)
Look for "stretchtext"