TinyStories: How Small Can Language Models Be and Still Speak Coherent English? (2023)

Re:TinyStories: How Small Can Language Models Be and Still S... by Pamar on Saturday January 04, 2025 @10:02AM (#42595072)

Thanks... I also have another lingering doubt about the ability of RAG to make sense of "history", i.e. how to make sure that a more recent document on a given topic has more "weight" than older documents on the same issue.

I try to explain this a bit better here: https://pa-mar.net/Study/AiKiDo/VirtualBudoPass.html

[ Parent | on HN ]

Re:TinyStories: How Small Can Language Models Be and Still S... by ankit219 on Sunday January 05, 2025 @07:50AM (#42601434)

This is done at a reranking step. It's again custom. You have two variables - 1/ relevance (which most algos focus on) 2/ Date. Create a new score based on some combination of weights for relevance and date. Eg; Could be 50% of date. If the document has 70% relevance, but was published yesterday, it's overall score would be 85%. (A conceptual idea). This is similar to how you do weighted sorting anywhere.

[ Parent | on HN ]

Re:TinyStories: How Small Can Language Models Be and Still S... by Pamar on Tuesday January 07, 2025 @10:42AM (#42623570)

Thank you!

Btw, I notice only now that the link that was supposed to explain my question better is completely wrong.

https://news.ycombinator.com/item?id=42589014

(But you still provided much needed clarification).

[ Parent | on HN ]

Re:TinyStories: How Small Can Language Models Be and Still S... by ankit219 on Tuesday January 07, 2025 @12:11PM (#42624730)

No worries. If you need any specific help with the use case, please feel free to reach out on my email. ankit at clioapp.ai

[ Parent | on HN ]