News for hackers. Stuff that matters.
Re:Show HN: I made an app to use local AI as daily driver by eevmanu on Wednesday February 28, 2024 @03:00PM (#39542766)
Sure thing, your RAG approach sounds intriguing, especially since you're sidestepping vector databases. But doesn't the input context length cap affect it? (chatgpt plus at 32K [0] or gpt4 via open ai at 128K [1]) Seems like those cases would be pretty rare though.

[0]: https://openai.com/chatgpt/pricing#:~:text=8K-,32K,-32K

[1]: https://platform.openai.com/docs/models/gpt-4-and-gpt-4-turb...

Re:Show HN: I made an app to use local AI as daily driver by bigfudge on Friday March 01, 2024 @03:22AM (#39559750)
Yes, context window is a limiting factor, but that's true however you identify the content to augment generation.