Re:Show HN: Pykoi – a Python library for LLM data collection...
by lmeyerov on Friday August 11, 2023 @05:05PM
(#37094016)
i was curious b/c we're building a lot of this inhouse for louie.ai just out of need
using the current seems unclear for us:
* we need to own the data & database, and align with our regular+vector infra -- where do they live here?
* we spend a lot of time on security annotations as the data isn't just for training but feeding back live in RAG, and in both cases, need rich expressivity for partitioning for sharing&tuning between different users/teams.. this seems to assume one big pile?
using the current seems unclear for us:
* we need to own the data & database, and align with our regular+vector infra -- where do they live here?
* we spend a lot of time on security annotations as the data isn't just for training but feeding back live in RAG, and in both cases, need rich expressivity for partitioning for sharing&tuning between different users/teams.. this seems to assume one big pile?