Re:Ask HN: What is the current (Apr. 2024) gold standard of ...
by aliasaria on Monday April 01, 2024 @10:02AM
(#39894285)
You can chat with the models directly in the same way you can chat with GPT 3.5.
Many of the opensource tools that run these models let you also edit the system prompt, which lets you tweak their personality.
The more advanced tools let you train them, but most of the time, people are downloading pre-existing models and using them directly.
If you are training models, it depends what you are doing. Finetuning an existing pre-trained model requires lots of examples but you can often do a lot with, say, 1000 examples in a dataset.
If you are training a large model completely from scratch, then, yes, you need tons of data and very few people are doing that on their local machines.
Re:Ask HN: What is the current (Apr. 2024) gold standard of ...
by montgomery_r on Monday April 01, 2024 @10:03AM
(#39894296)
+1 on these questions.
Can I run a local llm that will, for example
- visit specified URLs and collect tabular data into csv format?
- ingest a series of documents on a topic and answer questions about it
- ingest all my PDF/MD/Word docs and answer questions about them?
Re:Ask HN: What is the current (Apr. 2024) gold standard of ...
by aliasaria on Monday April 01, 2024 @10:19AM
(#39894421)
Some of the tools offer a path to doing tool use (fetching URLs and doing things with them) or RAG (searching your documents). I think Oobabooga https://github.com/oobabooga/text-generation-webui offers the latter through plugins.
Our tool, https://github.com/transformerlab/transformerlab-app also supports the latter (document search) using local llms.
What do you do with one of these?
Does it generate images? Write code? Can you ask it generic questions?
Do you have to 'train' it?
Do you need a large amount of storage to hold the data to train the model on?