News for hackers. Stuff that matters.
Re:Ollama now supports AMD graphics cards by rkwz on Saturday March 16, 2024 @02:08AM (#39723726)
> I use it for RAG and as a fancy knowledge extraction engine

Curious, can you share more details about your usecase?

Re:Ollama now supports AMD graphics cards by idncsk on Saturday March 16, 2024 @07:43AM (#39725004)
Try ollama webui(now open-webui). Sry on my phone now => no links
Re:Ollama now supports AMD graphics cards by kergonath on Saturday March 16, 2024 @08:58AM (#39725504)
The use case is exploratory literature review in a specific scientific field.

I have a setup that takes pdfs and does some OCR and layout detection with Amazon, and then bunch them with some internal reports. Then, I have a pipeline to write summaries of each document and another one to slice them into chunks, get embeddings and set up a vector store for a RAG chat bot. At the moment it’s using Mixtral and the command line. But I like being able to swap LLMs to experiments with different models and quantisation without hassle, and I more or less plan to set this up on a remote server to free some resources on my workstation so the web UI could come in handy. Running this locally is a must for confidentiality reasons. I’d like to get rid of Textract as well, but unfortunately I haven’t found a solution that’s even close. Tesseract in particular was very disappointing.

Re:Ollama now supports AMD graphics cards by kergonath on Saturday March 16, 2024 @04:43PM (#39729331)
> layout detection with Amazon

Amazon Textract*.

It got lost in editing somehow