VSCode is such a bloated hog of an editor!
Every time I open VSCode it’s bugging with badges to update extensions… and it’s so slow!
There's a "PrivateGPT" example in there that is similar to your third point above: https://github.com/jmorganca/ollama/tree/main/examples/priva...
Would love to know your thoughts
Things like bad default values, no tooltips, an no curated model list to one-click download is what separates a tool like Oobabooga from a paid commercial product. These things require time/money and it would be very unlikely that an open source tool could find resources for all the testing and R&D.
I think there is a big market for products where you pay and can just start chatting with the model without having to ever go to the settings tab or google anything unless you need to do something out of the ordinary.
That said, non-subscription is essential, and that's probably going to be a heavy lift considering how quickly things are evolving.
I've not yet been able to solve the challenge of needing CUDA etc for some models though!
Plugins so far: https://llm.datasette.io/en/stable/plugins/directory.html
lmstudio actually does what both of you want: provides an easy GUI and serves up your model over a local endpoint that mirrors the OpenAI API.
There's just too much noise in terms of the tooling for LLMs, the solution is fewer higher quality solutions, not more solutions
The concerns are 2 fold - 1. We might inadvertently use someone else’s intellectual property. 2. Someone else might gain access to our intellectual property.
What you are describing would help alleviate the concern about issue 2, but I’m not sure if it would help alleviate the concerns with issue 1.
I also don’t know any professional devs who have used tools like copilot and said they were anything but a toy. I am more bullish on LLMs than most of my coworkers. I think there is a lot of potential there. I do not see that potential in the current commercial offerings, and the financial outlay to fine-tune an open-source model and run it at scale is…prohibitive.
Really? I'm academic now but I find Copilot at least moderately helpful when I'm writing a library. It's pretty good a lot of boilerplate functions, docstrings, regex, etc. I certainly don't want to go back to not using it, my code is a lot closer to production quality now and looks nicer.
Thinking back to my days in back-end it seems like it would have been very helpful/sped things up so I'm surprised to hear it's just a toy but I've been out of the professional game for a while now. What's the main criticism?
Just like annual sexual harassment training - it's mostly corporate CYA on liability. If it ever goes to court, they'll plead ignorance and blame the employees who should have known better as they were trained/informed on what they ought not to do.
Paying for co-pilot could bite them though, so I suspect it's a case were the one part of the organization isn't aware of what the other is doing
The Copilot lawsuit should answer concern #1 more definitively.
#2 is already solved by running your own model or using Azure OpenAI.
- Easy plug & play model installation, and trivial to change which model once installed.
- Runs a local web server, so I can interact with it via any browser
- Ability to feed a model a document or multiple documents and be able to ask questions about them (or build a database of some kind?).
- Absolute privacy guarantees. Nothing goes off-machine from my prompt/responses (USP over existing cloud/online ones). Routine license/update checks are fine though.
I'm not trying to throw shade at the existing ways to running LLMs locally, just saying there may be room for an OPTIONAL commercial piece of software in this space. Most of them are designed for academics to do academic things. I am talking about a turn-key piece of software for everyone else that can give you an "almost" ChatGPT or "almost" CoPilot-like experience for a one time fee that you can feed sensitive private information to.