Won’t ChatGPT eventually eat your lunch? Once ChatGPT allows uploading documents (embeddings), what good will your app be?

The tools you’re talking about like math, wiki, or search are already built as plugins on ChatGPT.

I see so many AI apps being built, but I think ChatGPT will be general enough to cover 85-90% of use-cases using the chat UI.

It's more or less an advanced personal project to stay on top of the LLM learning curve, rather than just being exposed to news and press releases. I also have an appetite for further wrapping my mind around all of this. I already work in the AI space as web-developer on the B2B & enterprise side of things. My opinion here is that there are going to be loads of use-cases and necessary plugins, which for privacy, legal and security reasons need a proprietary solution and won't be able to interface with any third party APIs, plugins or frameworks.

I completely agree with you on AI being extensively integrated into existing apps and being leveraged that way.

But most of the AI apps I see are just like a “skin” on ChatGPT API.

I do think there is value in a universal chat UI that can connect to GPT-3 and other models.

I think that is a phase, which will come to pass, eventually most of the plugins and apps which are going to be popular, will likely run their own models or use open source models. Because the API calls for complex applications to OpenAI are currently far from economical. In the end you will have to charge the user, and that is going to be the crux. As a sole developer, doing loads of experiments on less than 50 documents, I am already crossing 50$ in API calls within a month. I can already run llama.cpp, but its just not good enough; but the cost would effectively be 0$ (not counting my hardware).

The most important aspect of langchain is NOT using OpenAI for the LM.

The most useful aspect of using langchain is to use it with Galpaca (or vicuna/koala/etc) to spin up an assistant for your home.

This way, you can push all of your files through it - even petabytes or terabytes of files, at a fraction of the cost - and have it organize things for you. No privacy problems, no extreme costs, just ease of use, low latency, offline, blazingly fast beauty. That's at least the trajectory.

Meta may soon release an improvement to Galactica similar to Galpaca (GeorgiaTech attempt) more officially (perhaps with more multimodal focus), which will likely improve upon the llama based models even further.

ChatGPT is just one model among many here, and it's not even the first to use RLHF (Deepmind, as usual, was a bit earlier).

The simple task of downloading Redpamajas/thePile/etc and getting a vector db for it locally, and enhancing it with local files effectively brings a local Google to everyone, and it may only require a decent spinning disk HD for the DB storage with the typical langchain LLM setup to have a completely local 'jarvis'-like assistant. (Sure, I know some people care about 'news'-like info that requires connectivity, but most things don't)

The vast majority of people building LM apps with (or without) LangChain are using OpenAI.

I sincerely hope local LM tech like Galpaca (or vicuna/koala/etc) succeed but I don't understand why we are collectively pretending they are currently anywhere near gpt-3.5-turbo both in terms of speed and quality. Honestly the local models feel more like first generation BERT/GPT-1 models that have been fine-tuned for QA using RLHF.

The models now are probably closer to GPT-2, and gets very close to GPT-3 with decent hardware (larger param models) and effort. They're certainly worse when it comes to 'knowledge' or memorized parts of the training data, but a decent bit of this is ameliorated by having "the pile" + other data locally available for reference.

There are some issues that arise from not having decent priors due to that lack of knowledge, which may or may not be important for the given task. A (perhaps somewhat bad) example may be: if you ask me "what is a good one-liner in bash for parsing xml from a stream", I may give you an answer using xmlstarlet. However, this may not be the best answer - since Xalan can handle XSLT version 3, but xmlstarlet can't (XSLTv3 handles streams).

So if looking up information in the database, some things may be slightly missed like that - but this behavior would be close to what ChatGPT offers (ChatGPT is quite awful in this way most of the time).

You are right that it would miss GPT-4 by a good bit though in these cases, but most people aren't using GPT-4 for this anyway.

Ultimately both can be used. OpenAI can do things you really want it for, such as a programming assistant, or things that may require much more "reasoning" (not a proper word, but conveys the message) capabilities.

Local models can do the really useful base work of completely re-organizing or re-encoding files to free up space if you set it to do so, integrating with a HomeAssistant system, setting up a HomeAssistant system if you want one, answering your vocal 'Alexa/Siri'-like questions completely offline, setting up backup solutions for all your computers, setting up servers that perform more tasks that you may want - essentially a complete personal assistant. OpenAI shouldn't be needed for this, and it is highly desired to not have them do any of this (due to costs and the number of credentials it would give to them).

We cannot give my company's information to OpenAI/MS. No legal paperwork will change this. This information is so important, it is only on offline computers.

I’m not convinced yet, but if they can provide nice templating and reusability.

There is plenty room for code reuse in prompting.

We've been building LLM chains for over a year and found that whilst for simple use-cases it's easy enough to grab a couple of APIs. However, having a Notebook experience built for iterating and collaborating whilst managing the complexity is something we've seen companies care deeply about.

I've experimented with LangChain for my chatbot as well, but ultimately, I resorted to using custom Python. Here are a few issues I faced with LangChain:

- By developing your own solutions, you can engineer specific components that would be provided by LangChain to better suit your use case. For example, by fine-tuning to your use case you can have better results with converation history, context and summarization better by prompt engineering. If you look at prompts within langchain they are pretty basic.

- LangChain is designed around the idea that an entire chat logic resides within a single "REPL loop." In my use case, I had a single-page web app frontend, a standard web "RESTful" backend, and a separate chat service. Different parts of the information are stored and managed by these components. Using LangChain would have forced me to consolidate all logic into the chat service, which doesn't align with the overall architecture of my system beyond just the chat functionality.

Please note that I'm not a LangChain expert, so my assessment might not be entirely accurate about its capabilities. However, based on my evaluation, LangChain introduced too many constraints in comparison to what it provided.

This, I just pretty much ended up using the basic LLMChain and do my own custom flow. The built in agents are close to useless for anything but a toy project; it is simply way too unreliable.