riter

OP here.

i agree. perhaps you're confused on the intent. the only flag being planted is for folks using rasa looking for a reference implementation just like i was a week ago. not sure if you're being intentionally cynical but trying is good thing. why? bc most ppl don't try. you make 0 of the shots you never take. and of course, if you're not intentionally being cynical -- gucci. if you are i encourage you to make your next comment substantial or encouraging :)

LOL.

you're not alone my dude. i have a similar challenge w/ my engineers. my best lead is an artisan and its proud of what he authors, and like me sees their code as part of the product UX / funnel (1%). the others (99%) i have to get a bit draconian or simply create company templates they must adhere to or PRs get rejected and they hear from me on their 1:1s.

you either love it or you don't. and if you don't, follow the rules like a big boy or get called out.

ultimately you have to set a culture for it even if it is pulling teeth because you net net it impacts the PnL.

thanks for the share, will check out

lol

well to be fair, when you're scaling it does matter. i would want my techlead or seniors to care and know when/where to make specific trade-offs bc cloud costs are not forgiving.

i think that's where folks that make those comments are coming from.

in terms of search store and engine, would you agree that pgvector is sufficient for most text-specific cases?

very much agreed re: dust settling.

it makes no sense deploying any of these libraries to prod. as-is. best to understand a configuration / workflow / tuning / etc. that fits your data best and write it from scratch in golang/rust/whatever.

<3 ty, i totally misinterpreted the comment for cynicism. u guys are kind ty!

ah! in that case glad you asked.

my objective falls into neither bucket. i want rasa users to find it so i optimize for search (GH tags, clear description), ease of use (video, addt'l MD files) and perception (logos) but i'll be honest, for my intention it has a diminishing rate of return.

at minimum i find canonical README sections like quick start, installation, how it works is necessary if you want to be helpful. helpfulness is difficult to measure outside of inbound emails thanking you / forks w/ actual commits.

hope that gives some kind of insight. just make everything awesome :)

OP here.

that's a somewhat cynical interpretation. what if i just care about aesthetics and want to raise the bar.

my primary motivation was to get users of Rasa out of a directional hole bc that's where i was.

of course i like stars. it's a video game and i like winning. it was actually created in a few days all by me. no ulterior motive, literally indexing a solution to my problem from ~a week ago.

my bg is eng + product so i do these things as reflex and have a love for good UX.

nothing more. nothing less.

there are a few decent YT videos on this topic (dated 2022 though)

vast.ai is decent if you want to rent.

the next best platform I could find for my friend I was helping was google's dialog flow. again, it was managed, closed-source opinionated and not as flexible. and most importantly design considerations were for a pre-LLM world.

i personally think there is an acute opportunity for creating a bare bones rasa built with LLMs in mind. the core concepts behind rasa are useful (domains, intents, actions, etc.) but the underlying NLU technology and assumptions around the platform are obsolete so 70% of the footprint is unnecessary.

just my humble Ξ0.02

it solves how to integrate LLMs (Langchain) an application API pipeline with Rasa... of which I could not find an out-of-the-box public example on github. and so here we are :)

TL;DR: i solved a friend's headache (at the time)

yes. there are a few approaches which i intend to take and some helpful resources:

You could implement a Dual LLM Pattern Model https://simonwillison.net/2023/Apr/25/dual-llm-pattern/

You could also leverage a concept like Kor which is a kind of pydantic for LLMs: https://github.com/eyurtsev/kor

in short and as mentioned in the README.md this is absolutely vulnerable to prompt injection. I think this is not a fully solved issue but some interesting community research has been done to help address these things in production

very interesting abstraction. very DBT-esque. i will dig into the docs, thanks for sharing!

Amen. Constructive feedback to Langchain dev(s):

- Reduce bloat, make packages optional e.g. pip install langchain[all] - Reduce opinionated implementation of vector stores, I want my own schema - Don't unnaturally force the chain abstraction - Invest more in document retrieval

lol @ knowsitallkaren. i smell a winner.

I was referring to Langchain who raised $10mm from Benchmark

https://blog.langchain.dev/announcing-our-10m-seed-round-led...

I'm fairly Jerry Liu (LlamaIndex founder) already has angels or will see enough traction to warrant a seed.

To be clear (apologies if I haven't made it so) this is not an LLM. This is an implementation of Rasa leveraging Langchain under the hood.

A user technically does not need to dig into Langchain themselves, but they would want to if they find their query results sub-optimal.

There are a many indexing strategies and superficial parameters you could modify to tune output response. They are mentioned in the README.md.

I agree. I mentioned in a thread below that these frameworks are useful for discovering appropriate index-retrieval strategy that works best for you product.

On PGVector, I tried to use LangChains class (https://python.langchain.com/en/latest/modules/indexes/vecto...) but it was highly opinionated and it didn't make sense to subclass nor implement interfaces so in this particular project I did it myself.

As part of implementing with SQLModel I absolutely leaned on https://github.com/pgvector/pgvector :)

Thanks for the observation.

It itself is not a GPT. It is a a framework of a framework project built on top of Rasa (https://github.com/RasaHQ/rasa) and Langchain which by default uses gpt3.5-turbo (change it in the .env file) or any foundation model you wish.

lololol. i think this opportunity gets bigger post $10m seed round. they'll likely double down and expand footprint vs the inverse.

check out llama-index. its purpose-built for document indexing and retrieval and less agents and "everything else"

Not off-topic at all. After struggling with LangChain's hyper-opinionated implementation of classes I agree.

In fact, this is better off leveraging Llamaindex. This is a proof-of-concept and ultimately leveraging a library / framework helps afford the following:

- easy implementation of chunking strategies when you're unsure - OpenAI helper functions - embeddings and vector store management

Again, even with the above I struggled and had to implement PGVector myself. Going into production once I have my document retrieval strategy and prompt-tuning optimized, I would never use Langchain in production simply bc of the bloat and inflexible implementation of things like the PGVector class. Also the footprint is massive and the LLM part can be done in 5% of the footprint in Golang and 5% of the cloud costs.

So I actually agree with you :)

why is that exactly? is it offensive, if so I'm unaware and appreciate the feedback.

I appreciate the feedback. I didn't realize they were acting on it. Would Rasa-LLM sound as compelling?

Totally. Rasa (https://github.com/RasaHQ/rasa) is an open source chatbot platform.

It allows you to setup "Input Channels" e.g. slack telegram, and has an intents and response pipeline.

It leverages pre-LLM NLU models (NLTK, BERT, etc.) to score intents and based on that intent it will automate a pre-configured response.

My implementation allows you directly route (or fallback to) to GPT-3 or GPT-4 via Langchain document retrieval. So essentially this is an example of a knowledgebase customer support bot.

I hope that makes sense, let me know if not!

thanks for asking.

this implementation leverages Rasa and stands up a FastAPI server where it receives the user response webhook first and gets processed by (or bypasses) Rasa.

The LLM queries a set of documents indexed by Langchain. Dummy data has been included (Pepe Corp.)

Rasa has support for a "fallback" mechanism whereby if a user's response scores low on your pre-configured Rasa intents (like Greet) you can have it route directly to the LLM as well. But for now RasaGPT capture and routes the Telegram response to the FastAPI webhook endpoint.

the LLM itself and prompts I configured provides a boolean on whether the response should be escalated to a human or not, based on LLM+Langchain not knowing the answer to the user's query from the indexed documents.

I hope that answers your question, if not happy to follow-up!

Unfortunately there were not a whole lot of end-to-end examples of integrating Rasa with OpenAI nor functional boilerplates on github so I put a working prototype together in a few days and thus RasaGPT was bron.

RasaGPT is a python-based boilerplate and reference implementation of Rasa and Telegram utilizing an LLM library like Langchain for indexing, retrieval and context injection. FastAPI end-points are made available for you to build your application on top of. Features include:

- Automated hand-off to human if queries are out of bounds - "Training" pipeline done via API - Multi-tenant support - Generate category labels from questions - Works right out of the box with docker-compose - Ngrok reverse tunnel and dummy data included - Multiple use cases and a great starting point

Hope you like it, more @ rasagpt.dev

acid not necessary. ammonia.