News for hackers. Stuff that matters.

sq42na

We actually use Langchain under the hood for some things, and are in the process of figuring out which abstractions make sense to put on the SDK interface. For example, we already use the `langchain.schema.Document` class as the main document dataclass.

With making it possible to bring any LLM, we're considering adding support for bringing anything that implements the Langchain LLM interface.

In general, we're still trying to figure out how much customizability to give to the user and how (Langchain is a great option here) -- power should not come at the expense of ease of use.

It's embarrassingly simple and of course has infinite room for improvement, but I figured it might be interesting for you to check anyway: https://github.com/taivop/speed-copilot

It's built with OpenCopilot; the bulk of the work is in the prompt (you could probably copy-paste it into ChatGPT and get not-too-bad resultS). The RAG part is not great right now since the documents are not chunked in a reasonable way. But it has been somewhat valuable for me as well.

Let me know if you tried it out and whether it helps you do anything faster ;)

Yep, currently gpt-3.5-16k or gpt-4. We wrote the example prompts in a relatively Llama-compatible way though (we actually started building this onto Llama 1 before switching to OpenAI as default), and make few assumptions about the LLM so it's easy to switch out. Mostly this is waiting behind us adding an option to pass in any LLM, and we're planning to add support for this.

Generally, we leave the LLM up to the user -- if OpenAI or Google is a no-go, then you probably are anyway in the territory of self-hosting or even self-training your LLM, which means you're fine setting up your own inference endpoints as well.

RAG comes out of the box - see here: https://docs.opencopilot.dev/improve/knowledge-base

We were hoping to make dynamic RAG part of the very first release but didn't - we'll be adding that soon.

Sure! Taivo here - CTO & co-founder of OpenCopilot.

The most recent LLM app we've built has been the Ready Player Me copilot (which you can see linked in the main repo as well, and which was an inspiration for actually making something open-source based on what we had learned into). The problem it solves is onboarding developers onto their platform, and is currently live for a subset of their users.

We've built a ton of copilots that we haven't published. Some have been for work like founder copilots for ourselves and a few friends, sales outreach copilots, etc. Some have been for fun: I wanted to see if I could create a copilot that coaches me to do things faster a la https://patrickcollison.com/fast - so I did. As a PoC I created a copilot that explains the rules of an obscure sport I like (floorball).

In addition, with the same team, since the beginning of this year we've made a bunch of experiments that I'm sure many others have tried: * "chat with your knowledge base" style bots: for tech support, for developer docs, for internal HR use, and probably some I can't remember * an LLM-based quiz app that dynamically makes a quiz based on your Spotify history * an automated engineer (a much more basic version of smol-ai/developer) * a website builder where you mostly uses a chat UI for making the website * making AutoGPT better (higher quality, faster, more straightforward to use) * we also briefly considered making an OSS LLM before Llama 1 came out and kickstarted the OSS LLM wild ride we've seen since then

We killed almost all of these experiments within a few weeks after release, because none found early traction; we'd probably been wrong about the problem these were solving.

About my AI background: I built a 25-person computer vision team from zero at an identity verification unicorn. After that tried and failed to build a computer vision labelling tool startup. Before that, built robot perception software at Starship robots and unsuccessfully tried to make a contribution to AI safety with my thesis on active reinforcement learning in 2017.

About Johannes (OP) & the rest of the team: in 2018 him and part of the current team tried to make Sidekik: something like character.ai, but training models from scratch - which eventually didn't work out. From 2020 (after a pivot) the same team was building Sentinel, a deepfake-detection startup - which again did not work out.

FYI: Sidekik's original website: https://web.archive.org/web/20181008082142/https://sidekik.a...