News for hackers. Stuff that matters.
Re:Building a Local Perplexity Alternative with Perplexica, ... by rcarmo on Thursday August 01, 2024 @02:19AM (#41126609)
I just did that with docker-compose: https://taoofmac.com/space/notes/2024/06/23/1300

Much less fuss and zero weird stuff.

Re:Building a Local Perplexity Alternative with Perplexica, ... by flybird on Thursday August 01, 2024 @02:35AM (#41126669)
Cool stuff! Thanks for sharing. Docker Compose is indeed a nice method for fast deployment.

But like the difference between FTP and Dropbox

Our guide are designed to enable less-pro users to enjoy the benefits of selfhosting at home cloud.

A key advantage is fast deployment and flexible assemble of apps like Ollama and Perplexica.

In the meantime, compared to Docker Compose, Terminus provides a dedicated domain name for each application and service out-of-the-box. This enables users to access from anywhere via a browser. It will automatically handles all the complicated network configurations, DNS resoluton, and HTTPs certificate.

Re:Building a Local Perplexity Alternative with Perplexica, ... by Onavo on Thursday August 01, 2024 @03:15AM (#41126858)
Looking forward to your Sandstorm.io successor, k8s is an interesting choice.
Re:Building a Local Perplexity Alternative with Perplexica, ... by flybird on Thursday August 01, 2024 @03:27AM (#41126915)
Thank you for your acknowledgment! We will certainly do our best.
Re:Building a Local Perplexity Alternative with Perplexica, ... by arendtio on Thursday August 01, 2024 @05:35AM (#41127420)
Actually, this 'out-of-the-box' thing makes it complicated for me. Don't get me wrong; I like it when things just work.

But when I already have a setup (e.g. traefik + multiple docker-compose environments), I do want to understand what your out-of-the-box setup is doing. Otherwise, I risk that it kills my existing setup.

So far I stopped, after I saw, that https://terminus.sh is just a script to download another installer.tar.gz

Re:Building a Local Perplexity Alternative with Perplexica, ... by flybird on Thursday August 01, 2024 @05:56AM (#41127548)
Thank you for your valuable feedback.

During the installation process, Terminus requires over 110+ images. Considering Docker Hub's rate limiting, the script initially prompts users to download all images locally.

The script executes the actual installation command at the end. The installation process generally follows these steps: 1. Install K8S/K3S 2. Import images sequentially 3. Install system applications using Helm 4. Wait for system startup, then user activation

We understand your concerns. As a new system without an established reputation, we recognize it's challenging for users to try it out.

We've added installation methods for Windows WSL and Raspberry Pi, allowing users to test in virtual machines or temporary environments without concerns about their exiting setups.

Over the next two months, we plan to develop a graphical installation tool to further simplify the process.

Thanks again for your input.

Re:Building a Local Perplexity Alternative with Perplexica, ... by rcarmo on Thursday August 01, 2024 @03:48PM (#41132838)
I can do all of that with docker-compose and Cloudflare tunnels…
Re:Building a Local Perplexity Alternative with Perplexica, ... by connorgutman on Thursday August 01, 2024 @06:22AM (#41127687)
Wasn’t aware of LiteLLM, thanks! Are you using it instead of OWUI’s pipeline feature or alongside?
Re:Building a Local Perplexity Alternative with Perplexica, ... by flybird on Thursday August 01, 2024 @06:31AM (#41127736)
Does OWUI stand for Open Web UI?

My understanding is that Open Web UI and Perplexica each have their own focus.

However, through this tutorial, they can jointly use the same local SearxNG and Ollama service instance.

Re:Building a Local Perplexity Alternative with Perplexica, ... by connorgutman on Thursday August 01, 2024 @08:50AM (#41128613)
Yeah OWUI = Open Web UI. I was referring specifically to rcarmo’s setup though.
Re:Building a Local Perplexity Alternative with Perplexica, ... by yowlingcat on Thursday August 01, 2024 @07:46AM (#41128192)
Very cool. The pieces you put together make sense, but I have to ask -- why use Node-RED over Jupyter notebook for prototyping? It seems like your reason is that it gives you access to a suitably flexible low code environment that lets you visually program and experiment without needing to reach into code until you have a working prototype.

Is the improvement in productivity that drastic that you'd rather take that approach when possible? I haven't given something like Node-RED a try before because I've been under the assumption that about 60% through any kind of low code prototyping exercise, you begin to hit limits of inflexibility that makes it more painful than dropping into pure code in the first place, but I wonder if I've been overly hasty in this presumption.

Re:Building a Local Perplexity Alternative with Perplexica, ... by rcarmo on Thursday August 01, 2024 @03:44PM (#41132808)
Because:

- I can write JavaScript inside nodes

- When each event is a JSON blob, it’s trivial to handle data

- You can break down arrays/lists into series of discrete events, and switch them along different paths with simple filtering logic

- You can build sub-flows and re-use them trivially.

- You don’t have to worry with doing HTTP requests or handle database connections (you just receive the request results as a JSON object or create an object you send to a database node

So for me a basic flow is [template node with prompt]->[LLM API call]->[parse and branch out depending on outcome]->[store output], and I only write two snippets of JS inside a couple of nodes.

I do have Jupyter on my stack (another stack, too, with GPU access), but it tends to be linear, messy and impossible to leave running as a service. My Node-RED stuff is effectively “testing in production” and exposing HTTP endpoints as well, and I switch by prod/test by linking the right nodes…

But yeah, when I’m happy with the prompt or the flow I look at it and write some actual code. Just not in Jupyter.