Show HN: Orra – The missing glue layer for production-ready multi-agent apps

Re:Show HN: Orra – The missing glue layer for production-rea... by ezodude on Friday February 28, 2025 @08:15AM (#43205233)

Great question! We explored local LLMs (including llamafile-type solutions) in our early development, but found that the reasoning capabilities and consistency weren't quite there yet for our specific needs.

That's why we currently optimize for cloud AI models while implementing intelligent plan caching to significantly reduce API costs. This approach gives you the best of both worlds: high-quality execution plans with minimal API costs, plus much faster performance for similar actions.

You might find our documentation on plan caching interesting - it explains how we maximize efficiency: https://github.com/orra-dev/orra/blob/main/docs/plan-caching...

We're always evaluating new LLM options though, so I'd be curious to hear about your specific use case.

[ Parent | on HN ]

Re:Show HN: Orra – The missing glue layer for production-rea... by betimsl on Friday February 28, 2025 @01:24PM (#43208736)

Running a 7b coder in laptops with 4060 is possible and with very good results. Orra looks like a very good tool to be integrated with any IDE. Take a look at this: https://github.com/huggingface/llm.nvim -- it has a backend option. Ollama exposes a REST API, I think you guys should support it :)

[ Parent | on HN ]

Re:Show HN: Orra – The missing glue layer for production-rea... by ezodude on Saturday March 01, 2025 @09:43AM (#43219648)

Thanks! Will take a look.

[ Parent | on HN ]