ashmil

same thing happened to me. was on codex for about a month, felt that exact shift, ended up going back to claude max. not sure if it's routing or tuning but something changed and prompting around it didn't help.

Hi HN,

I was spending over 5 hours manually testing my Agentic AI application before every patch and release. While automating my API and backend tests was straightforward, testing the actual chat UI was a massive bottleneck. I had to sit there, type out prompts, wait for the AI to respond, read the output, and ask follow-up questions. As the app grew, releases started taking longer just because of manual QA.

To solve this, I built Mantis. It’s an automated UI testing tool designed specifically to evaluate LLM and Agentic AI applications right from the browser.

Here is how it works under the hood:

Define Cases: You define the use cases and specific test cases you want to evaluate for your LLM app.

Browser Automation: A Chrome agent takes control of your application's UI in a tab.

Execution: It simulates a real user by typing the test questions into the chat UI and clicking send.

Evaluation: It waits for the response, analyzes the LLM's output, and can even ask context-aware follow-up questions if the test case requires it.

Reporting: Once a sequence is complete, it moves to the next test case. Everything is logged and aggregated into a dashboard report.

The biggest win for me is that I can now just kick off a test run in a background Chrome tab and get back to writing code while Mantis handles the tedious chat testing.

I’d love to hear your thoughts. How are you all handling end-to-end UI testing for your chat apps and AI agents? Any feedback or questions on the approach are welcome!

What i liked

Generate a Blueprint Creates a basic structure and suggests necessary components.

Firebase Integration: Likely designed for seamless integration with Firebase services (though not deeply explored in the review)

Public Link Sharing: Enables sharing a public link to the prototyped application

I was drowning in scattered notes, reminders, and follow-ups—some in WhatsApp, some in emails, some just floating in my head. Traditional CRMs felt like overkill, and to-do lists lacked the context I needed. I wanted something as easy as jotting things down on a sticky note but smart enough to organize everything for me.

So, I built WorkToDoToday.

Imagine typing or saying:

"Remind me tomorrow to send a follow-up email to Customer XYZ regarding our PoC next month."

And instantly, the app:

* Creates a task with a due date of tomorrow * Tags it to Customer XYZ with a clear title and description * Lets me see all related tasks, notes, and past interactions in one place

Why It’s Different:

* No forms or complex workflows—just type naturally, and the app organizes everything * Manage tasks through chat or UI, whichever works best for you * See all interactions tied to a contact for a full view of your relationships

It started as a personal tool, but after sharing it, I realized a lot of founders and busy professionals needed this too. If you struggle with managing follow-ups like I did, check it out: https://www.worktodo.today Would love to hear your thoughts!

Hey HN,

About a month ago, I was lucky enough to participate in the Falcon LLM Hackathon, a crazy 5-day sprint to build something innovative with a cutting-edge language model. My team and I had one goal in mind: to make it easy for anyone to build conversational AI apps using their own data.

After countless late nights, a lot of debugging, and a few coffee-fueled “aha” moments, we built the first version of RAGGENIE. To our surprise, we won first prize! That win gave us the confidence to take RAGGENIE to the next level – an open-source low-code platform for building retrieval-augmented generation (RAG) applications.

Why RAGGENIE?

During the hackathon, we noticed how hard it was to integrate large language models with custom data sources, like databases or documents. Most existing tools required too much manual setup or technical expertise, making it hard for smaller teams to experiment with RAG tech. So we set out to simplify that process.

RAGGENIE enables teams to quickly spin up custom Copilets (our take on bots) that can chat with your data – without the heavy lifting. It’s perfect for small businesses or teams who want to get into GenAI without dealing with complex pipelines.

The Open-Source Decision

We’ve always believed in the power of open-source software. After the hackathon, we decided that RAGGENIE should be available to everyone, especially smaller teams who need flexibility and transparency. By making it open-source, we’re hoping to build a community of developers and businesses who can push this technology forward.

What You Can Do with RAGGENIE:

• Create low-code conversational AI apps that work with your own data (databases, documents, etc.) • Easily deploy retrieval-augmented generation tools without heavy technical setup • Customize it to fit your specific use case, whether it’s internal tools or customer-facing bots

The Road Ahead

We’re just getting started, but I’d love to hear from you! If you’re interested in the tech, have feedback, or want to contribute, check out our repo: github.com/sirocco-ventures/raggenie.

I’m also happy to chat about our experience at the hackathon, the challenges we faced, or how you can use RAGGENIE for your own projects. Feel free to ask anything!

what if it teaches some wrong content due to hallucination ?

When we decided to join the Falcon Hackathon conducted by the Lablab team, we had 7 days left to do it. Our first challenge was deciding what idea to submit. We split this task among our 6-member team, with each of us coming back with 3 ideas. These ranged from Fashion and Retail to Healthcare, Travel and Tourism, and more. We spent almost 6 hours on a call discussing each idea, weighing their pros and cons.

The criteria we discussed were:

- Is the problem statement a good fit for LLM? - Why hasn't this been solved before without LLM? - Is there a better solution than LLM for this? - What's the industry, and what's the potential for expansion?

After this consideration, we shortlisted some use cases, but none were unique—we found they'd either already been solved or weren't necessary. So, we decided to return to our comfort zone: building a dev tool.

We chose to build a low-code platform where users could create all the use cases we'd discussed without coding. We found a few alternatives, but none quite met our needs.

We started creating wireframes while some of us began building the backend. In one day, we completed the full wireframe. Then, while discussing how to present the demo, we chose one of our previously discussed use cases: an insurance chatbot.

To expedite the demo, we decided to use Airtable as the destination and built agents for Airtable and PDF to fulfill this use case. This took us a day. By the next day, we had also developed the front end.

On day 5, we began integrating in the morning and finished before evening, then started recording the demo. We weren't entirely satisfied with some aspects, but then we learned the deadline had been extended by a day.

We worked for another 24 hours to improve it. Finally, we received an email announcing we were finalists. That's when we decided to take this project further.

After the final demo, we won first prize, which was incredibly exciting. We're now using the prize money to develop this project further and plan to open-source it once it's complete.