mathis-l

Of course. That’s why I opened the issue so that the team can improve it.

Not sure how good the Harvey benchmark is. I recently looked into traces from an agent solving their firm knowledge task subset and found multiple problematic tasks and rubrics.

Here are 3 examples (there are more): https://github.com/harveyai/harvey-labs/issues/191

At least when working with local MCP servers I solved this problem by wrapping the mcp tools inside an in-memory cache/store. Each tool output gets stored under a unique id and the id is returned with the tool output. The agent can then invoke other tools by passing the id instead of generating all the input. Adding attribute access made this pretty powerful (e.g. pass content under tool_return_xyz.some.data to tool A as parameter b). This saves token costs and is a lot faster. Granted, it only works for passing values between tools but I could imagine an additional tool to pipe stuff into the storage layer would solve this.

Dspy handling it openly https://github.com/stanfordnlp/dspy/issues/9500

CrewAI (uses litellm) pinned it to 1.82.6 (last good version) 5 hours ago but the commit message does not say anything about a potential compromise. This seems weird. Is it a coincidence? Shouldn’t users be warned about a potential compromise?

https://github.com/crewAIInc/crewAI/commit/8d1edd5d65c462c3d...

I’ve worked on a code base that was about 15 years old and had gone through many team changes. It was a tricky domain with lots of complicated business logic. When making changes to the code, the commit history was often the only way to figure out if certain behavior was intended and why it was implemented this way. Documentation about how the product should behave often lacked the level of detail. I was certainly always thankful when a dev that was long gone from the team had written commit messages that communicated the intent behind a change.

I’m 190cm and tested luna rail’s prototypes. I was amazed how much space I had, even in the smallest cabin. Definitely much better compared to any night train experience I’ve ever had

You might want to take a look at https://github.com/segment-any-text/wtpsplit

It uses a similar approach but the focus is on sentence/paragraph segmentation generally and not specifically focused on RAG. It also has some benchmarks. Might be a good source of inspiration for where to take chonky next.

Haystack [1] is another good option. It‘s modular, doesn’t get in your way and is particularly strong at retrieval. People like the documentation too.

Disclaimer: I work at deepset

[1] https://github.com/deepset-ai/haystack