the1024

The search space being large feels like a fundamentally hard and costly problem to solve for. Where do you recommend folks start given such a high number of possible scenarios?

This is definitely something the companies most focused on agent experience are thinking about:

- https://supabase.com/evals - https://nextjs.org/evals

What is a better analogy for how you think developer tools should think about agent experience: an RL environment (hillclimbing a task like you mention), or a test suite (to be used in CI/CD, etc) as referenced by Supabase/Vercel?

How do you think about enabling fully agentic onboarding without proper human identification? It feels like any rate limiting would be pretty easy to get around.

> How does it work? We don't know! We built this to learn a little bit more. We've seen that LLMs tend to prefer user-generated content (sites such as wikipedia, reddit, etc.) and strangely even youtube.

> How do marketers rank higher? Will LLMs prioritize other LLM content? At least so far, LLMs and search engines tend to downrank LLM created content. I could see this becoming indistinguishable in the future and/or LLMs surpassing humans in terms of effectively generating what reads as "original content"

> Who will pull the strings? At this point, it seems like whoever owns the models. Maybe we'll see ads in AI search soon.

https://www.tryprofound.com/_next/static/media/honeymoon-des...

We absolutely do lose information here; that's a great point. The goal for us wasn't necessarily to surface the best ranking; it was to learn how LLMs produce a given ranking and what sources it pulls in.

The nugget of real interest here (personally speaking) is in those citations: what is the new meta for products getting ranked/referred by LLMs?

https://x.com/rauchg/status/1910093634445422639

Interesting call out - we're working on adding Gemini soon!

That nuance is really important/hard to piece apart. Have you found any good techniques to solve for it?

OP here - looking at what the models pick up as sources for "Trustworthy News Sources" is especially interesting. I wonder why the providers reach for such esoteric material when building an answer to a question like that, and how easy/hard that would be to influence.

That's a great point - we built this moreso to learn a bit about how the AI models interpret ranking products, and less so to actually be a trusted source of recommendations. Seeing the citations come through has been really fascinating.

The use case for that is to better understand where the gaps are when looking to capture this new source of inbound, given people are using AI to replace search.

There's definitely a whole bunch of features missing that we'd need to make this a genuinely useful product recommendation engine! Price constraints, better de-duping, linking out to sources to show availability, etc.

"kay-len" - confusing I know

Thanks! You can think of Grimp as a lower-level tool for interacting with the import graph in Python, while Tach is a high-level tool responsible for 'modularity' as a whole (e.g. modules, interfaces, layers, deprecations etc.)

Tach is also more opinionated - so it doesn't require you to write any custom code, and uses declarative config to enforce your desired architecture.

Thanks for this feedback! Here's a more structured walkthrough: https://www.loom.com/share/7c03d72ea2b54212a3509d4333f61b99?...

We'll add this to the README.

Yes, good call out - I actually wrote a bit more about this and how we arrived at our solution with `tach`.

[0] https://www.gauge.sh/blog/the-trouble-with-all

(author here)

borplk hit this right on the head - the public/private distinction not sufficiently nuanced enough to handle the requirements of more complex codebases.

To put your example in other words, I simply want baz scoped to foo. baz may be public for all members and sub-modules of foo, but external uses of baz outside of foo would create brittle and unwanted dependencies across my codebase.

My experience primarily comes from scale-ups, where the companies grew fast and hired fast.

Without putting specific companies on blast, I hear about this problem consistently from growth stage startups that scaled on Python. It's a common reason people reach for microservices - trying to use network boundaries to cover for the lack of module boundaries.

Ideally you can trust coworkers; unfortunately that's not always the case. Here's a quote from a developer in response to the article in another forum: """ It can even be very useful sometimes to be able to import private stuff. In a large ecosystem where you use many modules from other teams, you might need to use some private functionality. """ If something is available for an individual developer that solves their problem, often they're not thinking about the fact that it's private or creates a brittle dependency with no contract.