roampal

You're right, it is a form of tagging technically. The difference is you're already saying "thanks that worked" or "nah that's wrong" anyway. No extra step, it just listens.

Ran a 4-way comparison test across 200 query-memory pairs:

- Baseline RAG (embedding similarity only): 10%

- RAG + reranker: 20%

- Outcomes only (no reranker): 60%

- RAG + outcome scoring (mature memories with 20+ uses): 60%

"Accuracy" = correct memory ranked #1 for the query. The outcome scoring uses Wilson score lower bound - memories that consistently get positive feedback from the "user" get boosted, ones that fail get demoted.

Test methodology: https://github.com/roampal-ai/roampal/blob/main/dev/benchmar...

Fair point, the install instructions at the end were meant as a "here's how to try it if interested" but I can see how it reads as pushy. The core of the post is about the outcome scoring approach itself. Should've led with more depth on the methodology. Thanks for the feedback.

Absolutely, go for it!

Run whatever benchmarks you have. I would love to see how it stacks up against RL methods.

Ping me if you need help with anything.

Thanks!

Dude, you literally wrote the exact motivation paragraph for Roampal right around the same time I posted this

Thorndike's Law of Effect is the entire reason I built the outcome-scoring (+0.2 for worked, −0.3 for failed) and shift weighting toward proven memories. You're not half-baked — you're 100% right. I just happened to ship the PoC first.

Would love to hear your take on the cold-start problem and whether those reward magnitudes feel right in practice. Shooting you a connection request on LinkedIn if you want to swap notes.