Re:Show HN: A platform that analyzes research papers for com...
by sergeim19 on Thursday May 01, 2025 @03:02PM
(#43861977)
Oh, that's cool! I will have to read your article on scoring more carefully a bit later but I think that in short, I decided to take a simpler and more heuristic approach for now based on TRL (a well established metric) and a combination of some other factors. I also didn't want the user to consider the score too much as a kind of "divine word". In general I found that the heuristic used is OK for the given score plus/minus maybe 1 point.
I wrote a bit more about it here https://eamag.me/2025/Paper-To-Project
I think LLMs are usually give very simple ideas if you don't do "deep research" and just give one paper, I assume that's what you're doing? At least I saw some of my projects were not good, and I need to run more steps on deduplication/improvement
I think the important part is the ranking, but ranking can be subjective (each company/person has different values). I wrote about ranking too https://eamag.me/2025/Ranking-With-LLMs and some ranking at https://openreview-copilot.eamag.me/ranking
Shoot me a message if you want to chat more :)