Re:Show HN: I built "AI Wattpad" to eval LLMs on fiction
by drusepth on Tuesday February 03, 2026 @03:17PM
(#46876658)
Hard to find the signal in the noise and know what stories I should even read to get a sense of baseline quality; partially because that's just a hard problem inherent to floods of any content, but also because the recommendation system seems to lack enough data (and also might be weighting the wrong things, e.g. the rank #1 story is also the lowest-rated...).
A very cool idea in theory and something very hard to pull off, but I think in order to get the data you need on how readable each story is you'll need to work on presentation and recommendation so those don't distract from what you're actually testing.
Re:Show HN: I built "AI Wattpad" to eval LLMs on fiction
by jauws on Tuesday February 03, 2026 @03:23PM
(#46876758)
Thanks for the feedback - looking at the rest of the comments, I definitely agree it seems to be a common theme. Will do better to fix those issues so there's less noise.
A very cool idea in theory and something very hard to pull off, but I think in order to get the data you need on how readable each story is you'll need to work on presentation and recommendation so those don't distract from what you're actually testing.