A lot of people think they're subject-matter experts and may comment on your article, but may be mistaken.
In all the peer review processes I've been involved in reviewers review each others reviews before they go out. Other reviewers will challenge a bad or misleading review. If there aren't people who know enough about the subject they'll go out and find someone who is.
You could literally do this process in a web interface that looked like Hacker News, but they already pretty much do.
I think if you built your dream review system... I think you may find it looks pretty much like what we have already.
> callouts on Twitter and Hacker News
Sounds like a popularity contest.
It’s kind of like arXiv with voting and comments, but also supports collaboration for research in progress.
> You could literally do this process in a web interface that looked like Hacker News, but they already pretty much do.
I would love to hear your thoughts about how the current systems might be improved and if we’re on the right track.
Software should be citable. You can make it so by publishing a paper describing the software. Usually the authors then request anybody who uses the software in their research to cite that paper.
(No reason for doing it, I was just curious if I could.)
What researchers want is discoverability and credit. Some researchers are already famous and own their own distribution via arxiv or Twitter but for new researchers publishing a paper at a prestigious conference is a way to bootstrap a reputation.
If you really wanna get rid of large publishers what you really need to solve is discoverability and you can solve it with a combination of virtual mini conferences and callouts on Twitter and Hacker News.
I've gotten some of the best constructive feedback on my blog posts from Hacker News directly, I don't see how that's different from peer review. I don't find it very likely that someone would even be willing to give me feedback unless they themselves are already subject matter experts.
But really the impact of a piece of computer science research is directly correlated to how many people directly use the OSS inspired by the research. People in industry know this, what's left is is for tenure committees to also take software impact into account.