But how do I do that? I can interpret a paper and apply it to my problems, but I'm not qualified to tell if it's a good paper[1]. In my field -- computing -- research papers are often written in a language that only just makes sense to me. I'm sure it'd be easier if I was a computer scientist that regularly published, but that's why I suggest you'd need a team of actual scientists who are used to reading, writing, and reviewing papers.

As Kernighan almost wrote:

Reviewing the paper is twice as hard as reading the paper in the first place. Therefore, if you have to be as clever as possible to read the paper, you are, by definition, not smart enough to review it.

[1] Although I can tell whether the code works, the quality of a paper is also about how well-written it is, whether the arguments are valid and supported by the data, and whether it provides something future research can be built on. A terrible paper that happens to contain code that works is useful for me, not so much for science in general.

I believe tomp's suggestion is that you can bootstrap as an overlay journal (see https://en.wikipedia.org/wiki/Overlay_journal ) to build up that expertise. You don't need to have experienced people from the start, because they can learn while doing. At the start, no one is going to pay attention to the journal. Then as it gets known for being a good curator, people will start requesting to be reviewed for inclusion. Poof you've got a journal.

You might even start with only yourself. And in 5 years if you want to distance yourself from your earlier, naïve efforts, then simply announce a name change.

Here's the ugly truth - it's hard to tell if most papers are "good", and there are many types of "good".

Distancing yourself from early failures is never a good strategy. Too often we flee from our past failures. For me, I don't shy from them but admit they occurred, and that I have learned from them since.

Perhaps I used the term incorrectly. I don't mean "distance" to require someone to "flee" nor not admit its connection, nor do I regard naïvety as a failure.

In my field, the main journal used to be "Journal of Computer Documentation". It spun off from "American Documentation" in the early 1960s. The JCD title changed in the 1970s to "Journal of Chemical Information and Computer Sciences" because "documentation" - a 1920s term - was considered already old-fashioned by the 1960s, and the title didn't capture the new focus on computers, and the underlying principles of information science.

It then became "Journal of Chemical Information and Modeling" in 2005 because the 1990s showed how to apply those information techniques to make predictive models.

Each name change is a form of distancing itself from its previous focus as part of a shift to a new focus.

Or is "distance oneself" not applicable for that situation?

I think I probably got my wires crossed. By "distancing" I did indeed think you meant "pretend as best as possible previous work never happened in an attempt to protect your reputation", but that's not at all what you meant! Sorry about that.

Do these actually exist? I would love a "This Week in Vision Research" that has 1-2 paragraph summaries of recent papers, maybe with some attempts at critiques and placing the results in context.

Faculty of 1000 is the only thing I know of that's even close to this.

"Discrete Analysis" - http://www.nature.com/news/leading-mathematician-launches-ar... just started and was in the news

http://processalgebra.blogspot.com/2015/12/the-novelty-of-ar... lists a few more, including "Logical Methods in Computer Science published its first issue ten years ago and has become one of the favourite publication outlets for researchers working on logic in computer science, broadly construed."

Right, you probably can't do it yourself. Even established journals often have a group of peer reviewers etc. Probably the best idea is for a group of academics to get together and start "sharing" the papers they found useful, acting as a "peer review" group of sorts.

I was just commenting about how to get around the "the journal isn't prestigious so no important research is published in it so it's not prestigious" problem.

Your journal wouldn't ever be considered prestigious or even moderately average if you're re-publishing old articles instead of publishing actual novel research.

You could create a newsletter that's useful to some, but if anyone would consider to publish their new research findings in your journal, then that would be bad for them since it wouldn't be considered anywhere equal to a "proper" publication (since you openly claim to accept non-novel findings as well) and would also prevent them from ever publishing it in any "better" journal since that would not be unpublished research any more.

In practice new good journals are generally started by (sub)communities who know each other and feel that [a topic of] their discipline is not adequately covered by the existing journals. This means that since day 1 they have (a) a source of good new papers that they will want to send there and not elsewhere - the whole reason for them founding the journal, (b) a pool of respected and knowledgeable editors/reviewers (the same community), and (c) initial interest of the community who would all read (and cite) the journal. Given that, the only missing part is recognition by various indexes and funding agencies, which will arrive within a few years (3-5?) if the new journal is well run, productive and gets appropriate citations outside of it.

It could select papers from preprints. That's what several overlay journals do.

But I'm commenting now more to point out that some journals do accept non-novel findings, like 'Journal of Negative Results in BioMedicine'.