Yes, the checklist would not be all-encompassing or foolproof, and there would likely be revisions to the checklist, and maybe even domain-specific variants, but it would be an extra level caution that the media could report or choose to ignore at their will. Over time, the apparent level of scientific rigour would improve. No, it’s not bulletproof and there will be people who will try to meet the checklist and still present erroneous conclusions as reliable science, but it would be an improvement in the status quo for a layperson who is aware and values said checklist.
On the whole though, most of the things you should and should not do are so domain specific its very hard to give much useful advice at the level of "all science." Right now this seems to work because researchers are so eager to anticipate objections and and avoid unnecessary arguments during peer review they stick slavishly stick to the same methods used by seminal papers in their field, and this has the same effect as running down a checklist.
There probably is a case to be made for using an actual checklist, though[5].
[1]: http://www.consort-statement.org/
[2]: https://en.wikipedia.org/wiki/Design_of_experiments#Fisher's...
[3]: https://en.wikipedia.org/wiki/The_Lady_Tasting_Tea
[4]: https://www.goodreads.com/book/show/27311742-the-seven-pilla...
"if you can't repeat and predict, stop calling it science" seems like a nice bright line.
Peer-review obviously isn't enough, I'd like to see peer-replicated studies become a thing.
Some of these cases can be rescued by considering "retrodiction"[1][2] as valid substitute for prediction in the right circumstances, but not all.
I personally think the analysis of the Mott problem[3] points the way to the solution to some of these kinds of issues. That is, a prediction can take the form of a likelihood function which assigns high probabilities to certain combinations of events and low probabilities to others. Theories with low perplexity[4] can be considered correct even if they can't make predictions, and the study of such theories can be scientific. But as far as I know I am the only one who thinks so.
[1]: https://en.wikipedia.org/wiki/Retrodiction
[2]: https://afdave.wordpress.com/2007/09/04/sir-karl-popper-and-...
"if you can't repeat and predict, stop calling it science" seems like a nice bright line.
For what it's worth, there is science meeting the "repeat and predict" definition that can be done in each of the fields you list.
(Also, most of the fields you listed do allow for replication.)
I don't see a "No True Scotsman fallacy" here because I think I define it quite clearly: Can you provide accurate predictions? I'll concede that there exists a bit of grey area in that question, but the answer is heavily bi-modal.
So we're left with a "No True Scotsman fallacy" where have to say that some science is "good" and some is "bad" and the only way to tell is to ask someone knowledgeable to evaluate each paper on a case by case basis. Not terrible useful to the layman.
And why do we want any kind of science to automatically get respect anyway? Good science is good because its already been subjected to an incredible degree of scrutiny. It will hold up to a little more. The real problem is disingenuous, bad faith arguments which are allowed to dominate the conversation. The real problem is to teach the general public to distinguish between sincere, good faith arguments and patent bullshit. This is much more difficult than it sounds because bullshit can easily conform to any merely superficial characteristics.
[1]: https://en.wikipedia.org/wiki/Demarcation_problem
[2]: https://en.wikipedia.org/wiki/Bright-line_rule
[3]: https://en.wikipedia.org/wiki/Falsifiability