For example, a puzzle: why do SAT scores underpredict female performance in Calc 1, and overpredict grades in higher level courses? I suggested girls are more conscientious than boys, and that a study being proposed did not control for such factors. I even suggested a test of this hypothesis: compare grades in classes which are conscientiousness-weighted (25% homework, 10% attendance, tests questions are practice problems with numbers changed) to ability-weighted coursework (50% midterm, 50% final, test questions all new).
I'm not sure my idea was a good one. But rather than explain why either a) this had been tried, and didn't work, or b) why my test wouldn't work, I was simply told to leave the field to experts.
I've come up with plenty of dumb ideas in other areas, e.g., global existence of wave equations. The closest I've ever gotten to "stay out, you aren't an expert" was "Terry tried that and failed, ask him why before spending lots of time on it."
[edit: really confused why you are being downmodded.]
If you want to see real and useful research look into how long the optimal study period. There are significant and useful study’s which suggest 2 hours of nonstop instruction is less useful than two one hour periods with a moderate break in-between. Yet how many collage lectures follow this approach.
The professor of the last class I took was very conscientious about taking a break at the 1 hour mark of an 80 minute lecture.
The methodology is what makes research good or bad.
In terms of predicting how likely someone is to finish their freshman year of collage having a test where you can increase your score significantly with moderate levels of preparation is not a bad thing. However, the fact you can easily game the test means it is a less accurate indicator of a student’s innate capability. The fact you can retake the test creates yet another sort of bias. etc.
More generally, it's easy to focus on defects in the test, which are irrelevant in a larger contest and subject to change.
My wife and I were discussing this recently. She is in an unusual program where she should graduate with both a masters in history and an education certificate with hopes to teach in history at the High School level after that.
This is both second hand an anecodatal, but from what she tells me the education courses she takes are less rigorous and generally easier than the history courses. But the reason for this is that the education course she is taking are vocational. They are trying to teach her how to get teaching done in a modern classroom, not the science behind it.
It is akin to learning to the difference between training to be a mechanic and a mechanical engineer. Both are very hard and important jobs worthy of respect, but they have very different focuses. The education courses are focused on learning the technique of teaching effective, it looks like most of the research on the theory of effective education is done in departments like psychology, sociology, etc.
PS: Education might seem like a flaky field and much of the discussions are devoid of good science. At the same time there is a lot of great research that has been which has real and important implications.