Yeah...and it didn't cite me :)

Yeah, that's a data point as well. I found a model that was good with citations by asking it to recall what I published articles on.

That's what I did. It came up with smart-sounding but infeasible recommendations because it took all sources it found online at face value without considering who authored them for what reason. And it lacked a massive amount of background knowledge to evaluate the claims made in the sources. It took outlandish, utopian demands by some activists in my field and sold them to me as things that might plausibly be implemented in the near future.

Real research needs several more levels of depth of contextual knowledge than the model is currently doing for any prompt. There is so much background information that people working in my field know. The model would have to first spend a ton of time taking in everything there is to know about the field and several related fields and then correlate the sources it found for the specific prompt with all of that.

At the current stage, this is not deep research but research that is remarkably shallow.

> It took outlandish, utopian demands by some activists in my field and sold them to me as things that might plausibly be implemented in the near future.

Reminds me of when Altman went to TSMC and bloviated about chip fabs to subject matter experts: https://www.tomshardware.com/tech-industry/tsmc-execs-allege...