Copilot can solve a few, mainly just simple matrix multiplication and radix sorting that appear dozens of times in its training set.
Can it understand the Harwell-Boeing sparse matrix format?
https://bugfix-66.com/cb75bc9d9f331733904d9c4a2a7782f98fb058...
BUGFIX-66 is a fair test. Give it something quite straightforward that mostly wasn't in its training set (except some of the puzzles near the beginning) and make sure a bullshit answer is marked wrong.
So, how does it do?
Everybody in ML (e.g., me) knows that.
But that's what we're seeing.
I'm saying: "Here's a test outside the training set."
We're practically neighbors, and I am looking forward to the day we meet face-to-face.
It will be an heated discussion and I expect a sincere apology from you for your outrageous behavior.