pbzcnepu

maybe if IEEE wanted to keep students, they could support them, unlike during the 2019 ISCA peer review fraud incident where an IEEE fellow was caught in peer review fraud that lead to a student's death, and essentially it took years of intense pressure for them to stop allowing a cover up, do a minimal investigation, and in the end most of the people involved got a secret punishment that was likely a minimal slap-on-the-wrist at best

https://pbzcnepu.net/ieee/index.html

you should notice that the professor involved is listed as being on the Program Committee for IEEE S&P 2022 already, meaning he must be well-connected with various higher ups at the conference. This is probably why they are reluctant to remove the paper. There were similar issues with the ISCA'19 peer review fraud case. IEEE/ACM are having some major issues these days.

if they release a statement as fast as they did for the ISCA'19 peer review fraud case you might be waiting a while (took more or less 2 years)

for more details than you possibly want go here: https://pbzcnepu.net/isca/timeline.html

though it is more about the ACM/IEEE side of things rather than what was going on in Florida.

Many researchers seemed to know there was weird stuff going on with peer-review at ISCA, but if you asked the researchers they'd just shrug and more or less say something to the effect of "don't hate the player, hate the game"

also, at ISCA'19, over half the papers were PC papers, meaning they were papers where one of the authors was on the committee that picked which papers get in. It's probably why the highers ups have been so reluctant to change the culture at the conference because it would so negatively effect all the higher-ups' publication counts

ISCA is a particularly horrible "top" conference. We've been trying to reproduce some papers with questionable methodology from ISCA'20 (including using "1 cycle access L1 cache" in simulations, something that's not really possible on modern CPUs).

Of 5 paper groups contacted, 4 have not responded despite repeated requests, even when VPs of research at the university contacted.

One group eventually replied, said that they were "too busy" to release their simulator changes, and then also saying they weren't going to because they were worried about it being classified as a munitions export.

the IEEE probably won't do anything. We've been trying for 2 years to even get an e-mail reply from them on the issue, let alone say anything useful. See the whole story here: https://pbzcnepu.net/isca/timeline.html

For a full rundown of the history of this incident see: https://pbzcnepu.net/isca/timeline.html

it's anecdotal.

another anecdote, when first trying to raise awareness of the issue, when mentioning it to members of the community I never got the response "wow, how could this happen in computer architecture?" rather the response tended to be "wow, I can't believe it finally got bad enough that a student died"

> If you don't want cycle-accurate simulator based papers, you'd have to eliminate a large variety of body of work only possible with this approach.

yes. Most "cycle-accurate" results are garbage. Do you show error bars on your results? Can you? Did you run on a variety of independently implemented simulators and show the results on all of them? Did you run the full reference inputs to SPEC all the way through? Why not?

The answer seems to be that it would be hard. But guess what, science is hard. Try complaining to a biologist sometime about your architecture paper that took so long to write, where you gathered all the results 2 weeks before the paper deadline.

It's fine if you come up with a new idea and run some simple proof-of-concept runs to show it might have merit, but don't pretend the results from an academic simulator hacked together by a sleep-deprived grad student have any real world merit.

it isn't always that the work is bad, it's just the review process for getting in has always seemed to be easier for people with the right connections. Not necessarily in an overt conspiratorial way like the recent allegations either.

The reason people have latched on to the current set of allegations is there seems to be actual concrete proof of misconduct that can be acted on.

well let's take Torrellas. Check out the two papers at ISCA'20 with him as an author.

Both use modified "cycle-accurate" simulators for the results. For now let's ignore all the issues with the accuracy of these simulators.

Let's validate how "solid" the work is. Drop an e-mail to Torrellas and ask for the code they used so you can see if you can reproduce their work. Hopefully things have changed and they'll send you the code but in my experience they'll just say no.

So they got two papers in which are unverifiable, and none of the reviewers ever saw the code involved. This bothers some people, but it's not unusual at ISCA.

there's really not much one can do. In Comp Arch, most of the honest people left the field years ago because they got sick of all this nonsense. Most were happy to let those left behind play their silly peer-review games, but when the student suicide happened it was like a call to action.

It is easier to fight things from the outside, as the internal higher-ups have less power to mess up your life.

It's interesting you mention UIUC, as traditionally that was the source of a lot of suspect papers. It was a running joke about the "coincidence" of how many people with UIUC connections managed to get papers into ISCA each year.

The program officers were contacted, leading to the OIG being notified on 6 December 2019 and supposedly an investigation was openened, but we have not heard anything back since then.

some of us made a big effort to get anyone at the IEEE or ACM to care about this ISCA'19 incident. The initial investigation by Torrellas was run mostly by ISCA insiders with conflicts of interest, and mostly seemed to be a stall tactic. They did eventually open a new investigation.

ISCA'20 should really have been cancelled until this was resolved.

See a timeline of our attempts to get anyone at ACM/IEEE to care: http://pbzcnepu.net/isca/timeline.html