Yeah that is a good point. One can argue that technology is a tool and it can be used for evil. But with this stuff:

"without IBM's machinery, continuing UPKEEP and SERVICE, as well as the supply of punch cards, whether located ON-SITE or off-site, Hitler's camps could have never managed the numbers they did" [emphasis mine]

The idea is basically:

So there were not pocket calculators that they bought and then deployed in concentration camps and IBM can claim complete ignorance over the use. These machines need constant & professional service. Having them onsite meant that IBM sent technicians there probably. One had to wonder what those technician's response do the 24/7 billowing crematorium and emaciated bodies were.

However, on the other side (and I have not read the book, just looked at the topic many years ago superficially) I am not aware of any IBM engineers' stories about an actual site visit... So I don't know what to think and how guilty IBM really is...

I don't know, I think that IBM per se was really in a position to know how the census machines they had given to Germany were actually being used. Remember that this was before the days of satellite communication, and as part of their war strategy the British cut Germany's telegraph cables (just like they had in WWI) and intercepted packets of mail between the US and Germany whenever they could, and generally did a pretty good job of it[1]. I'm happy to believe that people who were nominally IBM employees were servicing the machines, but if IBM headquarters wasn't being told what was happening, and if the employees faced getting shot if they didn't do what they were told, I'm not sure how much you can say that IBM knew in any meaningful sense.

[1] This was a point of contention between the UK and US before the fall of France.

I doubt IBM would have given away any engineering stories about this unless the Germans won.