+1! Infinite density => way blacker than a black hole, no?

I have no idea about cosmology, but I think maybe that if this dense matter where everywhere, then forces would cancel out and that's why you don't get a black hole.

Not a physicist at all either; but on a per particle basis, I know gravity is by far the weakest of the fundamental forces. However, unlike the others, it doesn't have spin/charge/polarity that results in their vectors repelling each other. As such, black holes form when the large scale weak gravitational forces combine into a greater force than the strong short scale repelling forces exerted by the others (and especially so when nuclear fusion is involved). In a hot primordial plasma, the particles are by definition extremely energetic; pushing each other in all directions with far more vigor than the extent to which gravity attracts them. Hence being a big bang rather than a big crunch, I suppose.

Because the primordial universe was in a state of extremely fast expansion, while black holes today exist in space-time that is almost exactly flat. In fact, if the universe really started from a singularity, the expansion was infinitely fast for an instant.

To hedge the next question then: Why was the universe expanding so quickly?

It seems that the expansion must have been so fast that it was going faster than light speed, right? Otherwise the very dense universe must have gone black-hole.

The expansion of space itself is not comparable to the speed of light, but yes, I would say it's probable that it would have caused particles (or whatever there was at that time) to spread out faster than light, thus disconnecting them causally and preventing gravitational interaction. As for the why, answering that would involve answering why the Big Bang happened, which given our current understanding would veer into metaphysics. As far as we can tell, it's just what the universe does.