As an example, the natural numbers are infinite because they contain the even numbers and the even numbers are the same "size" (or cardinality) as the natural numbers themselves.
As a second example, the real numbers are infinite because they contain the interval (-1, 1), which has the same "size" as all of the real numbers.
More exactly, I say that translating the concept/idea of infinity from mathematics to physics is not idempotent (not sure if this it's the correct term, but you get the idea), saying that infinity "exists" in the physical world the same as it does in mathematics is kind of tricky when you think of the consequences through and through. To take your second example, applied to Physics that would mean that no matter what two small particles we choose as being "fundamental" there will always be an infinity of (presumably even more small) particles separating the two, and so on and so forth (it's "turtles all the way down", so to speak).
Like I said, I'm not a specialist, just a programmer, but I think we should be aware of the presumptions we implicitly make when discussing about different topics but using the same language (in this case the of concept of "infinity" in maths vs physics). Hume explained it better than me (maybe in an "An Essay Concerning Human Understanding" ?!) when he said that the mathematical idea of a straight line (I'm talking about the geometrical line) is just that, an human-constructed idea, there's no absolute straight line in the real world (not sure what he thought about the idea of infinity, probably not much, judging by his being critical against mathematical induction).
Not a specialist, but "infinity" is indeed a tricky thing to make sense of.
First, if you decide to use and define it only in a "theoretical" sense (constructed through the set of axioms you mentioned) things are fine, there are lots of things we have constructed in an abstract way in the past which have proved to be intellectually solid (think of the idea of God constructed by the scholastics).
The problem appears when you want to apply your idea constructed on a "set of axioms" back to the real life. The scholastics' idea of God had a very short life when that happened (more or less during the early Renaissance), and I wouldn't say that the mathematical idea of "infinity" would have a better life were people to view it in a more critical way.
Because were you to apply the idea of infinity to the real life (the life made up of stars, planets and Hacker News) that would mean that there are an infinity of planets and stars out there (some of us have started to be ok with that) but that would else mean that there are an infinity of Hacker News instances out there and there are an infinity of "me", the user paganel, making this comment on HN. This is hard for a lot of people to grasp, including me.