The projectively extended real line doesn't support adding infinity to itself, which is what the grandparent to your comment was talking about. The problem is that the projectively extended real line does not have both a positive and negative infinity.
[1] https://en.wikipedia.org/wiki/Hyperreal_number [2] https://en.wikipedia.org/wiki/Dual_number
But, anyway, if you can claim infinity is a number because someone thought of the real projective line, I can say polynomials are numbers. Square matrices, too --- I think of square matrices as being big numbers; rank measures how invertible a big number is.
A stranger consequence of my definition is that a continuous real-valued function on a space X is a "number." If X is a single point, then such a function is the same as a real number. If X is a discrete set of n points, then the set of functions is R^n. If X is compact (I think that's sufficient?) then the maximal ideals in the set of continuous functions is in correspondence with X itself. This suggests that for any ring, one may imagine there to be a space that it is the functions of, whose points are the maximal ideals of that ring. For the ring of complex one-variable polynomials, the points correspond to C itself (the maximal ideals are generated by (x-c) for varying constants c). So, yeah, polynomials are numbers now.
> The extended complex numbers consist of the complex numbers C together with ∞.
It may be an "extended number", but it is not a number.
In general, the boundary of the category of ideas (if you like, elements of some formal model) that can be called “numbers” is a very fuzzy and arbitrary one.
Some people might reject as “numbers” anything other than the counting numbers 1, 2, 3, 4. Others might allow “negative numbers” or ratios. Still others are happy to include quaternions or infinite strings of digits output by some computer program. Meh.