Agreed, it is a little confusing. What makes it even more confusing is bias can be introduced as a regularization technique (to address overfitting) and these are not the same thing.

Bias in this context can be simply interpreted as analogous to the intercept in y = mx + b. Kind of like the baseline when no additional information is given.