They also involve addition, but that doesn't make them arithmetic. Knowing linear algebra gets you perhaps 10% of the way to having the math background for a first course in GR.