f'(x) is just A , not Ax,
f'(x) != f(x)
It's unfortunate that the connection between matrices and linear transformations, and thereby between derivatives and linear transformations, is not adequately emphasized in the math courses that most people take.
The derivative (or differential, or total derivative) of a function f : R^m -> R^n at a point x ∈ R^m is defined, formally, to be a linear transformation A : R^n -> R^m such that
|| f(x+h) - (f(x) + A(h)) ||
lim ------------------------ = 0
h->0 ||h||
> h is a vector in R^m
> f(x+h) is the exact value of f at x+h
> f(x)+A(h) is a linear approximation to f(x+h), using x as the base point and A as the linear approximator
So this just a formal way of saying that the dervative A should be a good linear approximation to f.
Every linear transformation has a corresponding matrix, and every matrix corresponds to a linear transformation (with respect to a choice of basis). So, when we say "the derivative of f at x is the matrix A", we really mean that "the derivative of f at x is the linear transformation represented by the matrix A".
EDIT: In response to the parent poster, who has dug their heels in by posting a now-dead rant in reply to this post, I encourage you to refer to a copy of Rudin, "Principles of Mathematical Analysis", for a rigorous treatment of derivatives. See also: representation theory and group actions.
(0a) An (nxm) matrix A represents a linear transformation f(x)=Ax from R^m -> R^n
(0b) A linear transformation f(x) = Ax is its own derivative, f'(x) = Ax
(3) The derivative of a function f : R^m -> R^n is a linear transformation with the same "type" R^m -> R^n as the original function. This means it is an (nxm) matrix.
The other two rules can be derived from these rules:
(1) The derivative of a function f : R^m -> R is a (1xm) matrix, aka a row vector.
(2) The derivative of a function f : R^n -> R is an (nx1) matrix, aka a column vector.
This perspective is helpful because, aided by the chain rule, we can easily derive all the other matrix algebra rules on the fly. Distinguishing row / column vectors helps catch conceptual type errors in places where it's inappropriate to add vectors and covectors together.