> Why do we use such large networks then? For a given trained neural net, there might be a much smaller one that does the same thing, but in practice, it's difficult to get the same performance by training a smaller network.

Also, because it's cheap to run a network with a billion parameters. Lots of real-world applications are constrained not by compute, but by size of the training set.