It is not really clear. In the graph on page 137 does the "dollar of compute" includes only compute required for inference to provide the answer? Or does this include training as well?

It would be logical to assume that they show expense of compute required to maintain inference. Leaving training compute (which is huge expense) out.