I have been very impressed at how performant the runtime's cpu inference is. It beat out hand written avx intrinsics by almost an order of magnitude. I had to go find a machine with no discrete gpu to convince myself it wasn't using one despite it saying it wasn't.