Of course the actual ML is not going to be faster in Rust, because the main tensor operations as used from Python are already written close to the metal. The cost of these operations is 99% of what it takes to run ML. Optimizing the remaining 1% is just not going to have a noticeable effect.

I've spent a lot of time writing python ML code, and I think 99% (of time inside fast C/cuda code) is much higher than most programs achieve. Off hand I'd say 80% is average and 90% is good (usually comes after a performance refactor). Also, one aspect that people often overlook is that optimizing compilers can often combine or vectorize operations in ways that are an order of magnitude faster than the same operations performed consecutively in, say, pure numpy. in that scenario, even though each numpy operation is very fast for what it is, the python program that combines a bunch of them ends up running much slower.

gRPC would account for very little time relative to the k-means algorithm. I think what results in the large speedup is the "embarrassing" parallelism that an async Rust server (gRPC in this case) would give. Knowing the CPU time during the tests could confirm this. I'll try run the benchmarks on my machine this week to verify my assumption.

I've hit some degenerate cases on gRPC serialization. The golang grpc/proto library accounted for something like 40% of cpu time doing serialization on mapping types for an API I developed under load test. Changing that to a list of key/value pairs dropped that 40% to <1%.

The gRPC python sever implementation does a lot of work using python builtins, and doesn't use recent asyncio primitives. It uses built-in python thread-pool execution, which means you also have the GIL to worry about: https://grpc.io/docs/tutorials/basic/python/#starting-the-se...

Given that, I would expect its connection / request management to be pretty darn slow. gRPC has better support for Python as the client. The thread pool execution model also prevents you from doing a number of things you might want to do with gRPC servers (long-lived request streaming is out of the picture).

There are a couple third party packages out there trying to make this better (e.g. https://github.com/vmagamedov/grpclib) but I hit compatibility issues for a few cases when trying to use them for some prototypes.