Re:MDMA: An open source federated LLM inference engine
by sylware on Wednesday October 14, 2026 @07:27AM
(#50085961)
This is where you would be surprised. You can prototype binary specifications with a classic computer language (like plain and simple C, I don't know how much micosoft rust support binary layout definitions and language independent core ABIs), then you can "plug" implementations in any language, the benchmark being assembly (have a look at dav1d, and probably dav2d). The only thing to keep on eye on: the runtime dependencies, for instance a generic memory allocator (usually not a good idea), and for assembly the abuse of a macro pre-processor (ffmpeg was bitten by it).
Re:MDMA: An open source federated LLM inference engine
by seuros on Thursday October 15, 2026 @08:12AM
(#50102535)
Author here.
Handrolled Assembly will be better, but i preferred code i can read and understand.
MDMA is faster than llama.cpp in some models, because i did read the c code and rewrote it better. Rust is not the cause, i could the C code and open a PR and then hope it get merged.
The point of MDMA is you can have multiple nodes providing models while your main mode see all of them.
Re:MDMA: An open source federated LLM inference engine
by sylware on Friday October 16, 2026 @06:43AM
(#50116747)
Handrolled assembly is always better (on the long run).
You can read this assembly code the same than C until your data structures are properly exposed and documented.
There are some tricks though: avoid abuse of macro pre-processor (ffmpeg was bitten by just that), reasonable name mangling/hierarchical namespaces (usually done with a pre-processor that to avoid too long identifiers on deep projects). Ofc those pre-processors should be as small and lean as possible runtime and SDK included (don't python and don't hesitate to include their code in the project, yep, better if handrolled assembly too).
Does the C code generated by the rust->plain and simple C transpiler work?
Handrolled Assembly will be better, but i preferred code i can read and understand.
MDMA is faster than llama.cpp in some models, because i did read the c code and rewrote it better. Rust is not the cause, i could the C code and open a PR and then hope it get merged.
The point of MDMA is you can have multiple nodes providing models while your main mode see all of them.
You can read this assembly code the same than C until your data structures are properly exposed and documented.
There are some tricks though: avoid abuse of macro pre-processor (ffmpeg was bitten by just that), reasonable name mangling/hierarchical namespaces (usually done with a pre-processor that to avoid too long identifiers on deep projects). Ofc those pre-processors should be as small and lean as possible runtime and SDK included (don't python and don't hesitate to include their code in the project, yep, better if handrolled assembly too).