tension000

Yet another TEDIOUS BATTLE: Python vs. C++/C stack.

This project gained popularity due to the HIGH DEMAND for running large models with 1B+ parameters, like `llama`. Python dominates the interface and training ecosystem, but prior to llama.cpp, non-ML professionals showed little interest in a fast C++ interface library. While existing solutions like tensorflow-serving [1] in C++ were sufficiently fast with GPU support, llama.cpp took the initiative to optimize for CPU and trim unnecessary code, essentially code-golfing and sacrificing some algorithm correctness for improved performance, which isn't favored by "ML research".

NOTE: In my opinion, a true pioneer was DarkNet, which implemented the YOLO model series and significantly outperformed others [2]. Same trick basically like llama.cpp

[1] https://github.com/tensorflow/serving [2] https://github.com/pjreddie/darknet

Check out this leaderboard for training CIFAR10 to specific accuracy on various hardware: https://dawn.cs.stanford.edu/benchmark/CIFAR10/train.html

Keep in mind that the last entry was from 2020 *(94.39% in 10 seconds)*, so it may not be a fair comparison to newer hardware like the A100.

it seems someone skipped the tutorial on "building a solid team"; elaborate.

about accountant: find some dropouts from big finance companies, they should be cheap, and with solid knowledge.