urcyanide
Rewrite the pre-commit with features requested from the community
Re:Rewrite the pre-commit with features requested from the community
on Tuesday September 09, 2025 @09:57AM (#45181993)
Rewriting it in Rust not only brings high performance, but also adopts more feature requests from the community.
Earth Observability with the Help of Vectors
Re:Lessons learned from profiling an algorithm in Rust
on Monday October 14, 2024 @01:15PM (#41839596)
Thanks for pointing out. I have updated the post. I use the opposite shift since I store the binary in u64 with a different endian from the C++ version. Sorry for the confusion, my bad.
Re:Lessons learned from profiling an algorithm in Rust
on Monday October 14, 2024 @01:04PM (#41839473)
Sorry for the confusing example. The bitwise one is correct since I store the binary in u64 with a different endian from the C++ version. (this happens because the C++ version is using a numpy script to do the preprocessing) My bad, I should explain it in a better way. Will update the post.
Re:Lessons learned from profiling an algorithm in Rust
on Monday October 14, 2024 @11:59AM (#41838771)
Yes, i&32 is more readable.
Re:Lessons learned from profiling an algorithm in Rust
on Monday October 14, 2024 @11:18AM (#41838328)
Not really, i \in (0..dim).step_by(32), shift \in {0, 32}, but for `i % 32` it's 0..32
Re:Lessons learned from profiling an algorithm in Rust
on Monday October 14, 2024 @10:02AM (#41837641)
The end2end QPS is about 3x of the 1st Rust version on both SIFT & GIST datasets.
Re:Lessons learned from profiling an algorithm in Rust
on Sunday October 13, 2024 @07:41PM (#41832729)
I also see the profiling shows that the "iter::next" takes a large percentage in the flamegraph. Are they the same reason?
Re:Lessons learned from profiling an algorithm in Rust
on Sunday October 13, 2024 @07:23PM (#41832642)
Thanks for pointing out, this can be optimized by the compiler when enabling opt-level=3
Re:Lessons learned from profiling an algorithm in Rust
on Sunday October 13, 2024 @07:22PM (#41832630)
It has the same QPS as the C++ version for GIST dataset. While Rust has more SIMD, C++ has const generic. I guess there is still some space for future improvement.
Lessons learned from profiling an algorithm in Rust
Re:Mpire: A Python package for easier and faster multiprocessing
on Friday August 11, 2023 @08:02PM (#37095505)
Some potential issues about Python multiprocessing https://blog.mapotofu.org/blogs/python-multiprocessing/.
COW is quite tricky. BTW, most of the related official Python docs doesn’t mention the usage under ‘spawn’.
Re:PostgresML is 8-40x faster than Python HTTP microservices
on Thursday October 20, 2022 @01:23AM (#33271115)
If you are trying to compare the performance as a ML service, maybe you should try to compare it with other ML model serving frameworks like https://github.com/mosecorg/mosec or https://github.com/bentoml/BentoML. Flask/FastAPI are not built for ML services.
Python Machine Learning Service Can Run Way More Faster
Bridging the gap between infra and machine learning: start from envd
Re:ContainerSSH: Launch containers on demand
on Saturday September 10, 2022 @10:06AM (#32790762)
Recently, I found another tool built for machine learning environment: https://github.com/tensorchord/envd. It integrates with buildkit and can share the config like Python functions cross the team. This can reduce a lot complexity for data scientists.