ubedan

Wonderful... I'm so happy to see a Rust version of llama.cpp.

The true value of this will be proven over time with wide use and as PRs are merged.

Do you have a feel if you'll try to drive this to stay feature parity with llama.cpp, or are you willing to diverge with new features like NVME/SSD MoE weight streaming etc.

Github: https://github.com/simdjson/simdjson

It showcases an especially ingenious scheme for escaping json strings, as well as the other tokens necessary to parse json.

Daniel Lemire - One of the authors at QCon 2019: https://www.youtube.com/watch?v=wlvKAT7SZIQ

Another Youtube video that explains it: https://www.youtube.com/watch?v=vd9J9PPmAMM

Spectacular... Reminds me of the SimdJson algorithm in terms of jaw dropping nearly unbelievable speeds through creative programming. I hope this code get popular, as it will save tons of electricity, money, CO2, etc.

Have you considered publishing a rust crate as well? (If not, I volunteer.)

I believe so, with Rust being exhibit 1. Rust is known for finding rather a lot of bugs while compiling instead of during debugging. The quote you'll often see is: "If it compiles, it usually runs."

The Rust compiler enforces many code style conditions that you most probably want in your finished program... Type checking, only one thread writing to memory at a time to avoid race conditions, etc.

It is possible to imagine an AI version of Rust that takes this even further in the future.

I agree that both Andrew, and the author have numerous good points, and yet I see things differently.

- I too am a fan of Andrew, and Zig, although all of my future plans currently are tied to Rust.

- I think the crux is that we are talking about 2 Vibe-ish AI rewrites, one to Zig, and another to Rust. I believe the investment in both reasonably minimal beyond the cost of tokens.

- I think that more adherence to a Zig style guide would probably have been very beneficial to the final work product.

- I have high expectations that the Rust rewrite will yield a more stable work product out-of-the-box. I am looking forward to the post-analysis.

Wonderful tech, and video example. I think there may also be a special forces application, but I don't know enough about how well their current solution works.

In the office, a non-contact video solution (lip reading) is likely to be far more popular, but a lot depends on which is more accurate.

Kudos...

I have noticed the same issue with agents/llms, and you appear to have solved it the best way possible. Spectacular! Thank you for sharing it openly!

Kudos... Wonderful work that I think has real value in certain situations... Thanks for sharing!

Brilliant Job! I think you have landed on fundamental concepts that will be core to Agents and AGI going forward. Congrats on a very impressive project, video, website, etc. I think this is going to be big!

Please consider writing an academic paper since that could help you reach many more top AI researchers, etc.

"There’s a big difference between mostly dead and all dead. ... Now, mostly dead is slightly alive. Now, all dead…well, with all dead, there’s usually only one thing that you can do."

I personally believe that illumos will survive for decades, and it very well could rise again for those users that want/need robust stability.

One particularly notable design win was Oxide Computer Company as their hypervisor. They published their reasoning for choosing illumos here:

https://rfd.shared.oxide.computer/rfd/0026

https://www.youtube.com/watch?v=5P5Mk_IggE0&t=2556s

It’s fantastic to see Makepad reach its 1.0 release. Congratulations to Rik and the entire team for this spectacular achievement! The dedication and effort put into this project are truly commendable.

Makepad presents a compelling approach to cross-platform GUI and application development, and I believe it has the potential to become a leading open-source solution. Several key features stand out:

* *Rust-Native Design:* Built specifically for Rust, Makepad leverages the language’s strengths in performance, safety, and concurrent programming, resulting in robust and efficient applications.

* *Integrated Development System:* Makepad offers a unified environment encompassing a GUI framework, a development system, and an IDE, streamlining the development workflow.

* *Fast Compilation:* The system demonstrates impressively fast compilation times, accelerating iteration and development cycles.

* *Embedded Rust-like GPU Shader Language:* The inclusion of a dedicated, Rust-like shader language allows for powerful and flexible GPU programming within the framework. The design choice to facilitate variable referencing by name – both in the program and the shader code is a particularly nice touch.

* *Broad Platform Support:* Including Windows, Linux, macOS, Android, iOS, and the web via WebAssembly etc.

While Makepad 1.0 represents a major milestone, further development is crucial to fully realize its potential. Areas for future focus could include:

* *Accessibility Features:* Implementing robust accessibility support is essential for inclusivity and broad adoption. * *Agentic AI Integration:* Exploring integration with agentic AI frameworks would position Makepad at the forefront of emerging technologies. * *IDE Customization:* Expanding IDE customization options, particularly support for various keyboard mappings (e.g., Vim), would cater to a wider range of developer preferences. * *Real-World Applications & Community Growth:* Encouraging the development of more real-world applications and fostering a growing community will be key to long-term success.

I’m excited to see what the future holds for Makepad and look forward to contributing to its future success.

The 8.19.2024 Oxide.Computer podcast talks about this.

https://www.youtube.com/watch?v=DNHMYp8M40k

Of special interest is that they are maintaining a completely free pre-rugpull version of CockroachDB that was forked before Cockroach's retroactively relicensed security fixes.

I would look seriously at using that instead of starting down Cockroach's free with telemetry offering.

It's scary to have you say that when logitech recently looked a computer mouse that needs a subscription:

https://arstechnica.com/gadgets/2024/07/logitech-has-an-idea...

I would also look at Qubes. It basically reinstalls a new desktop vm every time you boot the computer, and allows, but has extra safe guards for using usb, cut-copy-paste, templates, support for tor etc. But it's slow for obvious reasons.

https://www.qubes-os.org/

Are the requirements for fast rust compilation written down somewhere? Is that something you'd consider sharing, possibly with all of the tricks you've learned about how to get around various situations that would otherwise lead to slow compilations?