wolfgangK

Isn't ChatGPT benchmaxxing, then ? Responding "hmm…" isn't actually responding and latency should time to first relevant phoneme.

I have zero interest in world knowledge for my LLMs but this got me wondering : are there RAGs for that kind of data ? How could a LLM like DeepSeek-v4-flash-vision-exp accurately answer you question with an indexed database of labeled landmark pictures (or even 3D models ?).

Indeed ! LLM are creative writers, not journalists. Relying on overfitting for factual accuracy in not tenable. I don't understand why grounded RAG with judges in not the norm.

You understand that drones are used for precision strikes instead of indiscriminate bombing, right ? It's the opposite of "sponsoring a genocide" (if there actually ever was one happening…)

Everything is food and air. There is no such thing a "human labor" without food and air to sustain it. I'm not sure what your point is.

BTW, I don't understand why having the driver converting AC to DC inside the LED spots is the default. For a new house, it seems to make sense to me to have at least one external driver for a ceiling of spots, if not one per floor (not one for the whole house because pf the drop in voltage). But 24V led spots seem much less common and I cannot find many reviews. Any idea why it's not more common ? It seems efficiency and thermals would be much easier to optimize without the size constraint, plus economy of scale with components. Any specific 24V led spots / bulbs to recommend ?

I presume this works (will work) also for JupyterLite that is based on Pyodide ? Would be great if it helped getting the latest OpenCV-python version [0] and it's dnn goodies being available on a zero-install client side Notebook !

[0] https://news.ycombinator.com/item?id=48421858

OpenCV being in the list of Pyodide modules [0] was the biggest boon for my online teaching experience because remotely dealing with install woes (corporate proxies & cie) was a show stopper for regular Python. I'm hoping that they will package this new version and that it will bring the new neural networks engines goodies to the no-install crowd !

[0] https://pyodide.org/en/latest/usage/packages-in-pyodide.html

Nice ! My most pressing request for VSS would be efficient binary vectors : is this on the table ?

Only those who don't care/know about prompt processing speed are buying Macs for LLM inference.

For LLM inference, I don't think the PCIe bandwidth matters much and a GPU could improve greatly the prompt processing speed.

Indeed, recent Flash Attention is a pain point for non CUDA.

The idea is presumably that you would "sell" at an artificially low price.

> The Soviets had the […]first woman,[…]

That is quite the claim !

You forgot the "/s", or do you actually believe that it's capitalism's fault is a mother taking care of her children is "unpaid labor" ?

"is hard" ≠ "sucks"

Most interesting ! Would you mind sharing the prompt and the resulting CLAUDE.md file ?

Thx !

IMO, it would be more interesting to have a 3-way comparison of price/performance between DeepSeek 671b running on :

1. M3 Ultra 512 2. AMD Epyc (which Gen ? AVX512 and DDR5 might make a difference in both performance and cost , Gen 4 or Gen 5 have 8 or 9 t/s https://github.com/ggml-org/llama.cpp/discussions/11733 ) 2. AMD Epyc + 4090 or 5090 running KTransformers (over 10 t/s decode ? https://github.com/kvcache-ai/ktransformers/blob/main/doc/en...)

DeepSeek is not a model.Which model did you use (v3 ? R1 ? a distillation ?) at which quantization ?

Nice ! Is it possible to connect to an in browser DB like WASM DuckDB https://duckdb.org/docs/api/wasm/overview.html or https://github.com/babycommando/entity-db ?

That would be most useful imho !

It seems that this aims to refutes claims for inaction with facts about spending money. However, the high speed rail project or homelessness management seem to show that in California, $$$ spent doesn't always imply that the problem is actually tackled in a meaningful way.

How do we know that this extension can be trusted ?

«Unfortunately, I have only seen 3 models, 3B or over, handle RAG.»

I would love to know which are these 3 models, especially if they can perform grounded RAG. If you have models (and their grounded RAG prompt formats) to share, I'm very interested !

Thx.

Just played a bit with it. Were you working with ASCII ? This example didn't work for you ? https://github.com/jfalcou/eve/blob/a141ba93048bb2916c2157a9...

Counterpoint : your message is not synthetic data and will contribute to lots of LLMs saying the same. Many such cases ?

(It seems to me obvious that a fgrep would sanitize synthetic data obtained from competitors.)

DeepSeek v3 can run on CPU & RAM :

https://www.reddit.com/r/LocalLLaMA/comments/1hqidbs/deepsee...

Epyc Gen4 and 12 memory channels of DDR5 @4800 should give you 7 to 9 t/s.

I don't think that Python would be the right language for such low-level performance maxxing endeavor. I would have picked C++ but t was eye opening for me to see how rust enabled such low level optimization, so I'm grateful for the choice.

Amazingly thorough ! I love how the author leaves no stone unturned. I had no idea you could do the kind of low level efficiency shaving in Rust. I wonder how a C++ implementation with https://github.com/jfalcou/eve would compare.

Most interesting ! Amazing job at optimizing various parts of the task. It seems that being an MoE with 'only' 37B active params per token would put it within the reach of CPU & RAM inference for the lucky hobbyist with an Epyc homelab and 8 or 16 memory channels on a second hand single or dual Gen2 mobo (around $2500 used). Any idea of how hard it would be (will?) support the new architecture for llama.cpp ?

I must confess that my interest in LLMs is grounded RAG as I consider any intrinsic knowledge of the LLL to be unreliable overfitting. Is DeepkSeek able to perform grounded RAG like Command R and Nous-Hermes 3 for instance ?

Thx for this amazing model and all the insights in your report !

For training, doesn't checkpoint saving make high reliability a moot point ? Why pay for 99.99999? uptime when you can restart your training from last/best model ?