Also curious on this. They are too dumb to use with tools and if you ask it how to bake a pancake you only get rubbish.

I mostly agree today, but I'm trying to push for the future where smaller models are more capabable, and I believe it will happen + the fine tuned cases I mentioned in other commments.

On other notes : there is no free lunch, you have to sacrifice something for the other

Not op but my opinion is that It doesn't matter if it's not currently useful. I think it's important that the community focuses on this line of research because big companies will not. It's in their best interest to make LLMs resource hungry

Yeh; this. They see how GTA6 looks and know a hypothetical scenario like a future Nvidia 7000 series with 96GB VRAM and 512GB system RAM in everyone's home means the entire tech, media, knowledge work labor system; the rich peoples social moat; evaporates.

It's all quite clearly collusion among a few dozen, hundred. No way their education/experience or data analysis engines have not made them aware.

They’re just used to Boomer and Silent Generation politicians blithely enabling them cause line go up for them when they do!

Curious to see how well it works on Millennials and GenZ; the elders were educated before STEM education had matured in a lot of ways. Younger generations have not had debunked fantasy rammed down their throat as scientific fact; Nostradmus types of the mid-late 1900s pointing at some ignorant math assuming it's true because it's written in a scientific notation! Duh!

Science education of the past gave rise to thinkers like the Oceangate CEO. Nihilists who figure there's no such thing as forever and we all die anyway; so YOLO no matter who I take down with me.

WAT

Science education peaked in the 50s when people still believed there need to be a balance between theory and experiments and that grounding theory with experiments is worthwhile.

It went all downhill from there. In the 70s and 80s experimental physics got a bad rap and I think this mindset trickled down to education. The turning point for me was the 1993 cancellation of the Superconducting Super Collider, where this tendency must have become obvious to everyone.

Not sure about your other points and what you conclude from a perceived improvement of STEM education, but I am convinced boomers and Gen-X enjoyed more solid scientific training than later generations.

This is an interesting question, so the idea is, based on experience, fine tuned small models can do narrow tasks very well (my experience running models fine-tuned on customer data as low as 1-2B is very promising) also the goal is to find a breakthrough were instead of the line of work (which I respect) that tries to squeeze much bigger models into smaller hardware like strata and maya, I'm working backward to discover/search for custom kernels and modifications till we exhaust the limit of what a smaller machine is capable of, so instead of people doing 64gb machines and old gpus to run a deepseek, I want to see what is the biggest usable qwen small model can run with reasonable speed on smaller devices, and by scaling as we discover more tricks, normal laptops can unlock more compute - for example a qwen 3.8 27B can do pretty serious work when coupled with a specialized harness - but we are yet to figure out what is the smallest machine that can run it with usable output and usable speed, and this can unlock interesting business.

I'm told that qwen 3.8 27B is good enough for practical programming work, so if that's doable then I could easily see the utility. Or... the biggest qwen I see here is Qwen3.5-2B; is that actually capable of programming or agentic work with code?

[delayed]

> In the last month we've gotten 4B models that can do small tasks.

where can I read up on that?

Check LFM as a starting pointer : https://www.liquid.ai/models

The best one I know of at the moment is XHToken's Spark-x2.6 4B.

I find this compelling “merely” as an engineering challenge, but I’m curious if you’ve observed an actual practical benefit?

By that I mean, have you found a case where a fine-tuned, small model is outperformed by stuffing a heavily tuned larger model in its place?

I had this case in fine tuned small models (up to 27B) in finance and legal, over bigger models (cost-wise, on-prem for a regulated institution, and better perofrmance because we owned the data pipeline)

Speech recognition, speech generation.

This alone is quite useful in small/limited devices/contexts.

Realtime log parsing and summarization is another application I've seen for little LLM's..

Basically, anywhere you'd use an LLM, but at much, much smaller scales of ontology.

speech models are what I'm trying next because imagine a nice portable meeting transcription device that can hold around without internet and for NDA-ed meetings and so (with permission of course)

If you work in a highly regulated space subject to e.g. HIPAA, and you work with ancient on-prem systems within a slow moving and risk intolerant organization -- for instance, virtually every state run healthcare system in the US -- then these are all you can hope for.

In these settings, data is insanely messy beyond your wildest nightmares and hidden away in free-text fields that small models like this are great at cleaning up, normalizing, parsing, categorizing, etc

Agreed, and it's a new use case to me because actually in the cases I had the clients were okay with getting GPUs and handling the data mess for us, but none of them requested fine tuning, I believe trying to fine tune will be insanely complex from a regulatory prepsective.

Do you work at an AI consulting company or how are you getting exposure to these regulated industries?

(In another comment you mention being involved in finance and healthcare)