News for hackers. Stuff that matters.
Re:Meaning Machine – Visualize how LLMs break down and simul... by jdspiral on Tuesday April 22, 2025 @06:55PM (#43767059)
I built a tool called Meaning Machine to let you see how language models "read" your words.

It walks through the core stages — tokenization, POS tagging, dependency parsing, embeddings — and visualizes how meaning gets fragmented and simulated along the way.

Built with Streamlit, spaCy, BERT, and Plotly. It’s fast, interactive, and aimed at anyone curious about how LLMs turn your sentence into structured data.

Would love thoughts and feedback from the HN crowd — especially devs, linguists, or anyone working with or thinking about NLP systems.

GitHub: https://github.com/jdspiral/meaning-machine Live Demo: https://meaning-machine.streamlit.app

Re:Meaning Machine – Visualize how LLMs break down and simul... by macleginn on Wednesday April 23, 2025 @04:45AM (#43769915)
The presentation is nice! The main point, however, is a bit misleading. From the title, one would assume that we will see something about how LMs do all these things implicitly (as was famously shown for syntax in this paper: https://arxiv.org/pdf/2005.04511, for example), but instead the input is simply given to a bunch of pretrained task-specific models, which may not have much in common and definitely do not have very much in common with what today's LLMs are doing under the hood.
Re:Meaning Machine – Visualize how LLMs break down and simul... by toxik on Wednesday April 23, 2025 @04:52AM (#43769954)
You shouldn’t link directly to the pdf, here is the abs page

https://arxiv.org/abs/2005.04511

Re:Meaning Machine – Visualize how LLMs break down and simul... by selfhoster11 on Thursday April 24, 2025 @06:15AM (#43780978)
I''m getting an error message with Streamlit: You do not have access to this app or it does not exist
Re:Meaning Machine – Visualize how LLMs break down and simul... by jdspiral on Thursday April 24, 2025 @07:45AM (#43781570)
I moved the app, it’s now tokenizer-machine.streamlit.app.
Re:Meaning Machine – Visualize how LLMs break down and simul... by georgewsinger on Tuesday April 22, 2025 @07:53PM (#43767392)
Is this really how SOTA LLMs parse our queries? To what extent is this a simplified representation of what they really "see"?
Re:Meaning Machine – Visualize how LLMs break down and simul... by jdspiral on Tuesday April 22, 2025 @10:04PM (#43768037)
Yes, tokenization and embeddings are exactly how LLMs process input—they break text into tokens and map them to vectors. POS tags and SVOs aren't part of the model pipeline but help visualize structures the models learn implicitly.
Re:Meaning Machine – Visualize how LLMs break down and simul... by kazinator on Wednesday April 23, 2025 @02:50AM (#43769334)
It doesn't help whatsoever. It's false bullshit.
Re:Meaning Machine – Visualize how LLMs break down and simul... by helloplanets on Wednesday April 23, 2025 @01:39AM (#43768958)
This is partly completely misleading and partly simplified, when it comes to SOTA LLMs.

Subject–Verb–Object triples, POS tagging and dependency structures are not used by LLMs. One of the fundamental differences between modern LLMs and traditional NLP is that heuristics like those are not defined.

And assuming that those specific heuristics are the ones which LLMs would converge on after training is incorrect.

Re:Meaning Machine – Visualize how LLMs break down and simul... by sherdil2022 on Tuesday April 22, 2025 @07:27PM (#43767244)
Embedding Visualization (DistilBERT) How Embeddings Work

ValueError: This app has encountered an error. The original error message is redacted to prevent data leaks. Full error details have been recorded in the logs (if you're on Streamlit Cloud, click on 'Manage app' in the lower right of your app).

Re:Meaning Machine – Visualize how LLMs break down and simul... by andai on Tuesday April 22, 2025 @09:00PM (#43767714)
Re:Meaning Machine – Visualize how LLMs break down and simul... by sherdil2022 on Tuesday April 22, 2025 @09:15PM (#43767794)
Great job! Do you have any plans to visualize/explain how machine translation - between human languages - works?
Re:Meaning Machine – Visualize how LLMs break down and simul... by jdspiral on Tuesday April 22, 2025 @10:15PM (#43768094)
Thanks! Yes — that’s on the roadmap, along with some other cool visualizations I’m working on. Machine translation is definitely something I want to work on: showing how models align meaning across languages using shared embeddings and attention patterns. I’d love to make that interactive too.
Re:Meaning Machine – Visualize how LLMs break down and simul... by sherdil2022 on Tuesday April 22, 2025 @10:33PM (#43768159)
I would love to get involved with that (I speak a handful of himan languages). Let me know if you are looking for collaborators.
Re:Meaning Machine – Visualize how LLMs break down and simul... by Dwedit on Tuesday April 22, 2025 @10:09PM (#43768071)
Send tokens to model, model goes brrrr, get output tokens back.
Re:Meaning Machine – Visualize how LLMs break down and simul... by Der_Einzige on Tuesday April 22, 2025 @11:42PM (#43768442)
UMAP is far superior to PCA for these kinds of visualizations and they have a fast GPU version available within CuML for awhile.
Re:Meaning Machine – Visualize how LLMs break down and simul... by wrs on Tuesday April 22, 2025 @11:48PM (#43768465)
Is there evidence that modern LLMs identify parts of speech in an observable way? This explanation sounds more like how we did it in the 90s before deep learning took over.
Re:Meaning Machine – Visualize how LLMs break down and simul... by Xmd5a on Wednesday April 23, 2025 @02:05AM (#43769101)
Re:Meaning Machine – Visualize how LLMs break down and simul... by dz0707 on Tuesday April 22, 2025 @11:53PM (#43768484)
I'm wondering if this could turn into some kind of prompt tunning tool - like to detect weak or undesired relationships, "blur" in embeddings, etc.
Re:Meaning Machine – Visualize how LLMs break down and simul... by synapsomorphy on Wednesday April 23, 2025 @12:10AM (#43768536)
This is somewhat disingenuous IMO. Language models do NOT explicitly tag parts of speech, or construct "trees of relationships" between words [1].

It also feels like motivated reasoning to make them seem dumb because in reality we mostly have no clue what algorithms are running inside LLMs.

> When you or I say "dog", we might recall the feeling of fur, the sound of barking [..] But when a model sees "dog", it sees a vector of numbers

when o3 or Gemini sees "dog", it might recall the feeling of fur, the sound of barking [..] But when a human says "dog", it sees electrical impulses in neurons

The stochastic parrot argument has been had a million times over and this doesn't feel like a substantial contribution. If you think vectors of numbers can never be true meaning then that means either (a) no amount of silicon can ever make a perfect simulation of a human brain, or (b) a perfectly simulated brain would not actually think or feel. Both seem very unlikely to me.

There are much better resources out there if you want to learn our best idea of what algorithms go on inside LLMs [2][3], it's a whole field called mechanistic interpretability, and it's way, way, way more complicated than tagging parts of speech.

[1] Maybe attention learns something like this, but it's doing a whole lot more than just that.

[2] https://transformer-circuits.pub/2025/attribution-graphs/bio...

[3] https://transformer-circuits.pub/2022/toy_model/index.html

P.S. The explainer has em dashes aplenty. I strongly prefer to see disclaimers (even if it's a losing battle) when LLMs are used heavily for writing especially for more technical topics like this.

Re:Meaning Machine – Visualize how LLMs break down and simul... by AIPedant on Wednesday April 23, 2025 @12:32PM (#43773915)
I nominally agree with this point - AGI is theoretically possible according to the Church-Turing thesis, we can “just” solve the Schrödinger for every atom in the human body.

The more salient point is that when a model reads “dog” it associates a bunch of text and images vaguely related to dogs. But when a human reads “dog” they associate their experiences with dogs, or other animals if they haven’t ever met a dog. In particular, cats who have met dogs also have some concept of “dog,” without using language at all. Humans share this intuitive form of understanding, and use it with text/speech/images to extend our understanding to things we haven’t encountered personally. But multimodal LLMs have no access to this form of intelligence, shared by all mammals, and in general they have no common sense. They can fake some common sense with huge amounts of text, but it is not reliable: the space of feline-level common sense deductions is not technically infinite, but it is incomprehensibly vast compared to the corpus of all human text and photographs.

Re:Meaning Machine – Visualize how LLMs break down and simul... by synapsomorphy on Wednesday April 23, 2025 @08:00PM (#43777995)
When a model reads "dog" it associates the patterns it gleaned from the text and images about dogs - its past 'experiences'. What is the difference in kind between that and animal understanding?

LLMs do have language-agnostic understandings in their latent space. "Dog" and "Perro" have largely the same representation (depending on the model. I believe more advanced ones show this more strongly?) as does a picture of a dog. I'm not sure if that's exactly the form of understanding you're referring to?

I agree the human text/images corpus is very small compared to evolution's millions of years of learnings from interacting with the environment, which is why I'm excited for RLing LLMs because it opens up the same data trove.

Re:Meaning Machine – Visualize how LLMs break down and simul... by gitroom on Wednesday April 23, 2025 @01:12AM (#43768810)
Nice seeing tools showing how models break stuff down, tbh I still get kinda lost with all the embeddings and layers but it's wild to peek under the hood like this.
Re:Meaning Machine – Visualize how LLMs break down and simul... by dbacar on Wednesday April 23, 2025 @02:04AM (#43769097)
:) kinda works I guess. "ValueError: This app has encountered an error. The original error message is redacted to prevent data leaks. Full error details have been recorded in the logs (if you're on Streamlit Cloud, click on 'Manage app' in the lower right of your app)."
Re:Meaning Machine – Visualize how LLMs break down and simul... by larodi on Wednesday April 23, 2025 @02:52AM (#43769351)
broke with Cyrillic text for me
Re:Meaning Machine – Visualize how LLMs break down and simul... by pamelafox on Wednesday April 23, 2025 @03:40AM (#43769590)
This looks like a fun visualization of various NLP techniques to parse sentences, but as far as I understand, only the tokenization is relevant to LLMs. Perhaps it's just mis-titled?

I actually worked on a similar tree viewer as part of an NLP project back in 2005, in college, but that was for rule-based machine translation systems. Chapter 4 in the final report: https://www.researchgate.net/profile/Declan-Groves/publicati...

Re:Meaning Machine – Visualize how LLMs break down and simul... by igravious on Wednesday April 23, 2025 @08:10AM (#43771175)
Completely misleading title/description
Re:Meaning Machine – Visualize how LLMs break down and simul... by jdspiral on Wednesday April 23, 2025 @09:55AM (#43772257)
So I've taken the feedback and realized that I was misleading on the name and title. I'm updating the project accordingly.

https://tokenizer-machine.streamlit.app/

Re:Meaning Machine – Visualize how LLMs break down and simul... by fransjorden on Friday April 25, 2025 @02:56PM (#43797336)
Don't forget to update the link of the post itself, as that one is broken now