News for hackers. Stuff that matters.
Re:The Myth of AI Omniscience: AI's Epistemological Limits by dvt on Saturday August 05, 2023 @11:46AM (#37013172)
> This is not true and is easy enough to test.

How exactly is this not true? Embeddings are literally a mapping of (English) words to numbers.

Re:The Myth of AI Omniscience: AI's Epistemological Limits by cle on Saturday August 05, 2023 @11:57AM (#37013282)
Test it!

> Can you make up a word?

> Certainly! Here's a made-up word for you:

> *Glimberfleck*: The sparkling reflection of sunlight that dances on water, making it seem like the surface is sprinkled with tiny, shimmering gems.

> Feel free to use it or let me know if you'd like me to create another word!

> When I’m looking out of the window of an airplane at the ocean, am I looking at a glimberfleck?

> Yes, if you're looking out of the airplane window at the ocean and observing the sparkling reflections of sunlight on the water's surface, you could describe that beautiful phenomenon as a "glimberfleck." It's a whimsical term for those captivating, shimmering patterns you see.

Re:The Myth of AI Omniscience: AI's Epistemological Limits by Davidzheng on Saturday August 05, 2023 @11:49AM (#37013196)
But because the tokens are not word level they are capable of making up words that humans have never used before.
Re:The Myth of AI Omniscience: AI's Epistemological Limits by dvt on Saturday August 05, 2023 @12:00PM (#37013309)
Most tokens are word or sub-word level. A token can be "find" or two tokens can be "find" and "ing." To argue that this isn't inherently using the underlying grammar rules is plainly wrong. To argue that an LLM trained on English text is capable of "making up words" (in any meaningful sense of that sentence) is even more wrong.
Re:The Myth of AI Omniscience: AI's Epistemological Limits by jsight on Saturday August 05, 2023 @12:03PM (#37013345)
You might think it is wrong, but I think it is a perfectly cromulent case of AI "making up words".
Re:The Myth of AI Omniscience: AI's Epistemological Limits by jstanley on Saturday August 05, 2023 @12:09PM (#37013421)
Does this satisfy you?

  Me: give me a random character sequence that is not a pronounceable word

  ChatGPT: Sure, here's a random character sequence that is not a pronounceable word:
  "Jkplqxzv"
Re:The Myth of AI Omniscience: AI's Epistemological Limits by zarzavat on Saturday August 05, 2023 @12:02PM (#37013337)
It’s not a mapping of words. It’s a mapping of character sequences. You can ask chatgpt to define “hackernewsitis (zero hits on google) and it gives a plausible definition.

Also the fact that it can write code is evidence that it can understand new concepts. A variable declaration is a coining of a (very short lived) new word.

Re:The Myth of AI Omniscience: AI's Epistemological Limits by dvt on Saturday August 05, 2023 @12:08PM (#37013405)
> It’s not a mapping of words. It’s a mapping of character sequences.

This is plainly wrong if you read the word2vec paper.

Re:The Myth of AI Omniscience: AI's Epistemological Limits by jstanley on Saturday August 05, 2023 @12:12PM (#37013447)
It depends which tokenisation you use. Here's the token list for GPT-4: https://incoherency.co.uk/interest/gpt4-token-list.txt

Some of them are words, some of them are not. It uses the same list even if you have read the word2vec paper. It probably doesn't even know whether you've read it.

Re:The Myth of AI Omniscience: AI's Epistemological Limits by visarga on Saturday August 05, 2023 @01:03PM (#37014095)
Hackernewsitis is a made-up term that refers to the tendency of some people on the online forum Hacker News to get overly focused on or obsessed with certain topics, technologies, or companies.

Some characteristics of hackernewsitis include:

- Endlessly debating or commenting on the same subjects over and over, even when no new information is being contributed.

- Getting emotionally invested in certain technologies or companies, treating them like sports teams to cheer for rather than objectively evaluating them.

- Assuming the conversations and interests of Hacker News represent the tech industry as a whole, losing perspective on what is niche vs. mainstream.

- Spending too much time on Hacker News reading and commenting rather than working on your own projects and goals.

- Seeing the world overly in terms of tech industry buzzwords, losing appreciation of other domains.

- Acting overly cynical or pessimistic about new technologies based on theoretical risks rather than evidence.

So in summary, "hackernewsitis" refers to the potential for discussions on Hacker News to become unproductive echochambers if participants are not self-aware. It highlights the need for individuals to think critically and not get swept up in groupthink dynamics.

(Claude)

Me: Does this look like AI not understanding?

Re:The Myth of AI Omniscience: AI's Epistemological Limits by tekla on Saturday August 05, 2023 @01:16PM (#37014243)
This is literally a perfect definition.
Re:The Myth of AI Omniscience: AI's Epistemological Limits by K0balt on Saturday August 05, 2023 @11:52AM (#37013225)
It is possible to draw new conclusions from complex data. It’s not something that a human might not also see, just that it’s an undiscovered relationship. (Or at least not directly expressed except by inference in the training corpus)
Re:The Myth of AI Omniscience: AI's Epistemological Limits by squeaky-clean on Saturday August 05, 2023 @04:12PM (#37015876)
Embeddings are usually common n-grams of bytes, not whole words unless it's a very common word like "it". It's also not all English.

> Hey ChatGPT, can you give me a brief explanation of AI embeddings, but respond in Chinese?

> 当然可以。

> AI 嵌入(embeddings)是一种将词汇、句子或其他类型的信息转化为高维度的向量的技术。这些向量捕捉到了原始信息的某些语义或关系特性。通过嵌入,我们可以更容易地在高维空间中比较和处理这些信息,如在机器学习模型中。

> 希望这有助于你了解!

As for saying it can only mimic words it knew in it's training data, this one gives zero results in google

> Hey ChatGPT can you make up a word for when I'm hungry but there's nothing in my kitchen I want to eat, so I'm not that hungry.

> Sure, how about "snackstipated"? It's a blend of "snack" and "constipated", suggesting a feeling of hunger but being stuck with unsatisfying options.