News for hackers. Stuff that matters.
Re:Translating Akkadian clay tablets with ChatGPT? by jiggawatts on Tuesday May 16, 2023 @12:24AM (#35957726)
Some people have been "strengthening" Chat GPT by re-running the same query several times, then combining the various outputs. I asked GPT 4 to translate one of the linked snippets twice, pasted in the original plus the variants, and told it to merge all three into an improved translation. I also gave it metadata and context from the page. Without that, it complained that an isolated snippet of Sumerian or Akkadian is too difficult to translate:

    Esarhaddon, the great king, the powerful king,
    King of the Universe, King of Assyria, Governor of Babylon,
    King of Sumer and Akkad, King of the Four Corners of the Universe,
    Wisely nurtured by the great gods,
    From his earliest days, Assur, Shamash, Enlil, and Ishtar,
    The gods of Nineveh and the gods of Assur,
    For the kingship of Assyria, have determined his fate.
    The circumstances of my origins, my noble lineage, they caused me to understand.
    In the service of Assur, Sin, Shamash, Enlil, and Ishtar,
    The gods of Nineveh and the gods of Assur, they created me,
    Amidst my numerous heroic deeds, my valor has no equal,
    They said, "This is the one, let him rule!"
    Shamash and Adad revealed this to me through a favorable omen,
    They declared to me: 'You are the one who will shepherd our people',
    Their weighty destiny, they assigned, and the land of Assyria became great,
    My numerous offspring, the product of my loins, they caused to grow and multiply.
    Before Assur, Sin, Shamash, Ishtar, Marduk, the gods of Assyria,
    The gods, my helpers, who reside in heaven and earth, they have enlarged my reign.
    Their weighty command, they caused to be heard,
    In the days of abundance, from dawn to dusk, in their temples, they have decreed my fate.
This is from the first 20 lines of: https://cdli.mpiwg-berlin.mpg.de/artifacts/422293/reader/201...

I have no idea what a professional translation would say, but the GPT 4 output seems pretty coherent to me.

Re:Translating Akkadian clay tablets with ChatGPT? by singularity2001 on Tuesday May 16, 2023 @01:58AM (#35958175)
given how even professional translations sometimes vastly diverge it's not too bad indeed. Getting there from highly damaged and ambiguous cuneiform tablets with multimodal Models will be a bit more difficult but there's hope for GPT5.
Re:Translating Akkadian clay tablets with ChatGPT? by consp on Tuesday May 16, 2023 @02:04AM (#35958209)
The first text is not the same as the second. A lot of words are the same but the meaning is not. It looks like it recognised some parts and made the rest up (at least to me) with the provided metadata.
Re:Translating Akkadian clay tablets with ChatGPT? by namaria on Tuesday May 16, 2023 @04:03AM (#35958880)
Averaging out bad output cannot produce good output. Or as the old saying goes, garbage in, garbage out.
Re:Translating Akkadian clay tablets with ChatGPT? by jiggawatts on Tuesday May 16, 2023 @04:47AM (#35959175)
It's not necessarily "garbage".

A known limitation of word-at-a-time GPT models is that they can't backtrack, edit, or iterate on what they're doing. You can see these limitations when asking them to solve puzzles involving "term substitution". They can only do these up to a certain complexity, and then they run out of steam.

However, if you run their output back through for a second round, they can continue substituting!

Essentially, GPT models make mistakes, but can also recognise those mistakes and fix them on subsequent iterations. Much like humans.

Another factor is that the "temperature" setting means that the outputs have an element to them that's purely random. Combining multiple outputs "anneals" out this randomness by converging on output that is the most consistent. Essentially, it's like asking a team of people to brainstorm, and then combining their partial solutions into a complete one. Once again, very human-like!

This has been tested (semi) scientifically. Running GPT 4 across some puzzles or exams might yield a 60% success rate if you give it one chance. If you let it solve each problem by running it four times and then letting it combine its own outputs itself (not via human-selection!), then it will get up to an 80% success rate. See: https://www.youtube.com/watch?v=wVzuvf9D9BU

Re:Translating Akkadian clay tablets with ChatGPT? by namaria on Tuesday May 16, 2023 @04:56AM (#35959225)
There's nothing scientific about this. People are just holding a black box and saying "if I jiggle it like this, I can get better results". Approximating better results in simple contrived puzzles is hardly evidence that recycling the output makes it better in any absolute sense. Besides, translation being open ended and heavily context dependent will likely suffer from such recycling.
Re:Translating Akkadian clay tablets with ChatGPT? by jiggawatts on Tuesday May 16, 2023 @05:25AM (#35959427)
They're taking objective measurements of experimental outcomes while varying aspects of the setup. How is that not scientific!?

"If I jiggle this is behaves differently" is like... half of all science!

At the end of the day, better is better, and I don't think that can be hand-waved away.

That's like saying someone is not smarter because they just "studied harder" and that doesn't count somehow.

Re:Translating Akkadian clay tablets with ChatGPT? by namaria on Tuesday May 16, 2023 @05:40AM (#35959536)
None of the papers are offering falsifiable premises and testing them. This is not doing science, this is merely assembling metadata. I find shameful that serious academics and institutions are indulging in this kind of number games.

"Let's publish a bunch of measurements we took" might be rigorous and look like what advancing science would look like if you squint. But this is just gaming the publishing game.

Re:Translating Akkadian clay tablets with ChatGPT? by jiggawatts on Tuesday May 16, 2023 @06:43AM (#35959941)
A decent chunk of biology was simply collecting bugs and pinning them on a board.

A lot of science is just taking down measurements and recording the result.

It's not always a giant leap, some grand insight, or a world-changing theorem.

Re:Translating Akkadian clay tablets with ChatGPT? by namaria on Tuesday May 16, 2023 @07:06AM (#35960122)
No, a large chunk of biology was figuring out species evolve and were not created as they are right now. Or that every living thing is made of cells that share the same basic chemistry and structure, not special kinds of tissue.

The busywork in between might have looked like science and occupied a lot of time and space. Relativity and quantum mechanics generated a lot of falsifiable claims that were eventually tested. Yet we have been building larger and larger colliders to "collect and publish data" and the best outcome so far has been a job program for engineers (useful and good) and the Web (useful and good). But we shouldn't confuse collecting data and engineering with advancing science.