News for hackers. Stuff that matters.
Re:Voyages in sentence space by TuringTest on Friday March 02, 2018 @06:04AM (#16500561)
The voyage looks somewhat random, yet inspiring. I can imagine using it to grab ideas for the overarching story arc in some storytelling.

> I am waiting at the entrance of the fantasy kingdom.

...

> I have a revelation about reality.

Main problem is, it doesn't look at all robust. A small change in a single word at the start or end sentence produces a whole different result. That doesn't look like a "space" at all, as insights about closeness do not work in it.

Re:Voyages in sentence space by ppod on Friday March 02, 2018 @07:13AM (#16500846)
This is essentially the reason that differentiable methods alone don't work that well for NLP. Language is a discrete arbitrary symbol system, so small changes may have large effects at many levels. ("Dog bites man" is not like "Man bites dog", discrete is not like discreet), etc.
Re:Voyages in sentence space by OscarCunningham on Friday March 02, 2018 @06:55AM (#16500769)
I guess that can happen sometimes in actual spaces too. The shortest path from a point near the north pole to the south pole can change dramatically if you slightly alter its endpoints. Given a high-dimensional space it could be that most possible paths behave in this weird way.
Re:Voyages in sentence space by jimmytidey on Friday March 02, 2018 @07:52AM (#16501004)
How useful is the spatial analogy if the space in question defies all our spatial intuitions?

I can see it has mathematical meaning, but isn't the whole post about having intuitive understanding of sentence space?

Re:Voyages in sentence space by gugagore on Friday March 02, 2018 @09:23AM (#16501520)
I think it's less about intuitive understanding, and more about having a flat, numerical representation of a sentence that (qualitatively) satisfies some properties.
Re:Voyages in sentence space by TuringTest on Friday March 02, 2018 @06:27AM (#16500641)
Ok, I've played with the "Look around" example and it does behave as a space, with the "Closer to home" option producing somewhat similar results to the starting point, and the slider producing concepts that look farther away from it.

Maybe the "Gimme a gradient" option step is too large for sentences that are far away, and an option to produce more steps, closer between them, would produce more intuitive results?

Re:Voyages in sentence space by maljx on Friday March 02, 2018 @09:53AM (#16501789)
I feel like the "middle" sentence is not intuitively what you'd normally think of something in between. Even for sentences that are quite close. "Eve walked home slowly" -> "Eve walked home quickly" gives a middle result of "As white hand smiled."
Re:Voyages in sentence space by rockostrich on Friday March 02, 2018 @10:10AM (#16501980)
Robin is an author that dabbles in machine learning and farming olives for making olive oil. It's not surprising that his examples aren't robust, but I always enjoy reading about his endeavors because I end up in a rabbit hole on whatever subject he's writing about.