Every model learned by gradient descent is approximately a kernel machine (2020)

Re:Every model learned by gradient descent is approximately ... by xanderlewis on Sunday February 25, 2024 @04:27AM (#39499193)

I mean, I’m not sure what you’re trying to prove by asking for a Markov chain model like that. It’s trivially true that you can have a Markov chain output whatever you like (somewhat artificially, but we are talking about memorisation here) if you pick your training data carefully.

[ Parent | on HN ]

Re:Every model learned by gradient descent is approximately ... by CamperBob2 on Sunday February 25, 2024 @12:38PM (#39502815)

So "pick the training data carefully," and show me what I'm asking for, given that it is "trivial."

[ Parent | on HN ]

Re:Every model learned by gradient descent is approximately ... by xanderlewis on Wednesday February 28, 2024 @06:03PM (#39544660)

The data consists of a graph whose vertices correspond to the output sequence you desire and where there is an edge (with probability 1) from string x to string y if and only if x precedes y.

This model will arise from the desired sequence as a single training example (learning the probability of each pair of consecutive tokens), provided none are repeated.

Now run your Markov chain with initial input {first token in your sequence}.

[ Parent | on HN ]