fun generate_valid_json(seed):
return "{}"And 4 is tremendously better than 3.5, in my own experience. Not perfect, but actually useful.
https://platform.openai.com/playground
Monthly subscription is only for ChatGPT. When you use the APIs you pay per token.
I still don't have access, except through the regular ChatGPT interface, which is mildly annoying. It would be interesting to experiment with the API.
What if the training data contains malformed JSON? There ought to be a non-zero chance of the LLM producing invalid JSON, no?
Sure it could be useful, but not really impressive.
I studied a little (literally 'intro to') ML at university, about enough to grok it as an application of stats, tie into things seen elsewhere, but not really more than that.
Every supposéd tutorial or explainer I've seen posted here or been able to find has been a weird (IMO) mix if simultaneously assuming a decent (at least greater than mine) ML background, but also really dumbed down clone this repo download that model switch between them like this, fine-tune them by cd'ing to this directory and ... Ok but what's actually going on?
I haven’t done tonnes of courses so there might be better. But this is good as a free one.
Sounds like you have a case of the Mondays. You just need to turn that frown upside down!
I can understand that you might be interested in book form only, I was lile this for the longest time, until I bumped into some really high quality video series that changed my mind to be a bit more flexible.
Also Important to note that LLM development is moving at a very fast pace recently so a book form might not be ideal. The basic ideas might be same, most of it might be out of date 6-12 months from now. I dont see how anyone could write a quality book that covers everything on this.
But I truly am starting from pretty much 'zero', and maybe I wasn't clear but I'm not looking to be 'hero' in the sense of up to date with the cutting edge, or even necessarily putting anything in to practice at all, I'm more interested in the background theory, and fine with that missing the absolute latest extra technique, just want to understand the meat of it better.
A refresher on SVMs & PCA (which I barely remember - I think I could convincingly explain SVMs to someone numerate but non-tech/mathematician, but not otherwise) and then a catch up to roughly what's going on with LLMs & image/video as mentioned would be great.
> I can understand that you might be interested in book form only, I was lile this for the longest time, until I bumped into some really high quality video series that changed my mind to be a bit more flexible.
I enjoy videos for many things, but mostly entertainment, I don't personally find I can learn that well from them, especially more technical/theoretical stuff, sure to some combination of screen fatigue, it being harder to skip around and reference something, and distraction - something seems obvious briefly so my mind wonders, check something in another tab 'just quickly', and before you know it ten minutes have passed, I've been hearing the speaking but suddenly realise I haven't been listening, have no idea what's going on any more.
Obviously they work for some people, that's fine.
If that seems surprising, it is worth doing a course like Karpathy's zero to hero NN, and have all the magic peeled away a layer at a time.
The reason you can do this is because LLMs don't just generate the next word or token, it produces a probability distribution over all tokens. A JSON parser can give you a list of next valid tokens. The tokens in each case might be from a different set, e.g LLM thinks of " The" whereas the JSON parser might think of "{", so you need some conversion there. But if you sample randomly from only the valid tokens, the output must be valid JSON.
What you can't build a parser for though is ... the truth! You may still be told lies or made up stuff.