Re:YC's Winter 2024 Demo Day confirms that we are indeed in ...
by rockemsockem on Thursday April 04, 2024 @09:17PM
(#39937742)
Interesting, what systems are you asking questions to? I find GPT-4 is pretty much always correct about simple questions I ask. I just avoid using it for anything really deep and refer to documentation instead.
Re:YC's Winter 2024 Demo Day confirms that we are indeed in ...
by packetlost on Friday April 05, 2024 @09:55AM
(#39942424)
I've mostly used GPT-4 and Gemini Pro. For simple stuff, it can be correct, but it highly depends on the topic. I've found it very effective as a dictionary or thesaurus but for technical questions, even simple ones like small help with a Scheme macro, it's near useless. I imagine if I was asking it about, say, Python/Django APIs or basic HTML/webdev stuff, it would do better.
Re:YC's Winter 2024 Demo Day confirms that we are indeed in ...
by rockemsockem on Sunday April 07, 2024 @02:25AM
(#39958611)
When you say technical questions do you mean writing code, explaining code, or something else? If it's writing code I agree, I don't often ask it to explain code unless I want to know about some obscure syntax, but it works pretty well for that. I would not trust it to explain complex logic in a function.
Re:YC's Winter 2024 Demo Day confirms that we are indeed in ...
by packetlost on Monday April 08, 2024 @02:45PM
(#39972568)
I've tried all of the above. I've asked it to summarize information about what languages in a class had strong symmetric multiprocessing characteristics and it made up over half of the details. I've asked it to generate code for a simple linked list and it created garbage. I've asked it to summarize what code does and it gets it blatantly wrong.
These are just a few examples. I've found it to be nearly useless for anything where a semblance of correctness matters. Every single thing it says must be validated against external sources and it can easily lead you down rabbit holes.
GPT-4 was a little better than 3(.5), but I've yet to see it be good at anything enough to warrant the price tag or the time.
This matches my experience perfectly. It gets decent/usable results like 40% of the time, but latency kills it.
> IMO chat-based question-asking when encountering unfamiliar language constructs or framework idioms is where LLMs shine in coding assistance.
I have never found this to be productive. It gets details wrong frequently enough that I can't trust the output which effectively doubles (or more) the cognitive effort because you have to validate that it's correct against a more reliable source, or find out the hard way that it's wrong when code/systems break.