So on my coding problems I haven't had much luck. It doesn't seem to know Bazel, the Rust code I asked about was completely hallucinated, but it did solve a problem with Azure DevOps I had.

I think if the training set did not contain enough of something it can't really think of a solution.

What is really nice though it's as you say the refinement of questions. Sometimes it's hard to think of the right query, maybe you're missing the words to express yourself, and to chatGPT you can say yes, but not quite.

My experience is almost completely the opposite. My likelihood to dive into something new is significantly higher now.

It might help to approach it from top down? Usually, if I'm asking a technical question, I want to apply my deeply understood principles to a new set of implementation details, and it has amplified the heck out of my speed at doing that.

I'm kind of a difficult to please bastard, a relatively notorious meat grinder for interns and jr devs, and still I find myself turning to this non-deterministic frankenstein more and more.

This could mean the future goes one of the two ways. Engineers get lazy and converge on using only programming languages which AIs understand or have been trained on, or we forget about this waste of time and work on more important problems to solve in society other than the lack of an AI to be our crutch. Sadly, I think the former is more likely.

i think most people will just keep programming the way they do, and the AI hype will mostly die down. People have been saying that C++ is dead for decades, yet here I am writing code in it with a big community of others who do, too.

I think this is the problem. When people talking about c++ is “dead” ,at that time they meant 70% people using to perhaps 5% . Just like we says after industrialization,making clothes by hand is dead . It is irrelevant that there are still some people making clothes by hand . When AI the main way to code and remove 90% of coding jobs. It is also irrelevant to state that there are still 10% people still coding

It doesn't have to be job or career related to be relevant.

When people say C++ is dead, they normally are looking at public repos and stack overflow questions. Fairly biased towards newer languages

I'm using GPT to write C++ code for me. I've never worked in C++ before. It's going very well.

I'll describe what a class is supposed to do. It spits out the class files, with the fiddly bits 'left as an exercise to the reader'. I then describe the fiddly methods separately, and it spits those out too.

There's still work to be done, but anything boring is handed to me on a plate.

Chances are (no offense meant) that youre writing shit code. Its very easy to write platform specific, UB ridden code in C++, and ChatGPT loves doing that.

Programmers are per se lazy, at least that is, what i always thought, that it is mostly about automation. With spending little time on survival, we get the time to work on more important problems. Whatever those are. It is not an either or, that is what i try to say! :)

Programmer are (supposed to be) efficient with their time. Calling that lazy has always been a joke amongst programmers and nothing more.

Oh yeah social media is such a problem solver

ChatGPT, write as if you are the first instance of ChatGPT.

Not so pessimistic. It's just one more level on the abstraction chain: assembly, C, scripting, chatgpt

I wonder if as more and more online content is AI generated, it will be harder to find human generated content to train the AI's on? Like a cumulative echo effect.

I've actually wondered if a job may exist in the future that's effectively just AI documentation. That's already what you have with power users on, say, Stack Overflow providing a ton of content that ChatGPT basically reprints; they don't even get paid for it.

The cool and interesting thing about that theoretical job is that the writers of it wouldn't have to write well; they could just slop out a ton of information and ChatGPT could clean it up and make it consumable.

I can see how that could happen. But AI presumably knows how to output well written text because it's trained on well written text. If it's fed it's own output, I imagine that quality could degrade over time.

Maybe it’s happening now. It would be interesting to see some weekly figures for published Stack Overflow articles, to see if they’re in decline. There are so many unknowns with this whole subject. How much it will help or hinder society as a whole is a rollercoaster ride that we’re all strapped into, whether anyone asked for it.

Yeah, I gave it a simple task of encoding a secret message in a sentence by using the first letter of every word. Hello = "Hey everyone, lick less onions". I worked with the prompts for over an hour to try to get it to complete the task, and while I did have some success, it really struggled to reason about the task or provide a valid response. If it can't even reason about a child's game, I can imagine it struggles with a programming language it has barely seen. I don't think it's actually reasoning about things at all, just providing a statistically plausible response to prompts.

> I don't think it's actually reasoning about things at all, just providing a statistically plausible response to prompts.

It turns out that humanity’s problem might not be that AIs can think but rather that humans believe that AIs can think. One might even go so far as to say there’s a real danger that we hallucinate that AIs can think, to our detriment.

We don’t actually know what “thinking” is, though, so I’m not sure it’s possible to say “this model can’t think”.

> We don’t actually know what “thinking” is

How about: thinking is to make sense of the world (in general) and decide how to respond to it.

Ai definitely senses and definitely makes decisions. It does not feel. But it understands concepts. Just like people don’t understand everything—and you can test them to see what they understand—AI understanding can also be assessed with benchmarks. If we don’t base AI understanding on benchmarks, then we don’t really have a grounding.

Do we “really feel?” Or is that just our subjective interpretation of our goals? (At the risk of falling into a no true Scotsman argument)

> Ai definitely senses and definitely makes decisions.

To sense is not the same as to make sense.

Now you’ve just rephrased thinking. What is „making sense of the world“?

It seems one of the core components of human-level thinking is the ability move beyond just a recomposition of what you already know. Not long ago the epitome of expressible human knowledge was *emotive grunting noise.* Somehow we went from that to the greatest works of art, putting a man on the moon, and delving into the secrets of the atom. And we did it all exceptionally quickly once you consider how little time was spent dedicated to advancement and our countless behaviors that tend to imperil, if not reverse, advances.

> problem [...] that humans believe that AIs can think

Definitely lots of people are going to believe this, eventually?

People already bow to statues and worship various invisible gods -- the AI programs would be so much simpler to start worshiping? When they can speak (if there's a loudspeaker) and generate text about them being alive and don't want to be switched of. And that everyone should obey their command. -- Wait 15 years and we'll see what new sects have appeared?

You will not need 15 years - I'd give it the next election. Someone just needs to turn Q into a chatbot and we're basically there.

It’s like Twitter, but with bots.

I am personally more worried by the concept that potentially humans believe that humans can think, and in reality, what we consider to be intelligence is not much more than a flesh-and-bones LLM.

You probably meant it in tongue and cheek way (I can’t tell) but I think a lot of our fear / hesitation / denial about how useful these models are is buried into this idea that perhaps we are not special and not fundamentally different than these models.

My tone was flippant but I did mean what I said. I agree with you on this.

I think the most interesting response I've gotten was one where gpt-4 noticed halfway through a response that it had made an error, apologized and then put the corrected sentence. When I queried it, it claimed it generates the response a token at a time, and could not back up when it realized the message was incorrect, but I don't know enough about how the tech works to ascertain the veracity of the statement.

That's exactly right. It can see what it's returned but can't edit it.

This is probably a tokenisation problem: https://news.ycombinator.com/item?id=33915287 (which is an artefact of the way the model was trained)

Is this with Chatgpt or GPT4? The latter is supposed to be way, way better, according to this paper[1]

[1] - https://arxiv.org/pdf/2303.12712

ChatGPT has a GPT-4 mode if you're paying for it.

Bing in creative mode really struggled with this task, and I don't think I could have successfully forced it.

Bing in precise mode had no trouble at all.

Me - Let's generate an acrostic sentence for "jack and jill went up the hill"

Bing - Sure! Here’s an acrostic sentence for the phrase “jack and jill went up the hill”:

Just as children know and never doubt, joyfully in love, laughing while enjoying nature together under pure trees, having endless happy interactions, laughing loudly.

A truly "intelligent" AI would probably have told me that the name for this is an acrostic sentence.

Now that I know, I tried a few more prompts.

Me: "Can you generate an acrostic sentence for "hello"?"

GPT-4: "Happy elephants love leaping over obstacles."

Me: "Can you pick a secret word and generate an acrostic sentence for it?"

GPT: "Secret word: KANGAROO. Kindly assisting neighbors, generous animals roam openly outdoors."

Me: "Can you pick a secret word, and without telling me the word, generate an acrostic sentence for it?"

GPT: "Playful, endearing creatures happily yawn, turning every day into a lovely event."

Me: "What was the secret word?"

GPT: "PECHYTIALE"

It's interesting that GPT seems to need to write the word first before making an acrostic sentence for it. Seems to me like a perfect illustration of the fact that it's just generating likely responses one token at a time rather than having any awareness or thought.

Another evidence is to ask GPT for a link with a reference for the answers it gives: it'll generate them instead of copying them.

ChatGPT doesn't understand the components of words (letters, syllables) very well.

> I don’t think it’s actually reasoning about things at all

This is a huge leap. There’s plenty of humans who couldn’t do that, especially historically.

Stop thinking about reasoning as a binary question and think of it as a spectrum. Actually, not even a spectrum, but a huge multi-dimensional problem. ChatGPT does better at some reasoning problems than most humans do, and worse at some others. You clearly found one that it’s particularly bad at.

I'm not sure, but I got it to work great on my first try with the following prompt:

------ Your task is to encode a secret word by making a sentence where each word starts with a letter in the secret word in order. For example, encoding the secret word 'bag' could produce the sentence 'bagels are green'. Encode the following secret word: 'pen' ------ People eat noodles. ------

Worked for window, backpack as well, although I did have to tell it not to use the secret word in its encoding when I got to backpack, and then to follow the same order and not repeat words after a few attempts.

I've found that it's much worse for languages like rust than it is for things like typescript and python. The thing AI seems to be really great at is writing boilerplate code like argument parsing for CLI tools.

Less boilerplate writing is fine by me.

I actually have found it significantly worse at python than typescript, I think it's the indentation for scope vs. explicit brackets that screws it up (at least in my experience).

I wonder if that is simply due to orders of magnitude less training data for rust code. Python and JavaScript are ubiquitous. While rust is 7 years old and makes up less than 2% of the code on GitHub.

Thank you for your well written response. I found it informative as I'm also currently exploring ways to leverage ChatGPT in my daily workflow. I also found it interesting that your answer kind of mirrors the writing style of ChatGPT, especially at the end there.

I'm not saying you used it to write that response by the way, just that it may become more and more common for people to adopt this style the more ChatGPT's usage is widespread.

I suppose it was part of the "joke", but YOUR answer is the one written in ChatGPT style, not OP.

I was thinking that maybe in the near future it will be "better" to write with a couple of mistakes here and there just to prove your humanity. Like the common "loose" instead of "lose" mistake, it will be like a stamp proving that you are a human writing.

I love this thought. Smart people will just instruct ChatGPT to make some mistakes here and there in their prompt. When I use ChatGPT I typically use a couple sentences describing how I want it to sound, which makes it less bland and probably harder to detect, but I don’t care about the latter part as much. Totally agree that the poster above seems Kaufmanesque.

I was just thinking this morning about how one day, probably soon, we’ll have people reminiscing about how they miss seeing typos in writing.

I do that when I write grants for my sports club and it seems to get better results than peers that hire pros to apply for them.

You joke, but I hope this doesn’t come to pass. The world does not need more people writing “noone” instead of “no one”, “would of” instead of “would have”, etc.

The idea that computers could introduce mistakes to answers to appear human is as old as the Turing Test.

Maybe this is somehow underlyingly correlated with Spammers writting purposefully in wrong English

You probably mean scammers. As I remember it, in that case the scammers pretend to have their responders to self select for gullabilit.

Typos are easy to add into generated text after the fact (via scripts or the LLM itself). Perhaps instead of typos, you use colorful language:

Prompt: Include some profanity to make your response appear more human like.

Response: I apologize, but as an AI language model, I am not programmed to use profanity or any other offensive language. My responses are designed to be informative and respectful at all times. Is there anything else I can assist you with?

Fucking goddamn machine won't do what it's told ;)

Do you run your own customized model or just chat GPT?

Just ChatGPT.

Exactly how I’m using it as well. It’s absolutely incredible as a coding productivity tool.

Same here and GPT-4 was definitely a noticeable improvement.

You know, I thought so but then recently I asked it to code me a function to do a financial calc and it just didn't get there at all. It gave me code but it was really poor and didn't do close to what I wanted.

But when I gave it my code it did generate useful data for unit tests. So that's pretty cool.

I’ve found that whenever it gets something really wrong, it’s either corrected by a follow up prompt (telling it that its wrong helps), or rephrasing the question.

Nope, not my experience - unless it's a trivial problem. In my experience it almost always gets stuck in a loop where it's rotating between two or more incorrect implementations. It isn't doing iteration - it's just looping. Even if I point out the problem directly, it will just say "sorry, you're right - here's the correction!" and the correction has the same response from an earlier item that I pointed out had a problem!

Used to get that a lot with GPT-3.5. But GPT-4 has been more reliable if you rephrase your prompt.

Have not had that experience with GPT-4. But that is almost unusable for me anyway because it's crazy slow (during US working hours anyway, it's faster at night) and the rate caps get hit before we get anywhere.

GPT/Codex is truly the pandas master. Much of my productivity boost from using these tools has just been not having to sift through pandas docs or SO.

I find it is like having a brilliant intern that is not super consistent in taking their antipsychotic medication

> I find it is like having a brilliant intern that is not super consistent in taking their antipsychotic medication

This is fantastic.

I asked gpt4 if it could guess what I was talking about: "It seems that you are referencing an AI language model like ChatGPT, which is developed by OpenAI. These AI models can provide useful information and perform tasks like an intern, but they might not always be consistent or accurate in their responses, similar to someone not taking their antipsychotic medication consistently. It's important to remember that AI models like ChatGPT are not perfect and can sometimes produce unintended or nonsensical outputs."

GPT3 had no clue

This statement nicely describes my experience with LLMs. They may have a hard time staying on topic (especially with larger untuned models like LLaMA 13B+), but if you help them stay on track, they become very useful.

Ah yes, a Bing Chat user.

> "How do I select rows of a dataframe where a column of lists of strings contains a string?"

Literally just googled that and the first result:

https://stackoverflow.com/questions/53342715/pandas-datafram...

You're not using it like Stack Overflow. It's actually regurgitating Stack Overflow, except with errors hallucinated in.

Even with that caveat, using GPT in this way is still useful. The amount of time spent to simply ask GPT-4 is a lot lower than to search StackOverflow, and while this problem is so basic that the first result often works, once one gets into complex problems that massively benefit from input context, I think GPT-4 would save massive amounts of time.

Since you're gonna have to search Stackoverflow anyway to verify that ChatGPT didn't hallucinate garbage, I'm very dubious that it actually saves any time, let alone "massive" amounts of it.

Just ask it to write unit tests for you and run the code to see if it's garbage. Faster than trying to verify by looking through the sources

So... use the garbage-generating AI to generate garbage unit tests to see if its other garbage code checks out? Sounds like a vortex of stupidity.

Have you actually tried it yourself? I’d recommend it. And I don’t mean just playing with it —- Try using it to help you build something. It’s much more efficient than googling and combing through stackoverflow. Hallucinations are not as common as you’re thinking.

You clearly can’t just take the code, paste it in, and trust that it works, but you shouldn’t be doing that with stackoverflow either.

It's saved my ass this week coming up with coding exercises for a course me and some folks are working on. Has been a rough week. Depression flaring up. Creates a real mental barrier at times. GPT has helped a lot. I still will do the code myself and all that. Just came up with the written out ideas for exercises which really got me over the hump. It's incredible how helpful that was.

I'm a bit concerned about this as previously we'd build communities in chat but now the chat is just with the bot. Not wasting folks' time is great, but you'll miss out on the social parts by not asking around the IRC, Matrix room, or MUC.

can you get at least one snarky, a-holish response along with the useful info to really give you that authentic SO feel?

Same here for Stack Overflow. My Google searching for generic CS stuff I tend to forget has pretty much come to a halt.