Because I want to learn to fish, not have fish delivered to me.

My last hit on SO was a search for “.gitignore wildcard”. What’s GTP-4 going to do with a two word prompt?

Made me curious, so I tried that exact prompt in the GPT-4 playground. Response:

  A wildcard in .gitignore file is a special character used to match multiple file names or patterns. The most commonly used wildcards are:

  1. `*`: This wildcard matches any number of characters in a single path component. For example, `*.txt` would ignore all files with a .txt extension.

  2. `?`: This wildcard matches any one character in a single path component. For example, `file?.txt` would ignore files like "file1.txt" and "file2.txt" but not "file12.txt".

  3. `**`: This wildcard is used to match any number of characters across multiple path components. For example, `**/build/` would ignore all files and folders named "build" at any level in the project structure.

  Examples:
  - `*.log`: Will ignore all files with a .log extension
  - `**/.DS_Store`: Will ignore all .DS_Store files in every folder
  - `build/`: Will ignore the "build" folder on top level directory
  - `/*/temp/`: Will ignore any "temp" folder at the second level of the directory structure (ignores sibling/temp, parent/child/temp but not sibling/child/temp)

  For more information, you can refer to the official git documentation on gitignore: https://git-scm.com/docs/gitignore

This is slightly wrong. build/ will ignore the build directory at every level. /build will ignore the build directory at the top level.

I also have no idea what it’s trying to communicate with its /*/temp/ examples

> This is slightly wrong.

A perfect representation of my git usage then!

> This is slightly wrong.

I wonder how it does if you tell it it’s wrong. I’ve tried a few times and it seems to get confused pretty easily.

"I apologize for the confusion. [Repeats the same wrong thing]"

I once had a project with a "bin-name" in the gitignore, to exclude the "/bin-name" that "go build" will produce. Then I added a "dir/bin-name/file.go" which of course got excluded. The CI/CD builds failed, but it worked locally, and great confusion ensued. Since then, I've always added that leading "/" and generally try to be as specific as possible in gitignore files.

This is a great example where many people providing different answers and comments based on their different knowledge and experience provide value, opposed to just ChatGPT telling you stuff. Simply "correct" is nice, but "correct, complete, and with experience and context" is better.

A priori, you can’t know whether this is accurate or a hallucination.

I really think the whole "hallucination" thing is a terrible framing. All that gpt-4 does is spew out bullshit. Now, it applies a bunch of statistics to attempt to spew out believable bullshit and those statistics cause it to sometimes accidentally be correct. The base assumption with this should be that it's wrong, and so we should instead be framing it as like sometimes accidentally producing "actualities" or something.

I agree with the first sentence, but unfortunately ChatGPT does a pretty good job with that particular two word prompt.

Ironic, since even if you're using SO, you're simply having the fish delivered to you in the form of answers to your questions there. If you really want to learn to fish, read the documentation and code more to build up one's skill.

I read both, the offical docs were the second hit. People who want one answer to everything scare me far more than AI.

I don’t really accept that SO is just answering questions for you. There are usually multiple answers. There were several comments on SO that went into more depth than the official docs. There are second opinions, and answers pointing out what the wrong answer is. There are links to related questions.

It wasn’t what I was looking looking for initially, but sometimes answers to questions you didn’t ask are better than ones you did.

I just had this experience today, trying to decipher the output of an obscure linux command argument, failing to dig it out of the man page; Google sent me through almost related stack overflow pages. Then I put a question in Phind and it gives me all the details, beautifully formatted, just a little too verbosely but with all the main points covered, with references.

It's time to update my search reflexes.

It's the opposite for me, I tried Phind a few times and couldn't get a valid answer to questions I could answer myself... Perplexing.

Isn't that like, exactly the same thing? - because GPT-4 was likely trained on SO (and reddit) data.

I have never asked a question on Stack Overflow; I am too impatient. However, I frequently find answers on Stack Overflow through Google.

The problem is that, when I have "how do I do X using Y API"-type questions, I usually consult the API's official documentation (or sometimes the source code), since it's generally more complete and up-to-date. The questions that take me to Google are usually either obscure errors or other hard questions. Even Stack Overflow answers are often irrelevant or outright wrong. ChatGPT is considerably less trustworthy.

As a random example, today I Googled 'topological sort cyclic graph'. By definition a topological sort can only be done on an acyclic graph, because otherwise it's impossible to avoid violating some ordering constraints, but I wanted to find algorithms for minimizing the number of ordering violations. Among the Google results, there were a few true-but-unhelpful answers ('you can't'), a few wrong answers ('here's some code to do it', author doesn't seem to really understand the problem)… and finally a few answers, all on Stack Overflow, referencing the "feedback arc set problem" and "minimum feedback vertex set problem". Those were the magic keywords that allowed me to learn more, including the fact that minimizing the number of violations is NP-hard.

I just tried asking GPT-4, and it gave an answer that had some correct parts but also had significant errors, and did not include the aforementioned magic keywords.

SO gives me better insight into who provided the information. When an answer is provided by a project’s main developer or a contributor I know I can rely on the info. From what I’ve seen with GPT it’s a popularity contest. Common mistakes and misinformation are regurgitated like facts and the high quality grammar + spelling makes things sound plausible.

Ask it about something you know is a frequently repeated mistake.

Those of you who anticipate continuing to use ChatGPT to answer programming questions: where do you think it'll get answers from if people who actually understand stop using q/a sites?

Where do you think ChatGPT finds these answers?

Documentation :)

I really wish that was always the case

Underrated content :)

And the source code!

Same place copilot does, github.