News for hackers. Stuff that matters.
Re:Ask HN: Is anyone else bearish on OpenAI? by bmitc on Friday November 10, 2023 @07:15PM (#38226262)
Are you able to give a specific example of a problem it helped you solve? Especially one that you were at a complete blocking point, it provided some solution, and then you were able to continue to expand upon that solution.

I keep reading responses like yours, but I haven't seen any specific examples of problems being solved, so it all sounds very abstract. In my interactions with ChatGPT, it felt like just interacting with a search engine. There were zero continuations between questions and responses, and nearly 100% of the responses contained incorrect information.

Edit 1: As an example, I just asked ChatGPT to implement the Warren Abstract Machine for me. It gave me two different implementations, both with caveats that they are simple examples and not the whole thing, and neither implementation even type checked. It feels just like reading someone's homework where they copied off of someone else's work and had no idea what's going on. I don't see the point in this if it's just going to give me some high-level idea, which I already have, and an implementation that isn't remotely complete, much less one that doesn't even run. "Additionally, you'd need to ... handle backtracking". You don't say, ChatGPT?

Edit 2: I've kept on asking it to implement things I already know how to implement, even things that it probably has my code (being it was on GitHub) in its training data, and it keeps giving me code that doesn't even typecheck. ChatGPT is just doing what I've always imagined it doing: it's just doing a statistical merge. It has zero concept of anything I'm asking or it's saying.

Edit 3: I asked it something regarding WebGPU. It gave the typical, oh that's complex and "Here's a simplified example using GLFW and WebGPU through the F# WebGPU bindings provided by FableGL". But FableGL isn't a thing that exists, and even if it did, it wouldn't have anything to do with WebGPU which has nothing to do with OpenGL. And it imported Fable.Import.WebGPU and Fable.Import.GLFW, neither of which exist.

Re:Ask HN: Is anyone else bearish on OpenAI? by ghshephard on Friday November 10, 2023 @09:27PM (#38227090)
Random example from today - we have a new product being deployed from a prominent vendor, and they started running into a ton of "out of socket" errors. This product was important to one of our telemetry organizations Q4 OKRs, and they weren't getting traction with the vendor, so they started reaching out internally. I got tagged as someone with a background in networking (though this was clearly not a networking issue) - and started poking around. The first thing that comes to mind of course, is file descriptions, but ulimit -n returned a massive number and "ss -a" showed there were a tiny number of sockets opened (like, 1300 - we have lots of nodes with 500K+ sockets opened - so this really didn't feel like a node or docker container issue).

So - I just reached out to Chat, and we started going back and forth, starting with "In linux, how many open sockets can there be at a time?". What's nice, about that wide open question, is you don't get a single answer, instead you get a briefing on Linux Sockets. File Descriptors, sure, but also memory, port range, TCP/IP stack limits, etc... It starts to lay out a roadmap towards solving the issue, answering the question you were interested in, rather than the one you asked.

I do a bit of back and forth on some scenarios, asking about /proc, and ss, etc.. seeing if I can track anything else out. And then, after spending about 5 minutes, and building context, I ask it "Is there anything else that can cause an error regarding too many sockets with low socket use" - at which point it lays out a number of scenarios, one of which is FD_SETSIZE.

So - we dig into FD_SETSIZE, and immediately that looks interesting - it's the limit that you can use with a select() call, and, even better, I get recommendations to use POLL or EPOLL (which anyone who has ever straced has seen a ton of).

I ask it how to determine FD_SETSIZE, discover it's 1024 on the client, which matches our low socket count, confirm that we should never increase the size of FD_SETSIZE, check the vendor code, see they've got it hard configured to talk with select() instead of poll - we recommend they give us a new build with poll() defined - and voila - problem goes away.

On to the next issue.

Where Chat excels is not in solving or completing anything, or, in fact, even being correct 100% of the time (it clearly isn't) - but it's an endlessly enthusiastic savant intern - frequently wrong in hilarious ways, but always willing to dig in and pull on technical threads and come up with things that I can try with a bunch of rapid iterations, and close off an issue. It's willingness to write code that is 90% correct just reduces the time, and cognitive load of constantly having to do it all yourself.

Re:Ask HN: Is anyone else bearish on OpenAI? by LVB on Friday November 10, 2023 @10:10PM (#38227303)
“endlessly enthusiastic savant intern” is a good description!

I’ve had to sharpen my skills at verification and cross-checking, but the volume of pretty good insight from a much wider domain than I myself could consider has been so worth it.

It’s also worth noting to that I feel like I used to get this type of help, roughly, from good search engines. Maybe not quite as spoon fed but pretty good. But now… just blogspam and paid listings.

Re:Ask HN: Is anyone else bearish on OpenAI? by CodeyWhizzBang on Tuesday November 14, 2023 @08:08AM (#38262843)
This might be me being picky, but you mentioned at first:

"I've essentially got 15-20 high-priced world-class consultants in every field that I chose to pull from"

And then here said:

"It's an endlessly enthusiastic savant intern - frequently wrong in hilarious ways"

And those feel very different to me.

The second one feels more similar to my experience This is what I spot with ChatGPT (and advocates of it): in the abstract, it is a genius, but whenever we look at a specific example, it's not like that at all.

Re:Ask HN: Is anyone else bearish on OpenAI? by aprdm on Friday November 10, 2023 @11:29PM (#38227636)
I think this speaks to the bubble you're in and how your kind of software development probably represents 0.1% of what's out there. I don't even know what WebGPU, Warren Abstract machine or "whatever is that you're trying to do" is.

I've been doing dev for ~20 years now, most of the work out there is essentially plumbing events/databases/REST APIs/regexes/JSON/configs/ web UIs in existing and well established frameworks/tools to solve business problems.

You likely work more R&D style environments, which again, probably represent a very low % of devs out there

Re:Ask HN: Is anyone else bearish on OpenAI? by singron on Sunday November 12, 2023 @11:51AM (#38241838)
The parent comment claimed they had the value of 15-20 high priced world class consultants from every field, but this makes it seem like it does only the most basic common tasks.
Re:Ask HN: Is anyone else bearish on OpenAI? by bmitc on Monday November 13, 2023 @12:32AM (#38247296)
Agreed. I asked it things I actually need help with, not relatively mundane tasks that are easily accomplished by reading documentation. And the fact that it answers so confidently and yet wrong concerns me.
Re:Ask HN: Is anyone else bearish on OpenAI? by ad404b8a372f2b9 on Tuesday November 14, 2023 @01:02PM (#38266972)
I've never met a consultant who seemed useful and I've met many from the big 4 so in that regard GPT measures up.
Re:Ask HN: Is anyone else bearish on OpenAI? by sangnoir on Sunday November 12, 2023 @08:18PM (#38245837)
> I think this speaks to the bubble you're in and how your kind of software development probably represents 0.1% of what's out there.

Grandparent specifically said they find ChatGPT useful for things outside their wheelhouse:

>> [...]troubleshooting situations that flow into me - working in areas that I have the slimmest of backgrounds in

It sounds to me like it may not be safe to use ChatGPT if you can't check its work.

Re:Ask HN: Is anyone else bearish on OpenAI? by bmitc on Monday November 13, 2023 @12:33AM (#38247304)
Yea, I tried using it like that and got a bunch of "you're using it wrong responses".
Re:Ask HN: Is anyone else bearish on OpenAI? by meiraleal on Monday November 13, 2023 @09:42AM (#38250618)
Too bad for you to not be able to extract value from it. Millions are doing it just fine.
Re:Ask HN: Is anyone else bearish on OpenAI? by sangnoir on Monday November 13, 2023 @12:33PM (#38252770)
This comment sounds a lot like "Have fun staying poor", as applied to AI rather than crypto.
Re:Ask HN: Is anyone else bearish on OpenAI? by meiraleal on Monday November 13, 2023 @02:44PM (#38254370)
If someone wants to be poor by choice, I wish them to have fun.

There are millions of people making good use of chatgpt but some think it is a conspiracy theory and it actually doesn't work. What can be done? Let them enjoy their ignorance.

Re:Ask HN: Is anyone else bearish on OpenAI? by avion23 on Monday November 13, 2023 @11:19AM (#38251929)
Not the original poster, but here are some examples:

- Paste a bunch of log code about which I have no idea. Ask to identify and explain the problem.

- Wireshark / dmesg / OpenWrt configuration pasting, ask to fix the problem. For instance, I fixed a Wi-Fi issue in a heterogeneous setup, which turned out to be caused by a stray DHCPv6 server.

- Paste C code, along with an error log, and ask to fix the problem.

- Paste my program and a sample. Ask to extend my program.

- Proofread and format Markdown nicely.

- Paste government letters, asking for a response that includes <what I want>.

- Paste a chat log and obtain documentation.

- Paste a tax declaration, and ask to check for consistency.

- Paste my code and ask for critique.

When discussing versions, people often confuse versions 3.5 and 4. I am always referring to version 4.

Keep in mind that ChatGPT has been seriously and intentionally downgraded since March.

This debate frequently leaves me wondering if I'm encountering a coordinated effort by bots. The examples I listed above come very naturally to me. I can't understand why people don't try to paste whatever they're working on and check the results. If it's too complex, asking it to critique instead of create, because that's easier. It feels as though there's an effort to shape public opinion into viewing these tools as "immature" and suitable only for edge cases.

Re:Ask HN: Is anyone else bearish on OpenAI? by sangnoir on Monday November 13, 2023 @12:36PM (#38252817)
> This debate frequently leaves me wondering if I'm encountering a coordinated effort by bots.

Stay classy. The rest of your comment would have been much better without this ad hominem.

Re:Ask HN: Is anyone else bearish on OpenAI? by avion23 on Tuesday November 14, 2023 @08:57AM (#38263325)
It wasn't meant ad hominem, I am really wondering. I feel in a bubble.

I once spent an hour researching the responses on reddit and hackernews. Turned out if these profiles are fake, then they are really good. F.e. age is >5 years, they argue about sport teams and seem really human.

Re:Ask HN: Is anyone else bearish on OpenAI? by aprdm on Monday November 13, 2023 @12:55PM (#38253005)
Yeah I think the people who haven't found value are people who either don't have strong communication skills hence cannot articulate what they need OR work in very speciailized fields - either way they're clearly the minority seeing how many paying users chatgtp has and how fast they got there
Re:Ask HN: Is anyone else bearish on OpenAI? by LouisSayers on Monday November 13, 2023 @01:58PM (#38253770)
> don't have strong communication skills hence cannot articulate

Basically this - even in a highly specialised field you would still find value in it if you're providing enough context and describing the problem properly.

People don't want to hear that they're poor communicators - it's much easier to blame the tech.

Re:Ask HN: Is anyone else bearish on OpenAI? by bmitc on Tuesday November 14, 2023 @02:19AM (#38260104)
> People don't want to hear that they're poor communicators - it's much easier to blame the tech.

It's just hard to even know what to say to this. So you are implying that the problem lies with the human, who holds years of experience in education, problem solving, communication, etc., and not the "AI", which has no experience in any of that and has no real thought processes or problem solving or reasoning abilities? And people say ChatGPT is over-hyped!

Re:Ask HN: Is anyone else bearish on OpenAI? by avion23 on Tuesday November 14, 2023 @08:53AM (#38263285)
Not OP, but yes.

I've had some chatlogs where people are not able to articulate even one clear sentence. They were fine telling me what they want, but not able to type the same sentence

Re:Ask HN: Is anyone else bearish on OpenAI? by bmitc on Tuesday November 14, 2023 @02:16AM (#38260081)
> Yeah I think the people who haven't found value are people who either don't have strong communication skills hence cannot articulate what they need OR work in very speciailized fields

This is so weird and seems to be a common thread. You, and others, seem to be saying that ChatGPT is not useful if you're too dumb or too smart. It just seems both tautological and strange.

My communication skills are fine. ChatGPT is just not useful for me, period. I've tried taking what people have mentioned here, trying to have a so-called dialogue with it when things it suggests, for much more simple things, aren't quite correct. It keeps suggesting all these things, but it just feels like an interactive StackOverflow, except all the suggestions are wrong or don't work.

Re:Ask HN: Is anyone else bearish on OpenAI? by avion23 on Tuesday November 14, 2023 @08:51AM (#38263252)
If you really wanted to improve you could show a link to a discussion, then I can give you feedback.

I am switching to teaching / consulting in AI. Chatbots are a part of that. I've reviewed some chatlogs, and some people get abyssimal results, which are entirely explainable and are 100% their fault.

Re:Ask HN: Is anyone else bearish on OpenAI? by LouisSayers on Monday November 13, 2023 @02:04PM (#38253839)
> Keep in mind that ChatGPT has been seriously and intentionally downgraded since March

I found it getting worse for a bit but lately it's been really good. I would say it's being constantly updated, not necessarily getting worse.

Re:Ask HN: Is anyone else bearish on OpenAI? by avion23 on Tuesday November 14, 2023 @08:55AM (#38263300)
It can't solve problems it was able to solve before.

It's not able to output complete code.

It always adds comments, even when I ask it not to.

It doesn't follow instructions in general.

In complex issues it has less "memory", that is it's not able to observe multiple constraints at the same time. The limit was much higher before.

Re:Ask HN: Is anyone else bearish on OpenAI? by LouisSayers on Tuesday November 14, 2023 @02:00PM (#38267848)
I just got it to write some code without any comments.

Sometimes you have to ask it in very specific ways to achieve what you're wanting - it is definitely limited in some areas though.

https://chat.openai.com/share/64dc6384-1fc4-4506-9280-98dc8a...

Re:Ask HN: Is anyone else bearish on OpenAI? by avion23 on Wednesday November 22, 2023 @08:38AM (#38378936)
Thanks I'll try it out
Re:Ask HN: Is anyone else bearish on OpenAI? by aiquestions on Monday November 13, 2023 @06:11PM (#38256631)
Can you paste a transcript for any of those examples here? I’m genuinely curious. I’ve never seen a single actual transcript from anyone talking about how they use ChatGPT all the time for programming work and it’s great.
Re:Ask HN: Is anyone else bearish on OpenAI? by avion23 on Tuesday November 14, 2023 @08:58AM (#38263335)
Sorry I don't have anything un-confidential right now.

Just take any code which is < 3 pages long. Ask it to optimize a function. Or review the code. It should give good results

EDIT: I had some deprecation warnings in python recently, which it quickly fixed. Googled that on a different computer and found https://stackoverflow.com/questions/76689364/how-to-fix-depr...

This is the transcript. Forgive my dumb last question, I didn't understand the code, what it did worked well.

---

User: Please fix the deprecation warning on this code: for i in range(0,3): df.loc[df['date'] < "2023-06-01", 'date'] = df['date'] + pd.DateOffset(years=1)

ChatGPT: The code you've provided updates the 'date' column in a DataFrame df by adding one year to each date that is earlier than June 1, 2023. However, this operation is performed multiple times due to the loop, which is likely not intended and could be the source of inefficiency or unexpected results.

To fix this, you should update the dates in a vectorized way without using a loop. Here's the corrected code:

  import pandas as pd
  # Sample DataFrame creation for demonstration (you should use your own DataFrame 'df')
  # df = pd.DataFrame({'date': pd.to_datetime(['2022-05-01', '2023-07-01', '2024-08-01'])})

  # Correcting the date manipulation
  df.loc[df['date'] < "2023-06-01", 'date'] = df.loc[df['date'] < "2023-06-01", 'date'] + pd.DateOffset(years=1)
  *This code will effectively update all the dates that are earlier than June 1, 2023, by adding one year to them, and it does this in a single pass without a loop. This should also avoid any deprecation warnings and improve the efficiency of your code.*
User: give me the code in one code block. first the old version, then the improved version. Make sure it's executable on its own

ChatGPT: Certainly! Below is the code block with both the original version and the improved version. The improved version eliminates the unnecessary loop and applies the date modification in a more efficient manner.

  import pandas as pd

  # Original code
  df = pd.DataFrame({'date': pd.to_datetime(['2022-05-01', '2023-07-01', '2024-08-01'])})
  for i in range(0,3):
      df.loc[df['date'] < "2023-06-01", 'date'] = df['date'] + pd.DateOffset(years=1)

  # Print the result of the original code
  print("Original Code Result:")
  print(df)

  # Improved code
  df = pd.DataFrame({'date': pd.to_datetime(['2022-05-01', '2023-07-01', '2024-08-01'])})
  df.loc[df['date'] < "2023-06-01", 'date'] = df.loc[df['date'] < "2023-06-01", 'date'] + pd.DateOffset(years=1)

  # Print the result of the improved code
  print("\nImproved Code Result:")
  print(df)
  *This code first runs the original version, prints the result, then runs the improved version, and prints that result for comparison. Note that the DataFrame df is reinitialized before running the improved version to ensure a fair comparison.*

User: is it really the same?

Code: >>> import pandas as pd >>> # Original code >>> df = pd.DataFrame({'date': pd.to_datetime(['2022-05-01', '2023-07-01', '2024-08-01'])}) >>> for i in range(0,3): ... df.loc[df['date'] < "2023-06-01", 'date'] = df['date'] + pd.DateOffset(years=1) ... >>> # Print the result of the original code >>> print("Original Code Result:") Original Code Result: >>> print(df) date 0 2024-05-01 1 2023-07-01 2 2024-08-01 >>> >>> # Improved code >>> df = pd.DataFrame({'date': pd.to_datetime(['2022-05-01', '2023-07-01', '2024-08-01'])}) >>> df.loc[df['date'] < "2023-06-01", 'date'] = df.loc[df['date'] < "2023-06-01", 'date'] + pd.DateOffset(years=1) >>> >>> # Print the result of the improved code >>> print("\nImproved Code Result:") Improved Code Result: >>> print(df) date 0 2023-05-01 1 2023-07-01 2 2024-08-01

ChatGPT: You are correct. The two code versions do not produce the same result. The original code with the loop increments the date by one year for each iteration of the loop, but only for dates before "2023-06-01". This results in multiple increments for dates that are initially before this threshold.

In contrast, the improved code without the loop increments the date by one year only once, making it more efficient and potentially more aligned with what one might expect logically. However, if the intent of the original code was indeed to increment certain dates multiple times, the "improved" version would not be equivalent.

If the goal is to replicate the logic of the original code but without the deprecation warning and with improved efficiency, we need to understand the exact behavior intended with the loop. If the intent was to increment dates before "2023-06-01" by multiple years based on some condition, the improved version should be adjusted accordingly. ---

---

Re:Ask HN: Is anyone else bearish on OpenAI? by aiquestions on Tuesday November 14, 2023 @10:21PM (#38272915)
Thanks! I’ll give this interaction a try.
Re:Ask HN: Is anyone else bearish on OpenAI? by gaganyaan on Friday November 10, 2023 @11:30PM (#38227641)
Are you using GPT-3.5 or GPT-4? A hallucination like FableGL sounds like 3.5, which is not something you should base your expectations of the tech on.

Also, although I haven't seen it before, the Warren Abstract Machine seems like too big of a job for GPT-4. It excels at smaller tasks such as "Convert this C code to idiomatic Rust", or "Write a SQL query that grabs records from X, joins with Y, filters by Z". You might need to make small adjustments manually or by saying "Rewrite it with this in mind".

One really neat trick it did recently is that I uploaded a chart png I found on the internet, and asked it to write Seaborn code that matched the chart's style. It took ~3 follow-up tweak prompts, but then spat out code that worked. It also handles things like "How do I adjust this label that's positioned in chart coordinates over by 10 pixels?", which is an absolute pain to figure out from the docs.

Especially since you're getting answers noting the complexity, I think you're just asking too much of it for now. Try smaller tasks for now, and wait a while to try the big ones again.

Re:Ask HN: Is anyone else bearish on OpenAI? by xigoi on Monday November 13, 2023 @03:57PM (#38255282)
> A hallucination like FableGL sounds like 3.5, which is not something you should base your expectations of the tech on.

So in order to properly evaluate a service, we need to already be subscribed to it in the first place?

Re:Ask HN: Is anyone else bearish on OpenAI? by falleng0d on Monday November 13, 2023 @05:12PM (#38256092)
Yes. Or just use the api and pay for usage (but in that case It could also get more complicated since you have full control)
Re:Ask HN: Is anyone else bearish on OpenAI? by gloosx on Monday November 13, 2023 @04:02PM (#38255339)
So funny SQL query example, do you realize you could describe your exact query to the engine itself in the more precise and short terms? Writing "select X from Y where Z join XX" seems much more practical and rational than using this whole sentence "Write a SQL query that grabs records from X, joins with Y, filters by Z" to achieve the same.
Re:Ask HN: Is anyone else bearish on OpenAI? by wenc on Saturday November 11, 2023 @02:41AM (#38228343)
I'm using ChatGPT 4 (paid) and it has solved a few problems for me. (it sounds like you're using the free 3.5 and also having some pretty unrealistic expectations of ChatGPT like asking for a full implementation of a WAM)

A mundane one: I have a list of lat-longs mapped to region codes, and I needed write a query to find the area of the convex hull of each region code. I knew how to do it in code, but I wanted a SQL answer. ChatGPT gave me the correct Redshift SQL.

A more involved one: I needed to maximize match x people y groups, where people can rank their choices up to 3, but as soon as they accept the match, they drop out of the pool. Plus a bunch of different constraints. Sounds like a stable marriage problem. ChatGPT proposed the Gale-Shapley algorithm, which was the exact one I was looking for.

Re:Ask HN: Is anyone else bearish on OpenAI? by slau on Saturday November 11, 2023 @02:50AM (#38228384)
One of my clients has their own instance of ChatGPT running. It is running on dedicated hardware, is trained on different data, and we can change the core prompts with relative ease. We even get to tune some temperature coefficient, which adjusts whether the LLM behaves in a creative or factual way.

This is because the client in question is in a regulated industry.

This week, we used it a bunch of times to help rephrase objectives while working in brainstorm sessions, allowing it absorb ridiculously large regulatory PDFs to give us summaries and ideas of what our greenfield project should focus on.

Re:Ask HN: Is anyone else bearish on OpenAI? by pinkgolem on Saturday November 11, 2023 @08:14PM (#38236296)
Be careful, not sure what you are self hosting there but the last paragraph sounds like a good way to start working on a task the LLM imagined
Re:Ask HN: Is anyone else bearish on OpenAI? by nuancebydefault on Saturday November 11, 2023 @04:43PM (#38234852)
I'm surprised that nobody on HN seems to point to or mention bing chat. It is based on GPT, it is much better than free GPT 3.5 and it is free as well. I use it plenty of times and more than 90 percent of the time answers are exact and code just works as is. You just ask it extra questions about how it works and it explains it in as much detail as you possibly need, until you fully understand. Then you ask it to rewrite it this or that way and you try it out and add features step by step. Works like a charm.
Re:Ask HN: Is anyone else bearish on OpenAI? by GianFabien on Saturday November 11, 2023 @09:57PM (#38236820)
Until your comment, I had been using Chat GPT 3.5. Not only is Bing Chat more accurate (GPT 4 based) but it also provides footnotes to allow you to reference the sources for more specific information.

The code snippets tend to be bloated, but no worse than an intern would produce. A quick refactor, adding edge case handling and the code is ready to test and commit.

Re:Ask HN: Is anyone else bearish on OpenAI? by seydor on Tuesday November 14, 2023 @11:27AM (#38265450)
You can also upload pdfs to it
Re:Ask HN: Is anyone else bearish on OpenAI? by sigbottle on Saturday November 11, 2023 @05:37PM (#38235298)
Well, the things you're working on don't sound like things that I'd feed to chatGPT. But if I need to look up some language feature, or write some basic boilerplate code, for example, it's a big time saver. If it's something that will take mental energy to generate, but I can verify in a few seconds before adding to my project, it's good. I wouldn't ask it to code any algorithms for example, but syntax/language feature stuff and API gluing it can pattern match pretty well.

TL;DR don't ask it anything to do with logic, but anything to do with documentation and stuff like that it's pretty good

Re:Ask HN: Is anyone else bearish on OpenAI? by paulcole on Saturday November 11, 2023 @06:28PM (#38235656)
I don’t need it to solve problems where I’m completely stuck for it to be worth $20 a month.

Things I’ve gotten value out of in the past week or two:

• Writing a job description

• Making a python script to automate some stuff in Asana

• Simplifying some management concepts so I could slack them to a coworker

All of these are things I could easily do myself. But with ChatGPT, they’re done 75% as well in 10% of the time and I don’t have to think hardly at all.

Re:Ask HN: Is anyone else bearish on OpenAI? by fullsend on Monday November 13, 2023 @03:29PM (#38254956)
This is the sort of stuff I use it for all the time, great examples. Or summarizing new concepts, reformatting strings or tables quickly, asking what some strange code snippet might be doing.
Re:Ask HN: Is anyone else bearish on OpenAI? by bastawhiz on Sunday November 12, 2023 @01:08AM (#38237752)
Not that I'm trying to change your mind, but some specific things that LLMs do for me:

- Helping to refactor SQL

- writing jq commands (I simply cannot)

- writing shell code (it happens just infrequently enough that I can't justify spending the time to get good)

- brainstorming names or puns. Word association is easier with a second person (or an LLM)

- figuring out why my AWS CLI commands aren't doing what I'm expecting them to be doing

- asking for reasons why a piece of code wouldn't work

I can competently do all of these things on my own if I havw to, but now I don't have to do them on my own so my life is easier.

Re:Ask HN: Is anyone else bearish on OpenAI? by badpun on Sunday November 12, 2023 @05:44AM (#38239096)
This is just helping out with trivial tasks, which sounds very much in scope of LLMs. The OP was talking about replacing "world class consultants", which I'm very sceptical of.
Re:Ask HN: Is anyone else bearish on OpenAI? by rmilejczz on Sunday November 12, 2023 @10:57AM (#38241272)
I use it almost exclusively to help me approach novel problems, or at least problem spaces that I am severely lacking expertise in. It’s pretty incredible for helping me leverage my extant programming knowledge to accomplish things that I couldn’t before due to being blocked by research.

I’ve also used it quite a lot for advice as I was learning project management and moving away from individual contributions.

So I think as a consultant it has value, though I think “world class” might be a bit hyperbolic, since I still need to validate its output.

Re:Ask HN: Is anyone else bearish on OpenAI? by bmitc on Monday November 13, 2023 @12:46AM (#38247377)
> I use it almost exclusively to help me approach novel problems, or at least problem spaces that I am severely lacking expertise in. It’s pretty incredible for helping me leverage my extant programming knowledge to accomplish things that I couldn’t before due to being blocked by research.

Can you expand on this? This is what I'd like to use it for, but I don't ever have any experience remotely close to helpful. The typical response is just something like "that's complicated but here's some gibberish that tries to show a simple example but is still wrong in ways it's not useful at all".

Do you have a specific example?

Re:Ask HN: Is anyone else bearish on OpenAI? by bastawhiz on Sunday November 12, 2023 @02:49PM (#38243490)
How many people actually hire consultants, let alone "world class" consultants? Especially on a regular basis. The value delivered by solving problems you encounter daily (hourly?) will outstrip the value you get from problems you need to solve every year or two (or more).

I get a half decent expert to solve problems I don't have the patience or experience to solve myself. That's literally the definition of what a consultant does. What you call "trivial" is what lots of other people describe what consultants largely do.

Re:Ask HN: Is anyone else bearish on OpenAI? by avion23 on Tuesday November 14, 2023 @09:00AM (#38263363)
I've had it craft legal letters in German. Very good content
Re:Ask HN: Is anyone else bearish on OpenAI? by tomjen3 on Sunday November 12, 2023 @03:31AM (#38238438)
I prompted it with “what is wrong with this code” and my broken sql and it told me exactly what was wrong.

I could have looked up the individual parts and found the problems myself, but then I would have spent at least 15 minutes on it instead of 30 seconds.

Re:Ask HN: Is anyone else bearish on OpenAI? by usgroup on Sunday November 12, 2023 @04:18AM (#38238689)
My experience is much the same, far from "I've essentially got 15-20 high-priced world-class consultants in every field that I chose to pull from, working at my beck and call..." that OP mentions.

I suspect it is because I tend to have off piste questions which are not recipes or high-level ideas. It is where it'd be most useful to me, but it is at the same time where least training data is going to be.

It is also where actual legitimate experts are most useful: the special sauce, not the meat and potatoes.

Re:Ask HN: Is anyone else bearish on OpenAI? by razodactyl on Sunday November 12, 2023 @06:04AM (#38239194)
Yeah I've been down the exact same paths as you. You're spot on, it's a statistical merge and "RAG" is literally "filter the probabilities and change the odds".

The main argument is that you can give it a block of real-world data like an email or code and take advantage of collective knowledge to identify outliers like bugs, bad grammar or incoherent writing which translates exactly to code semantics too.

Re:Ask HN: Is anyone else bearish on OpenAI? by jerpint on Sunday November 12, 2023 @06:49AM (#38239393)
I had to implement a relatively simple auth flow recently for an app (something I haven’t done in the past). Gpt3.5 struggled to give me anything useful other than high level ideas but GPT4 gave me the exact boilerplate I needed and unblocked me.

I had searched all over the web for an example addressing my specific use case on google and couldn’t find one. GPT4 produced a working example for me and got me past that road block. I also use it regularly to suggest better coding patterns and I find it does a really good job at doing code reviews for obvious mistakes / anti-patterns

Re:Ask HN: Is anyone else bearish on OpenAI? by mettamage on Sunday November 12, 2023 @07:02AM (#38239443)
Yea, I have a very similar experience. ChatGPT unblocks me quickly in conceptual understanding when it comes to coding. My programming knowledge is pretty good, but in certain areas I had a hard time understanding something and just skipped it. Oauth is an example of this. ChatGPT unblocked me there as well and explained the theory behind it both in the abstract case and the case pertaining to my situation. So now it's much more clear.

But my mind kind of works like: once I'm stuck I endlessly do my best to understand it, which sometimes mean that I grind to a halt. ChatGPT stops that entirely.

The fact that it always produces 2 to 3 bugs is fine.

Re:Ask HN: Is anyone else bearish on OpenAI? by jorl17 on Sunday November 12, 2023 @07:29AM (#38239553)
Not OP. Also, I generally detest hype culture and hated the Web3/Crypto bubble. I'm a SW who works as the CTO of a small startup studio, so I end up dealing with a lot of everything -- from coding, to managing and hiring people, to working on proposals and finding clients, you name it.

I use ChatGPT 4, GPT-4, and Copilot every day. It is an "average intern" at many many things. Here's how I feel it helps me:

* Its interactivity lets me learn a lot (superficially) about new topics. I can then expand that with research of my own

* It helps me think outside the box or from other perspectives when thinking about e-mails, proposals, real-world scenarios

* When exploring a new language, framework or technology, it points me in the right direction.

* For quick scripts, using the code generation/analysis feature, if I direct it right (i.e. layout "the plan" beforehand and ask it to work the rest on its own), it gets a lot of it right pretty fast, saving me some time writing the code, figuring out the right libraries and the nitty-gritty details.

* It is great at giving ideas for why something might not be working.

Real things I've done with it:

* Discuss ongoing negotiations with clients, trying to better my proposal and better understand the clients point of view.

* Learn more about managerial or "business-y" topics, by allowing me to discuss things with it and iterate on that with my own research. It is a valuable "white board" to discuss with.

* Adjust my e-mails so they are more appropriate to the situation. This can involve changing the tone, shortening them, adding more detail, etc.

* In general, i've used it to find flaws in reasoning when dealing with people. For example, it has helped me question my own client proposals or approaches, by specifying where I was lacking (e.g. because I was vague, or pessimistic, didn't give a mensurable objective, seemed to ignore a client's request, etc)

* I use a command-line utility from the shell which lets me ask it to do something and then have it do it. I now use this with some frequency to just write the commands I would have to google because I haven't memorized. Things like ffmpeg or imagemagick arguments. Or combinations of grep, sed, ls, find, git, etc. Here are some examples:

i) "merge jpgs in this folder in order onto a pdf. each jpg should fill the page 100% without margin. Avoid complex calculations".

ii) "zip this folder with maximum compression and password 12345678'".

iii) "git remove branches matching pattern sprint-* remotely'"

iv) "use imagemagick to convert to jpg lossless and rotate 90 deg clockwise ~/home-en.png"

v) "Add _en before .jpg in all files in current directory. So they become _en.jpg"

vi) The list goes on and on...

* It has helped me cleanup nginx config files

* I have thrown code at it which I suspect has a bug. With a bit of context and some back and forth, it has helped me find it.

* In languages or frameworks I don't use often, it really shines. It has helped me write several applescript scripts which I have also cobbled together to create Alfred workflows. If I need to code something in a language I don't often use, what it produces is good enough for me to iterate on.

* It has helped people at our company improve their copywriting (when used with a lot of care)

* I have used it to help me critique my own poetry and improve how I approach poetry in general. Highly subjective, I know

* When trying to figure out how to use apps I don't often use, or dealing with unexpected behaviour in them, it often helps me find the issue. Notable apps in this category include Photoshop and Excel.

* I don't often do frontend so I'm particularly bad at styling and organizing things. When I ocasionally have to do frontend, it often gives me the initial "skeleton" very well.

I have seen many people try to use these tools, and here's where I think they SHOULD NOT use them:

* For facts, obviously -- which is unfortunately what many people actually try to use it for

* For writing (without check) most of your e-mails our posts, especially with very little context

* For ideas where you need "just enough creativity". It's a very fine line. Think brainstorming ideas for UI elements in a new website according to specific brand guidelines

* For incredibly specific details or overly complex tasks. You can't often have it write your 200 line function out of thin-air!

It is clear that GPT provides me with:

1. A better search engine, which skips past loads of outdated content or SEO-laden bullshit. It's just a shame that I can only use it this way for more specific "creative" or "problem-solving questions" questions. For important fact-based info, I always have to check what it says.

2. A partner with whom to discuss ideas and iterate over them, especially on topics I don't know. If anything, it's a great rubber-duck

3. A way to forget the underpinning of some of what I do and approach it with natural language. I find myself asking it to write my bash sequences instead of thinking them up myself. Of course I check them and 99.9% understand them, but it's just so much easier to have it do it for me.

4. (Copilot is an upgrade of my IDE. And a great one!)

If modern Google (and Bing and everyone else?) weren't so shit, I wouldn't need 1. To this day I still first go to google, eventually give up because it doesn't answer me properly and then go to GPT. It's ridiculous. I think 1. brings a lot of value to many many people. Google is so absolutely shit nowadays I can't believe it -- and the more I use GPT to get a comparison baseline, the more I am shocked. 2., 3. and 4. are incredible additions to my workflow.

I truly believe we are heading in the direction of interacting with computers via a new interface. Even writing code might eventually be affected. Perhaps, like we have built high-level languages on top lower-level languages and machine code, abstraction over abstraction, we might end up with some form of language writing which uses (very well controlled) LLM technology under the hood. Instead of directly writing the code, we tell it to write it. We still need to be smart to pick the right algorithms and structures, but no longer have to worry so much about writing the nitty-gritty syntax or details. Maybe?

Using GPT is its own art. Crafting the right prompts and getting a feel for how it works is very important. It is also essential to have critical-thinking and know when to question what it says and when not to.

A friend of mine told me this: "The companies who strive on hiring loads of shit people to do work are the ones who will suffer more. The others, which have hired smart people capable of critical-thinking, will benefit. GPT obliviates the work of many small shit people. Pair GPT with someone who is already smart and has very good problem-solving or critical-thinking skills, and now you've got a team that can obliterate the teams of many small resources.

I don't know if OpenAI will succeed or not, but I do know this technology is absolutely life-changing for many, and going back to a world without it being this acessible will likely be a net-negative.

Re:Ask HN: Is anyone else bearish on OpenAI? by Eggpants on Tuesday November 14, 2023 @09:05AM (#38263405)
> A better search engine, which skips past loads of outdated content or SEO-laden bullshit.

But ChatBLT was ‘trained’ using SEO-maddened bullshit thus your getting the statistical mean of all that bullshit.

The first rule of the internet is don’t believe everything on the internet. Hitching your financial wagon to it seems crazy to me.

Re:Ask HN: Is anyone else bearish on OpenAI? by jorl17 on Tuesday November 14, 2023 @09:19AM (#38263568)
ChatGPT somehow gets me the right answers and google doesn’t. I know they are right because often I’m just looking to remember something, and once I do, it clicks. Something as simple as googling “how todo X in Y”. Somehow google always gives me pages of bullshit or tutorial-like content which doesn’t quite answer my question. But when I ask ChatGPT, often without any more back and forth, it gets it.

How? Maybe because it was trained on lots of data and therefore isn’t just “spitting the first thing”, or perhaps it’s because it’s actually just answering the question I asked, without muddling it in a page with 50 other similar questions designed to up the SEO ranking. Whatever the reason is, it clearly happens often.

I’m not advocating for not fact checking ChatGPT. Quite the opposite. But even with that caveat, it adds tremendous value, at least to me.

Re:Ask HN: Is anyone else bearish on OpenAI? by kubiton on Sunday November 12, 2023 @08:51AM (#38240050)
I used it yesterday to generate a basic layout with image and input field based on tailwindcss.

Would have taken me much more time to figure it out than what chatgpt did at first try.

Re:Ask HN: Is anyone else bearish on OpenAI? by lucideer on Sunday November 12, 2023 @04:34PM (#38244295)
> Are you able to give a specific example of a problem it helped you solve? Especially one that you were at a complete blocking point...

I think it's worth stepping back here and re-examining the hurdles your setting for your own understanding.

The essential question here is: is ChatGPT useful to people. What you seem to be implying with your question is: I will not use ChatGPT unless it can solve problems with X level of difficulty for me. Why have you set that pre-requisite? Would it not still be useful to you if it simply increased the efficiency of solving simple day-to-day tasks that you're not blocked on?

Re:Ask HN: Is anyone else bearish on OpenAI? by tdeck on Sunday November 12, 2023 @08:19PM (#38245844)
I'm not the OP but I had a similar set of experiences with ChatGPT so I find the discussion useful. Essentially I already know how to find what I need using a search engine, and I know how to use cues from the source material to judge if it's correct. When I try using ChatGPT for technical problems I face, I frequently get hallucinated answers that feel like a waste of time.

Then I go on HN and read these paeans from other technologists who say ChatGPT has completely changed how they work and is 10x better than using Google. I'd like to have that too! I just don't get it, it doesn't match my experience at all. And yes I did try the paid ChatGPT model for a month.

Re:Ask HN: Is anyone else bearish on OpenAI? by bmitc on Monday November 13, 2023 @12:42AM (#38247348)
> Then I go on HN and read these paeans from other technologists who say ChatGPT has completely changed how they work and is 10x better than using Google. I'd like to have that too! I just don't get it, it doesn't match my experience at all.

This is exactly my experience and what I am trying to get at. I start feeling that maybe I'm missing out, so I go give it a try. My direct experience shows that it spits out gibberish, but then people say things like it's a godsend but don't give any examples besides maybe it generated some HTTP API requests for them.

Re:Ask HN: Is anyone else bearish on OpenAI? by avbjdkleosm698 on Monday November 13, 2023 @02:25AM (#38247824)
I think there are really people that have jobs where they mostly write the same http request code all day everyday and then go home. Especially in the web space and startup world. ChatGPT isn't optimized for us, it was designed for the JavaScript tutorial audience that somehow have remained gainfully employed.
Re:Ask HN: Is anyone else bearish on OpenAI? by Kim_Bruning on Monday November 13, 2023 @09:01AM (#38250186)
Example then:

GPT+ data analysis gives GPT4 access to a jupyter back-end.

I was dealing with a noisy sensor in a factory production line.

I asked GPT to remind me which averaging functions might be usable to smooth out the noise, within the constraint of a PLC with limited memory.

I got GPT to simulate the output from the noisy sensor, apply each averaging function, and supply graphs of inputs and outputs. It was then pretty easy to eyeball which function was most suitable for the task.

I then gave GPT a code-style example and asked it to provide an (IEC 61131-3) Structured Text implementation using that code style. Which it did. This turned out to be pretty close to the final implementation (after careful reading and testing.)

Because it's so cheap to do (time and money wise), I used GPT to generate quite a lot of throw-away code to get to the final result. I probably wouldn't have considered this particular approach if doing it by hand.

Re:Ask HN: Is anyone else bearish on OpenAI? by bmitc on Monday November 13, 2023 @12:37AM (#38247322)
Well I have yet to have been provided a specific example that matches the hyperbolic descriptions people are throwing around. I simply tried to find my own examples, and as is my experience with ChatGPT, it is nearly completely worthless. And then I get told that I'm using it wrong, asking it too niche of questions, and to reset my expectations.

I'm still waiting for the concrete examples that back up the above commentary about how much of a gamechanger it is.

> I've essentially got 15-20 high-priced world-class consultants in every field that I chose to pull from, working at my beck and call

In other words, this comment from above needs backing up. I've seen such descriptions everywhere but no one has ever provided examples that corroborate this.

Re:Ask HN: Is anyone else bearish on OpenAI? by moralestapia on Monday November 13, 2023 @12:58AM (#38247441)
Agree with you, why hasn't someone show a useful prompt or an example of it in more than 8 hours ¯\_(ツ)_/¯.

For me, personally, it has replaced Google Search for the type of query that looks like initial research on a specific subject. "What is X?" then ask a couple follow up questions and I can quickly get a rough idea of some subject that I happen to need at the moment.

Re:Ask HN: Is anyone else bearish on OpenAI? by stevesearer on Monday November 13, 2023 @01:06AM (#38247475)
I am decent at writing PHP/WordPress with a basic working knowledge of some js/jQuery.

I wanted to use my GPX files to generate my own private hiking/running log.

With basically zero knowledge specific of GPX files, D3.js, or mapbox before starting I was able to quickly write a plugin that displays a path on a mapbox map as well as generates an elevation profile.

It would have taken me so long to do this before by googling and reading documentation with trial and error I would have stopped the project.

Now I have a working system.

Re:Ask HN: Is anyone else bearish on OpenAI? by lucideer on Monday November 13, 2023 @05:28AM (#38248786)
I have yet to have been provided a specific example that matches the hyperbolic descriptions people are throwing around.

> > I've essentially got 15-20 high-priced world-class consultants in every field that I chose to pull from, working at my beck and call

I'll give you that: many of the claims are hyperbolic & ridiculous, including that particular one by the gp. The only way I can think of that statement being remotely true is if "high-priced" has no bearing on the quality of work of those consultants (though tbh that is often the case in reality).

Personally I have not found ChatGPT particularly good at anything I need to do. I have however found it very passable at many things I don't enjoy doing. It's reporting boilerplate is far more tailored & novel than anything I could ever put into a reusable reporting template. I dread writing reports: it's not a technical challenge for me but I gain no stimulus from it, so reducing the task to one where I just have to heavily edit something ChatGPT prepared for me is frankly incredible. The same goes for formal cold-call emails to people I don't know: another menial, unpleasant but necessary task.

It's also great at spitballing, and - as others have mentioned - quick shallow topic summaries of things you'd normally rely on a quick google & scan to get a quick topic summary (timely as Google gets worse and worse).

These are microgains throughout your day, nothing truly revolutionary, but that's the case for many successful tech we take for granted.

Re:Ask HN: Is anyone else bearish on OpenAI? by mliker on Sunday November 12, 2023 @05:14PM (#38244639)
With chatgpt I’ve been able to generate math 2d and 3d animations in just 1-2hr with no background nor prior knowledge.

I was also able to use it to generate software to check the blockchain to gather data I needed, again with no experience using the blockchain client libraries

Re:Ask HN: Is anyone else bearish on OpenAI? by bmitc on Monday November 13, 2023 @12:48AM (#38247382)
> With chatgpt I’ve been able to generate math 2d and 3d animations in just 1-2hr with no background nor prior knowledge.

Can you give some example prompts that will reproduce such an experience?

Re:Ask HN: Is anyone else bearish on OpenAI? by kdwikzncba on Monday November 13, 2023 @03:50AM (#38248287)
If you look at his profile and history, it seems what's he does is mostly to parse jsons and generate config files.
Re:Ask HN: Is anyone else bearish on OpenAI? by chrisallenlane on Monday November 13, 2023 @08:16AM (#38249771)
In my experience, it's more helpful in some areas than others.

This year I used GPT-4 to write a significant amount of Terraform that was necessary to migrate an application onto AWS.

Writing Terraform, in my opinion, is problem that's broad but shallow. GPT-4 needed to do little beyond summarize documentation, but it was able to do so competently, and that was hugely valuable to me.

Conversely: In my free time, I've attempted to use it for a game-development side-project, and very little of its output has been useful at all.

Re:Ask HN: Is anyone else bearish on OpenAI? by superhuzza on Monday November 13, 2023 @11:13AM (#38251866)
I feed ChatGPT 3.5 answers from surveys and it summarizes and categorizes the answers for me. I have a quick read through as well to be sure, but I've been very impressed so far. It's a massive timesaver when doing a lot of research.

I can also feed it a list of web accessibility issues and have it sort the list into most/least critical, along with providing references to the specific WCAG criteria. It has occasionally stumbled on this task, but again massive timesavings.

I also use ChatGPT for filler text in designs. Yes I have to build on what it writes, but it's way better and quicker than what I'd do myself. I know our communications team is doing this x10 more than I am.

Re:Ask HN: Is anyone else bearish on OpenAI? by gilthans on Tuesday November 14, 2023 @05:08AM (#38261348)
I can give two personal examples:

1) I needed to display a very high resolution image to a user. I have experience in GIS and imagery, so I knew I should use an image pyramid in some way, and from previous experience assumed I needed some server to cleverly serve the tiles. But I didn't want to implement it myself, and googling 'map server' led to rabbit holes. I consulted with chatGPT, and while it gave me several fake solutions, eventually it suggested using gdal2tiles.py to create the pyramid and then serve it directly with a CDN. This never occurred to me, and is a much better fit to this problem. This saved a LOT of time (on either building my own server or fudging with other solutions) 2) I have a streamlit service, and needed to use some of my infra inside, infra that was written using async-await. Unfortunately, streamlit and asyncio don't play nice together (boo streamlit). I went to chatGPT hoping it would find me a way to make it work anyway, and after trying everything it suggested (which failed), I tried googling myself, and spent several days without a solution. Eventually, I went back to chatGPT and it suggested building a small HTTP service that would serve the results, and access it in streamlit using requests, no async-await required. It's a hacky solution, but significantly faster than reimplementing my streamlit dashboard in another framework or rewriting my infra without async-await. It saved loads of time.

I think you'd agree these aren't junior level issues. ChatGPT definitely didn't solve every problem I came to it with, probably not even most, and even when it did I had to intervene signficantly. I feel the more experienced you are as a developer, the less valuable it is. But when you need to tinker in a field you're not proficient at, or if you need to brainstorm a solution to a tricky problem, it can be a great tool. I understand why many swear by it. It takes a while to learn what sort of issues are good to turn to chatGPT with and which aren't, and also how to phrase those issues