I never forget one of his remarks: There can be only one thing that is worse than someone not having a mother - that he has one.

So maybe a chatty LLM is not the worse thing that can happen with someone.

> There can be only one thing that is worse than someone not having a mother - that he has one.

However, this is not accurate.

I mean doctors and play therapists still have to do their job, We have interviewed some doctors who feel particularly frustrated about how to comfort children before tests or surgeries. They hope for a tool can help building comfort for kids -> which means time is faster to run tests.

I can only urge you to reconsider how honest, realistic, and credible those promises you make can possibly be. After all, you are playing with the lives and wellbeing of humans here. Every drug and therapeutic device has to go through rigorous vetting and testing before being cleared for human treatment. Ever heard of clinical trials? And you seriously think you can skip that with “we asked some pediatricians”? Please, think again. And ask someone with more domain knowledge than vague hopes in a technology they don't understand.

Junru and I will discuss the approach with pediatric care based on this. And I agree having domain expertise/advisors to guide us in the right direction is important

You're asking us to trust you, but why should we trust you in this matter? Regardless of if I think ChatGPT is any good at those things, you'd need some supporting evidence for that one way or another before continuing.

Why is that "not anywhere close to being able to substitute as an effective parent, therapist, or caregiver."?

Maybe I've had a bad parents/therapists/caregivers all my life, but it seems like an entirely reasonable response. If there's a more specific scenario you'd like me to pose and show me that it's advice is no good, I'm happy to ask it.

I'm not trying to convince you that it's, right now, a replacement for a human parent/therapist/caregiver. it's the "not anywhere close" part that I'm responding to. It's closer than talking with a speak and spell, or a See'n'say, for instance, but also ahead of static worksheets that you can't have a conversation with. I have no idea if this is good for society, and I have no idea where this technology will take us.

I want to know the limitations of this technology, and I'm willing to be convinced that, hey, maybe what some of it's saying isn't helpful as a therapist, because that's interesting. The number of R's in strawberry, for instance has a specific technical reason it's bad at, because of how tokenization works. If, after being fed every psychology textbook, the advice it gives would be egregiously or subtly bad/wrong/harmful, or biased towards, say, giving a Freudian analysis when the industry's moved way past that, I'd like to know and hear about it, so I better know when not to trust its advice and be able to warn others.

My instincts tell me that humans are shockingly good at detecting this difference. And when they aren't - they still won't like being lied to or tricked about it. You can see it already - generative art, or music for instance is (in some cases) objectively more impressive than art created by humans all else constant. You might trick a contest into giving you an award but the moment people find out it's generated, they almost immediately react cynically and no longer express interest in the result.

That's because they used to attribute the result to a person and now they know it's not a person. The psychology there probably isn't even fully fleshed out, but i feel it instinctively, as I said before. And I suspect others do as well based on the reactions here.

Sorry for assuming bad faith. i've met a lot of persons here who really do think LLM's in their current form are a kind of sentience. Blake Lemoigne (sp?) is a good example of that kind of naïveté.

I too have a human therapist, doctors, etc. And I too find myself chatting with ChatGPT, etc. about personal issues and in certain cases benefit tremendously from it. In particular, whenever it is something I would normally feel embarassed to say to another actual human. Since I am very confident ChatGPT doesn't have feelings or even an internal monologue with which it could "judge" me - I have no issue telling it such things.

But for precisely that same reason, it will never work (in its current form) as a _proper_ substitute for a human. Again, I'll just say that I don't think there's yet enough research on this but that "I know it when I see it". Any sufficiently serious topic I discuss with ChatGPT ultimately winds up with me drained because I feel as though I'm talking to a wall and not actually being acknowledged by anyone with agency who matters to me.

Unfortunately, I'm not sure what your point actually is. Is ChatGPT in it's current form, a replacement for human contact? absolutely not. do people have strong emotions around something using a GPU and a bunch of math and was generated instead of being organically hand crafted by a human being, and having it fall into the uncanny valley? totally. is this box of matrices and dot products outputting something I personally find useful, despite shortcomings? yeah.

I agree that there's totally this brick wall feeling when ChatGPT spins itself in circles because it ran out the context window or whatever.

at the end of the day, I think the yacht rock cover of "closer" is fun, even though it's AI generated. however that makes you feel about my opinions.

https://youtu.be/ejubTfUhK9c

It won't literally do that, the labs are all careful about the obvious stuff.

But consider that Google Gemini's bad instructions almost gave someone botulism*, there's a high chance of something like that in almost every field. I couldn't tell you what that would mean in therapy for the same reason I wouldn't have known Gemini's recipe would lead to culturing botulism.

These are certainly more capable than anything before them, but the Peter Principle still applies, we should treat them as no more than interns for now. That may be OK, may even be an improvement on not having them, but it's easy to misjudge them.

* https://news.ycombinator.com/item?id=40724283

I won't be surprised if in a couple more years this kind if thing is the norm. I don't think there's anything inherently different from a person that listens to you.

Humanoid robots are improving, so I won't say "never", but I will say "not yet". Not in isolation at least.

* and likely the closest we ever will, because it was disturbing enough to a big influence on animal welfare movement

OP, I would implore you to not listen to any of this "advice" at all and just keep on building really nice things.

I can already think of a dozen valuable applications of it in a therapheutic context.

Ignore those who don't "do".

I'm actually pretty ok with ignoring those who don't "think" before they "do", not that the OP is one of those people, but "doing" as a mark of virtue seems fairly likely destructive

The world is material, not imaginary.

"Charging head-first", even in the wrong direction, is the only thing worth doing.

Can you share anything you've done so that we can all see it?

I can say

- I have a wonder family that loves me (and vice versa)

- I have a place to live and can pay the bills

- The people I work with are glad I'm working with them (and vice versa)

- I've been a software developer for decades and I get to solve new and (what I find) interesting problems on a fairly regular basis. Most of that problem solving involves thinking; with a minor bit of doing at the end.

So overall, I'm pretty happy with where my life philosophy has taken me.

You're blessed.

Great answer overall, but this bit specifically is so crucial to maintaining interest long-term. Mechanistic programming needs to be done sometimes, and it's every junior's first impulse when presented with what they think has an obvious solution (see the problem -> start coding, or "why don't we just refactor this? I'll just put in a ton of extra unpaid hours and make everything better"), but that wears thin very quickly to the point of risking burnout.

You need to develop perspective on why you do something and what impact it may or may not have on small or large scales, and put yourself in positions where the majority of your labor goes to understanding how to apply your skills or resources in a sufficient way, given the constraints in front of you and down the line; this isn't just to advance your measurable skills or your resume, but to maintain your interest in what can quickly become incredibly dull and soul-sucking.

It does kinda send an interesting message to a child, doesn't it? "You're not worth the time of anybody human, so here's a machine instead."

And that's before the chat even starts (and eventually goes off the rails).

Like trying to learn to swim from a guy who’d drown in a water bottle.

Better a machine than a broken human. By the end of our generation, this overuse of 24 year olds with behavioral problems as diagnosticians will end.

Again, not shitting on the people creating this and this is the forum for it, but I feel all of this is just such a wrong direction for people and humanity in general.