News for hackers. Stuff that matters.
Re:Harvard’s new computer science teacher is a chatbot by userbinator on Sunday June 25, 2023 @12:50AM (#36465218)
Professor Malan said students would be warned of the pitfalls of the AI, saying they should “always think critically” when presented with information.

I want to believe that this will actually happen, but my experience with people blindly trusting what a tool (not even an AI-based one) says, including but not limited to students, suggests that's not going to happen.

Re:Harvard’s new computer science teacher is a chatbot by imchillyb on Sunday June 25, 2023 @01:36AM (#36465373)
> ..."always think critically"...

Y'know, the thing we /human/ were supposed to be teaching you.

Re:Harvard’s new computer science teacher is a chatbot by p-e-w on Sunday June 25, 2023 @01:56AM (#36465439)
That's because authority figures don't actually mean it when they tell people to "always think critically".

What they really mean is "think critically when others tell you something, but accept as gospel everything I am telling you".

Re:Harvard’s new computer science teacher is a chatbot by donkey_oaty on Sunday June 25, 2023 @03:38AM (#36465833)
On my experience most lecturers and professors welcome challenge because it allows them to expand on what they are teaching.
Re:Harvard’s new computer science teacher is a chatbot by p-e-w on Sunday June 25, 2023 @03:59AM (#36465922)
They welcome shallow challenges that allow them to demonstrate their intellectual superiority by correcting them.

They most certainly don't welcome challenges to the basic assumptions on which their teachings rest.

Not that professors are in any way unique in that regard; it is a trait shared by all people with power and authority.

Re:Harvard’s new computer science teacher is a chatbot by misnome on Sunday June 25, 2023 @06:05AM (#36466570)
It sounds like you had a bad experience with a bad professor, or - came in starting with this attitude.

Because this doesn’t match my experience, at all.

Re:Harvard’s new computer science teacher is a chatbot by literalis on Sunday June 25, 2023 @03:26PM (#36471713)
There is just enormous variability with professors. The more MOOCs you take the more obvious this becomes.

If you had good professors in college you should consider yourself lucky.

There is a lot of absolutely terrible professors.

Re:Harvard’s new computer science teacher is a chatbot by michaelt on Sunday June 25, 2023 @06:29AM (#36466724)
Well, there are challenges and there are challenges.

A professor teaching about evolution will answer students' questions within reason - and students with a religious background might have heard some anti-evolution gotchas the professor will be happy to explain - like how something as complex as the eye could evolve.

But that doesn't extend to debating bible verses, allowing so many challenges that it disrupts the class, or changing the exam so you can pass it while denying evolution exists.

Re:Harvard’s new computer science teacher is a chatbot by p-e-w on Sunday June 25, 2023 @08:17AM (#36467420)
That you leap from "basic challenges" to anti-science fundamentalism really shows how deep-rooted this problem is. Most educated people today earnestly believe that the only ones who would disagree on basic issues with the high priests of academia are fanatics, conspiracy nuts, and quacks.

The idea that academics are on the right path has turned from something that must be continuously demonstrated to something that is assumed by default, as part of a new orthodoxy that ironically is almost indistinguishable from the religious and ideological orthodoxies that science once sought to replace.

Re:Harvard’s new computer science teacher is a chatbot by fn-mote on Sunday June 25, 2023 @08:36AM (#36467568)
> The idea that academics are on the right path has turned from something that must be continuously demonstrated to something that is assumed by default [...]

You're talking about a Ph.D. level investigation, not something that is usually considered at the level of an undergraduate in the US.

Most beginners lack the contextual knowledge to recognize even glaring errors. For example, I once saw a distributed application that occasionally updated a central database without any locking. Occasionally the data would get corrupted by simultaneous writes, and the original dev had no idea why.

Consider adding more context if you want to continue to write about dogma in academia. Share your own experence. At the level of generality you are writing it's impossible to engage more because we don't share the same context.

Re:Harvard’s new computer science teacher is a chatbot by michaelt on Sunday June 25, 2023 @09:40AM (#36468084)
I don't know how you arrived at that interpretation of my post.

I chose evolution merely because a substantial and politically important group of Americans earnestly disagree with the academic consensus; it's an area where there has been genuine public debate; it's foundational to some academic fields; and it would believably come up in first year compulsory classes.

I might disagree with academics about whether mailing a survey to mentally competent adults counts as human experimentation that needs ethics board approval, as outside of academia people use surveys all the time. But that's hardly something an academic would refuse to discuss.

I might disagree with academics about feminism or marxism or underwater basket weaving - but that stuff's all elective, why would I have taken an elective module from a teacher I thought was full of shit?

I might disagree with the high tuition costs of universities, and the money wasted on sports, overpaid administrators, and overpriced journal subscriptions. But most academics would actively agree with me, they just can't change it.

I might disagree with details of how a course is taught, like whether Java is a good language for an introductory CS class. Or whether we really need so much math in the CS curriculum. But that's not really a fundamental belief.

I might disagree with academics because I think the moon is made of cheese, but that would be a straw man argument.

Re:Harvard’s new computer science teacher is a chatbot by chongli on Sunday June 25, 2023 @08:15AM (#36467407)
They most certainly don't welcome challenges to the basic assumptions on which their teachings rest.

Professors are trying to teach a class, not engage in a debate with a student. Challenging basic principles that everyone in the class are already assumed to have accepted is usually seen as an attempt to derail the lecture.

Derailing the lecture and drawing the professor into a debate with you effectively denies the other students access to education. It’s roughly equivalent to heckling a comedian.

If you want to debate your professor, do it on your own time, in office hours. If the professor is still offended and unwilling to debate then you have reasonable grounds to complain. Most professors I’ve met absolutely love to debate outside of class.

Re:Harvard’s new computer science teacher is a chatbot by civilized on Sunday June 25, 2023 @02:26AM (#36465554)
"Always think critically"?

"Simply read the textbook" would be a more practical instruction, and also what I did in university anyway.

Critical thinking is a useful description but not a good prescription. It's like telling someone to always be orange. If you want that to happen you have to eat carrots. It won't just happen by fiat or willpower.

Re:Harvard’s new computer science teacher is a chatbot by quickthrower2 on Sunday June 25, 2023 @05:31AM (#36466376)
Youtube + textbook + good discord (slack or IRC etc.) can replace an undergrad CS course pretty well IMO for at least the practical skills. AI can help but is a minor piece, at least right now.
Re:Harvard’s new computer science teacher is a chatbot by danuker on Sunday June 25, 2023 @07:12AM (#36466992)
Whatever a student encounters in a field probably has search results on the web. I find it hard NOT to find the same question on sites like StackOverflow, Quora, or Reddit.
Re:Harvard’s new computer science teacher is a chatbot by civilized on Sunday June 25, 2023 @09:23AM (#36467959)
My last contact with the educational system as a learner was around 2009, so I missed pretty much the whole YouTube / Khan Academy revolution. It would be very interesting to see what it's like to learn today, what resources I'd find most effective.
Re:Harvard’s new computer science teacher is a chatbot by dools on Sunday June 25, 2023 @08:17AM (#36467419)
I saw a project where someone had trained an AI on classic novels so you could have a conversation with the book.

I think AI trained on text books should be able to achieve the same thing relatively easily.

It’s just an interactive textbook, seems like a pretty good idea to me.

Re:Harvard’s new computer science teacher is a chatbot by msla on Sunday June 25, 2023 @02:28AM (#36465558)
It could be a sink-or-swim kind of thing: Saying you trusted the AI isn't accepted as a reason for bad work, so anyone who trusts the AI and gets burned learns a lesson.

Maybe I'm imputing too much intelligence to Harvard's administration.

Re:Harvard’s new computer science teacher is a chatbot by donmcronald on Sunday June 25, 2023 @02:28AM (#36465562)
Doesn’t thinking critical require enough basic knowledge about a topic to be critical of information? How does that even work if you’re there specifically to learn? Education requires an information source you can (mostly) trust blindly. How else do you learn?

If they think ChatGPT is ready to teach, they should offer an electrical course using it and force the administration to take it.

Re:Harvard’s new computer science teacher is a chatbot by Lio on Sunday June 25, 2023 @04:12AM (#36465982)
My experience with LLMs so far is that they very confidently give you an answer and very convincingly explain that answer but for certain portion of the time that answer is very wrong.

You can even get it to break its reasoning down into little convincing but, none the less, incorrect steps.

If I was running this course I would make that "artefact" part of the process.

i.e. I would tell students that the chatbot will occasionally very convincing lead them in the wrong direction but that they can fact check its answers in the course material.

I would set traps to ensure that they are doing that. ;)

Use LLMs but never trust them, they're the Cliff Clavin of information tools.

Re:Harvard’s new computer science teacher is a chatbot by Hendrikto on Sunday June 25, 2023 @06:48AM (#36466857)
> I would set traps to ensure that they are doing that. ;)

You won‘t have to. Students are extraordinarily skilled at fining endless pitfalls on their own.

Re:Harvard’s new computer science teacher is a chatbot by DistractionRect on Sunday June 25, 2023 @10:55AM (#36468809)
That can be a double edged sword, particularly when the lesson is about an edge case/pitfall. You'd be surprised how they can seemingly blunder their way to the solution while waltzing around the edge case, and thus not learning the intended lesson.

Setting the stage where they either land in the pit, or start from within, ensures they're interacting with the material as intended.

Re:Harvard’s new computer science teacher is a chatbot by ekianjo on Sunday June 25, 2023 @07:25AM (#36467077)
> My experience with LLMs so far is that they very confidently give you an answer and very convincingly explain that answer but for certain portion of the time that answer is very wrong.

This is true of humans just as well

Re:Harvard’s new computer science teacher is a chatbot by Topfi on Sunday June 25, 2023 @07:54AM (#36467262)
Whilst that is true, I feel that if that human is an educator employed by a reputable institution, answering questions in their field, the expectation of most is going to be that they provide accurate information. As this is an introductory class, any arguments that even experienced teachers can occasionally cling to incorrect or outdated information becomes moot in my opinion.

LLMs as of now simply appear not reliable enough for an institution like this one to attach their name (which brings credibility in the eyes of the public), especially as their comments on the need for users to be vigilant ("they should always think critically when taking in information as input") make it sound like they did not in fact find some new way of reliably grounding output in accurate information.

Mind you, this is coming from someone who has great hopes for the future of LLMs in education as they can more effectively deal with students on an individual level. I just feel that currently LLMs are still too error-prone for an institution such as Harvard to incorporate them into an introductory course to assist. Until these insitution stop putting the majority of responsibility on students finding erroneous output, I feel this is not ready for wider educational use.

Re:Harvard’s new computer science teacher is a chatbot by ekianjo on Sunday June 25, 2023 @11:53AM (#36469543)
> an educator employed by a reputable institution,

What's a reputable institution? These days most "experts from institutions" are proven to be wrong. The COVID-19 pandemic was a clear eye-opener on how little you could just trust anyone.

Re:Harvard’s new computer science teacher is a chatbot by hgsgm on Sunday June 25, 2023 @08:23AM (#36467461)
Humans are not programmed to avoid saying "I don't know."
Re:Harvard’s new computer science teacher is a chatbot by ekianjo on Sunday June 25, 2023 @11:51AM (#36469518)
Oh really? You will find very few people admit that they don't know something.
Re:Harvard’s new computer science teacher is a chatbot by nmz on Sunday June 25, 2023 @04:06PM (#36472067)
If you ask someone they will give you their approximation of an answer. However if you told them if they would bet money on it, and promptly make them do so, you will find them saying IDK more frequently.

Even this, I wouldn't bet money on what I just said.

Re:Harvard’s new computer science teacher is a chatbot by kevin_thibedeau on Sunday June 25, 2023 @01:45PM (#36470810)
There is a prominent culture where this is such a common occurrence that you have to be careful about accepting their confident assertions.
Re:Harvard’s new computer science teacher is a chatbot by 8note on Sunday June 25, 2023 @06:32PM (#36473346)
Humans also tend not to write "I don't know" into text content about a certain subject. Usually the idk is hidden by not writing at all
Re:Harvard’s new computer science teacher is a chatbot by sharma-arjun on Sunday June 25, 2023 @01:35PM (#36470721)
One useful trick to avoid some of the pitfalls of ChatGPT for non-creative tasks is to respond to its output with something like:

_"Point out incorrect assumptions or statements in the above answer."_

I got the idea from Khan Academy's [implementation](https://youtu.be/A7REVn9gzgs?t=1208) of GPT-4, where they allow it to 'think' by generating an internal response first, before generating a final response. This apparently improved the veracity of its output by a significant margin.

If ChatGPT doesn't double down on its original answer, then I know definitely not to trust it.

This doesn't solve the underlying problem, because it doesn't ultimately correct the error in a reliable way, but it does help me avoid some of the more obviously garbage output.

Re:Harvard’s new computer science teacher is a chatbot by latexr on Sunday June 25, 2023 @04:50AM (#36466180)
“Always think critically” is poised to become the “drink responsibly” of AI. A convenient and counterproductive way to push responsibility onto the users.

https://en.wikipedia.org/wiki/Alcohol_advertising#Drink_resp...

https://youtube.com/watch?v=DnSp2S7vzH4&t=31s

Re:Harvard’s new computer science teacher is a chatbot by grumpyprole on Sunday June 25, 2023 @05:50AM (#36466481)
Agreed, and due to the way LLM works, misinformation tends to look very plausible.
Re:Harvard’s new computer science teacher is a chatbot by scotty79 on Sunday June 25, 2023 @07:06AM (#36466941)
From the perspective of learning how to tell lies from the truth it's a very good lesson that things can be bs no matter how plausibly and nice they sound. So you always need to check sources and read a lot on the subject.
Re:Harvard’s new computer science teacher is a chatbot by ekianjo on Sunday June 25, 2023 @07:24AM (#36467070)
Misinformation was never a problem on the provider end but the receiver
Re:Harvard’s new computer science teacher is a chatbot by dr_dshiv on Sunday June 25, 2023 @07:10AM (#36466974)
Yes. Definitely users are responsible for knowing this. And it is one of the key things that education needs to teach.
Re:Harvard’s new computer science teacher is a chatbot by chongli on Sunday June 25, 2023 @08:04AM (#36467332)
I’m generally skeptical of the critical thinking movement. There is not much evidence that students can be taught to transfer their critical thinking skills from one subject to another. I think much of what people refer to as critical thinking is more aptly called subject expertise.
Re:Harvard’s new computer science teacher is a chatbot by hgsgm on Sunday June 25, 2023 @08:22AM (#36467457)
Huh? Are you saying that math and logic are not useful in a wide variety of subjects?
Re:Harvard’s new computer science teacher is a chatbot by webmaven on Sunday June 25, 2023 @08:43AM (#36467616)
Of course they are useful, but the utility is broader than you think. Those skills are just as easily used for rationalization, eg. "Lies, damn lies, and statistics."
Re:Harvard’s new computer science teacher is a chatbot by chongli on Sunday June 25, 2023 @10:02AM (#36468281)
I’m a math student. Of course I think math and logic are useful. They’re still subject expertise. They will not help you analyze a painting or a piece of music or a poem.

They also generally won’t help you evaluate scientific claims unless you also have enough scientific background. If someone says “humans will set foot on Mars within 30 years”, you’re going to have a hell of a time determining whether that’s reasonable unless you know a lot about: rocketry, physics, space travel, radiation, astronomy, human anatomy, nutrition, engineering, economics, and many other subjects!

If you’re just a math/logic wiz and not an expert in those other areas, you may be inclined to say “they’ve already put rovers on Mars, should be easy to get humans there!” because there’s nothing impossible or illogical about the question.

Re:Harvard’s new computer science teacher is a chatbot by some_random on Sunday June 25, 2023 @11:07AM (#36468977)
Critical thinking isn't about evaluating the claim "humans will set foot on Mars within 30 years" by examining rocketry, physics, space travel, etc. It's about asking questions like "who is telling me this, why would they say this, what other beliefs do they have that might influence this one, what do other people say about this claim?" It's universally applicable but gets more effective the more you know about a given topic.
Re:Harvard’s new computer science teacher is a chatbot by SamoyedFurFluff on Sunday June 25, 2023 @11:56AM (#36469587)
That’s tautological about whether or not the claim itself has merit. It only says if the claim doesn’t have merit, here is a set of motivations that might have biased the claim. But there’s no way to determine just from that whether the claim has merit. It goes double for the vast majority of the way we get information: ad hoc through conversations with other humans. If a neighbor tells me it’s thunder storming tomorrow, therefore I should postpone putting up my new fence plans, I’m not going through a bizarro world decision matrix involving surveying the neighborhood for validity!
Re:Harvard’s new computer science teacher is a chatbot by some_random on Sunday June 25, 2023 @12:23PM (#36469868)
You are massively overthinking this. Critical thinking isn't some giant flowchart of questions and rules that you have to run on every single thing you ever hear or think in your entire life, that's obviously insane. Frankly I don't understand how you could even think that. Critical thinking is just saying "Hey this neighbor has a history of lying to me and my other neighbor said that they tried to get a court order to stop them from putting up a fence. Maybe I should check the forecast before canceling my fence building plans".
Re:Harvard’s new computer science teacher is a chatbot by chongli on Sunday June 25, 2023 @12:48PM (#36470159)
Critical thinking is just saying "Hey this neighbor has a history of lying to me and my other neighbor said that they tried to get a court order to stop them from putting up a fence. Maybe I should check the forecast before canceling my fence building plans".

How is that not an application of domain knowledge (know your neighbours) with a bit of intelligence? How do you teach people to do that in general? How do you teach people to deal with questions when they lack domain knowledge?

Re:Harvard’s new computer science teacher is a chatbot by some_random on Sunday June 25, 2023 @12:58PM (#36470276)
Again, you're massively overthinking this. Then entire point is to not blindly accept things and instead actually evaluate claims. You can do that with domain knowledge you have, which appears to what what you're calling all knowledge. If there exists some kind of knowledge that isn't domain knowledge, I'm sure you could find a use for it too.
Re:Harvard’s new computer science teacher is a chatbot by some_random on Sunday June 25, 2023 @11:03AM (#36468913)
I don't believe that at all, the idea of challenging what you know and how you know it is extremely useful across all subjects. The problem is that the efforts to teach critical thinking has been colossal failures.
Re:Harvard’s new computer science teacher is a chatbot by tbrownaw on Sunday June 25, 2023 @12:03PM (#36469656)
Thing is, doing a consistency check on your collection of purported facts is the easy part. What's hard is the soundness check where you need a lot of domain knowledge to see if things actually match objective reality.
Re:Harvard’s new computer science teacher is a chatbot by some_random on Sunday June 25, 2023 @12:42PM (#36470085)
I don't see any domain (except politics of course) where asking yourself "is there another explanation for what I'm seeing" isn't useful.
Re:Harvard’s new computer science teacher is a chatbot by willcipriano on Sunday June 25, 2023 @08:04AM (#36467331)
Always think critically, but whatever you do don't do your own research.
Re:Harvard’s new computer science teacher is a chatbot by agumonkey on Sunday June 25, 2023 @07:21AM (#36467052)
Depending on the level of pupils it can be safe, but after taking part in moocs with people of various skill levels, it's fair to assume that a lot will be lost in confusion without the ability to discern correct from incorrect.
Re:Harvard’s new computer science teacher is a chatbot by c7b on Sunday June 25, 2023 @09:04AM (#36467792)
In some sense, for a class like CS50, experiencing first-hand that current LLMs are often confidently wrong could be seen as part of the educational mission. If you asked me what the first thing that a young user should know about today's tech is, then it's probably that.
Re:Harvard’s new computer science teacher is a chatbot by David_SQOX on Sunday June 25, 2023 @09:12AM (#36467867)
Humans love taking the path of least resistance. It's tempting to trust whatever ChatGPT says, especially if you are a young student. Getting burned and learning the hard way is how most students are going to learn this lesson, just like any other lesson in life.

Being efficient and being lazy are rubbing up against each other with LLMs. A lot of blurred cognitive lines with this budding technology.