News for hackers. Stuff that matters.
Re:Harvard’s new computer science teacher is a chatbot by chefandy on Sunday June 25, 2023 @02:29AM (#36465565)
CS50 was a gigantic intro class designed to whet non-comp-sci students appetite for coding and computer science, generally. (I took it when it was C/PHP, so they may have changed things.) While the lectures are very straightforward, the smaller 'sections' lectures with the dozen-or-so teaching fellows were more intensive, and p-sets were pretty challenging for newbies. The weekly help sessions ended up being a million lost students in a giant hall waiting to see one of a few dozen TAs offering uneven pedagogical quality. The assignments were often automatically graded upon submission and you often have some equivalent of a unit test to see if you got it right before submission, so you know if you got it wrong, but not how wrong. Just having a resource for sanity checks and walkthroughs of lesson concepts without trudging to the hall-o-TAs is prolly really useful.

Totally legit philosophical debates aside, I think this is a great use of the technology. It's a very, very well-funded class (bigger dedicated full-time year-round staff than some entire departments,) so I'm sure it's not a half-assed effort.

Malan is a great lecturer and his CS50 classes are all on YouTube. I sometimes send new coders to them but they're a fun watch, in general.

Re:Harvard’s new computer science teacher is a chatbot by eesmith on Sunday June 25, 2023 @03:07AM (#36465706)
I find it hard to reconcile "The weekly help sessions ended up being a million lost students in a giant hall waiting to see one of a few dozen TAs offering uneven pedagogical quality." with "very, very well-funded class".

Why didn't the funding go to hiring more staff?

My experience in a state school had far more access to education staff than what you describe, and for less funding.

Re:Harvard’s new computer science teacher is a chatbot by vanviegen on Sunday June 25, 2023 @04:42AM (#36466145)
My reading is that this class just has a huge number of students, each bringing in a bit of funding. This allows for relatively large investments in the quality of the teaching materials, but teacher/student ratios will probably not be great. Hiring capable teachers is hard, especially at scale.
Re:Harvard’s new computer science teacher is a chatbot by eesmith on Sunday June 25, 2023 @07:44AM (#36467200)
I must be missing something. Harvard has 7,153 undergrads says Wikipedia. At most that's 1,790 students per year taking the class each year. But surely not everyone is taking the course? So why does it need to scale?

Harvard tuition is $54,269, says https://registrar.fas.harvard.edu/tuition-and-fees .

Assuming it's a 3 hour class with a 120 credit hours to graduate, that's 30 hours per year, so each student pays about $5,000 for what seems to be a horrible staff/student ratio.

FWIW, I found https://cse.engin.umich.edu/stories/popular-intro-cs-course-... as a comparison, the EECS 183 course at the University of Michigan.

Three professors (Dr. Dorf, Dr. Bill Arthur, and Dr. Héctor García-Ramírez), and 24 student instructors for the 870 students who finished the course. (It doesn't say how many started the course.)

Full-time tuition at UMich is $8,780/in-state and $27,663/out-of-state.

Without class size numbers for Harvard, I can't directly compare the two.

I can understand that the distance learning version of the course does need to scale. But chefandy was talking about the in-person version of the course. (Unless "in a giant hall waiting" means some sort of VR-based course. :)

Re:Harvard’s new computer science teacher is a chatbot by wil421 on Sunday June 25, 2023 @08:03AM (#36467321)
My coworker’s kid just graduated from Harvard. He said it cost about $315,000k. Does your number include all the fees and such most colleges tack on to tuition?
Re:Harvard’s new computer science teacher is a chatbot by rocho on Sunday June 25, 2023 @09:30AM (#36468007)
You probably mean $315k.
Re:Harvard’s new computer science teacher is a chatbot by chefandy on Monday June 26, 2023 @01:54PM (#36483260)
If they lived in off-campus housing in Cambridge then $315m would be on the low side.
Re:Harvard’s new computer science teacher is a chatbot by eesmith on Sunday June 25, 2023 @11:20AM (#36469149)
I linked to https://registrar.fas.harvard.edu/tuition-and-fees which shows I quoted only tuition. That's the amount which most directly goes toward paying for educational staff salaries.

The total tuition+fees for Academic Year 2023-24 from the same page, is $79,450/year, giving $317,800 for 4 years, and matching your number.

Re:Harvard’s new computer science teacher is a chatbot by lelandbatey on Sunday June 25, 2023 @02:45PM (#36471358)
I think they're pointing out that $315k is different from $315,000k as both include the k (for thousand) but one includes the thousands as 0's and the other does not.
Re:Harvard’s new computer science teacher is a chatbot by eesmith on Sunday June 25, 2023 @02:53PM (#36471419)
I think wil421 made a typo. I don't think anyone believes Harvard costs $315 million for a degree.

Instead, I think wil421 was asking about the difference between "tuition" and "tuition and fees".

Re:Harvard’s new computer science teacher is a chatbot by wil421 on Tuesday June 27, 2023 @10:56AM (#36493805)
Typo!!
Re:Harvard’s new computer science teacher is a chatbot by Kon-Peki on Sunday June 25, 2023 @09:21AM (#36467946)
> At most that's 1,790 students per year taking the class each year

Anyone in the world can take CS50 for credit via Harvard Extension [1], and if you happen to be in the Boston area you are welcome to come to campus and attend lectures, sections, office hours, etc (but they also have everything you need fully online).

Like other large classes at Harvard (Stat 110, etc), these operate just like the weed-out courses at the big state schools. Unlike the state schools the deadline to drop a course without it showing up on your transcript or affecting your GPA is really, really late in the semester. So it's not uncommon for 50+% of the kids to be gone by the final exam/project.

[1] https://cs50.harvard.edu/extension/2023/fall/

Re:Harvard’s new computer science teacher is a chatbot by eesmith on Sunday June 25, 2023 @02:16PM (#36471109)
The pricing doesn't make sense. Harvard Extension charges $2,040/4 credit course[1]. I estimated this course cost Harvard students $5,000, for low-quality access to educational staff.

Does that mean Harvard Extension students get even worse access?

Surely paying twice as much in tuition should mean something more than getting to say "I graduated from Harvard."

[1] https://extension.harvard.edu/paying-for-school/

Re:Harvard’s new computer science teacher is a chatbot by Kon-Peki on Sunday June 25, 2023 @08:25PM (#36474164)
No, they get the same access to the courses they are enrolled in.

An extension student doesn’t have access to the full course catalog, and doesn’t live in one of the undergrad houses, doesn’t develop the same network of friends etc.

But they do attend the same morning graduation exercises in Harvard Yard and get the same lifetime benefits (if they carry through to graduation, that is).

Re:Harvard’s new computer science teacher is a chatbot by eesmith on Monday June 26, 2023 @12:11AM (#36475657)
> access to the full course catalog

That would (to me) only make sense if the extra $3,000 for the CS course subsidizes the other courses.

> and doesn’t live in one of the undergrad houses

Which is why I only compared tuition, and did not include the housing fee.

Re:Harvard’s new computer science teacher is a chatbot by chefandy on Monday June 26, 2023 @10:43AM (#36480249)
Not all of the classes are the same quality— though many are. Harvard College summer classes are all done through the extension school. I think extension students can actually enroll in the full undergrad catalog once they get to a certain point in their degree program if they're preforming well. Not positive though.

However, your degree options are very limited, you don't get access to the same interaction with the big famous names (though you can usually work some magic through side channels for genuine collab interests,) you don't really interact with the Harvard College undergrads, you don't get the same internship opportunities, you can't join the sports teams or go to most undergrad-specific events, you don't live in the Harvard buildings or eat in the Harvard dining halls, your email address is extension-school specific... etc etc etc. Even if there are undergrad specific things you technically can take part in, nobody is going to make sure you know about them. You DO get the same access to the online resources, all of the clubs, and all SEVENTY TWO libraries (some are like office-sized.)

Harvard knows how much cachet a Harvard degree has, and with the extension school, they must walk the razor-thin line between marketing that to extension students, and making the Harvard College students feel like ubermensch. You actually get a different degree from the extension school— a "Bachelor of Liberal Arts (ALB) in Extension Studies concentrating in XYZ" rather than the "BA/SB concentrating in XYZ" to differentiate it even more. Lots of extension school students try to push it on their resumes because most employers don't know the difference, and lots of Harvard undergrads get mad about it and feel like that cheapens their degree. In reality, the connections, outlook, exposure, and experiences uniquely available to them in Harvard College are the real secret sauces.

Re:Harvard’s new computer science teacher is a chatbot by eesmith on Tuesday June 27, 2023 @07:58AM (#36491628)
> Harvard knows how much cachet a Harvard degree has

Reading about how crappy their intro CS course it makes me downgrade their cachet.

I can't help but wonder if the richer students simply pay for a tutor, rather than depend on the school-provided services.

Re:Harvard’s new computer science teacher is a chatbot by Kon-Peki on Wednesday June 28, 2023 @10:58AM (#36507411)
Harvard isn't known for their CS program. I did CS undergrad at a school ranked quite a bit higher than Harvard (for CS). I think that CS50 would have been quite an improvement on the intro class at my school. But you have to understand, these classes are supposed to be hard. CS requires a huge amount of self-study and a huge amount of independent work. The kids that aren't willing to do that gain nothing by advancing to the second year curriculum.

The system isn't perfect - it never will be. But seeing lots of kids at sections and office hours is a good thing. It shows that they are willing to ask for help when they need it, and are willing to put in the hard work to succeed. I'm sure that some of the richest kids pay for private help. But one thing you need to understand about the wealthy is that they are more likely to make use of the services that are available to them. It is very striking to see how many poorer people suffer in silence when help is available, because they don't want to be a burden or are embarrassed to be in need of help. I assure you that no rich person ever feels embarrassed by asking for help.

As for Harvard Extension, a lot of people will tell you that it is around 100 years old. But in reality, it is a lot closer to 200 years old. Harvard alum John Lowell Jr left half of his estate to provide educational opportunities to greater Boston. They paid the most accomplished academics they could find to present lectures for free or cheap at night so that people who worked jobs during the day could attend. It wasn't limited to Harvard professors, of course, and it continuously evolved to serve the needs of the day. The trustees at MIT were the first to organize lectures into a real curriculum rather than a set of ad-hoc topics. By the early 1900s it had transformed into an organized group of all of the colleges and universities in the Boston area. Depending on the subject matter you studied, you "graduated" from the school that specialized in that subject even if you took courses at a different school. All those schools dropped out, one by one, until Harvard was the only one left - which shows their commitment to the concept (no entrance exam, inexpensive, targeted at non-traditional students).

Re:Harvard’s new computer science teacher is a chatbot by chefandy on Sunday June 25, 2023 @09:47AM (#36468148)
Not that many students on campus. From Wikipedia "The on-campus version is Harvard's largest class with 800 students, 102 staff..." they probably have that many or more online students but they aren't the ones in annenberg, the freshmen dining hall, waiting for TAs. (That hall is a dead ringer for the great hall at hogwarts.)

At the events, there were Facebook and Microsoft logos everywhere. I'm pretty sure corporate sponsorship took care of a lot of it.

Re:Harvard’s new computer science teacher is a chatbot by vasco on Sunday June 25, 2023 @05:03AM (#36466235)
Feom what I understand of these universities with endowments they use money to hire administrative staff and to "last forever". Teaching staff is secondary to those.
Re:Harvard’s new computer science teacher is a chatbot by chefandy on Monday June 26, 2023 @08:16PM (#36487558)
Yes. They lay off scores of staff in nearly every economic downturn. There's a whole lot of 'blah blah blah' about why they can't use the endowment to avoid screwing people who've given decades of their professional life to Harvard, but I think it mostly comes down to their wanting a big shiny endowment balance long after everyone on earth right now has died.
Re:Harvard’s new computer science teacher is a chatbot by chefandy on Sunday June 25, 2023 @09:34AM (#36468038)
Harvard is bizzaro land. I don't know what CS50s setup looks like internally, but after taking the course, and working there for a couple of decades— one full time— I think I'm pretty close.

I learned that they always have money for stuff but rarely for staff. The class seemed to have significant corporate support from Facebook and Microsoft, and probably others— both Zuck and Ballmer have given guest lectures— but I'll bet that Harvard's governance rules didn't let them hire academic staff with outside funding or something like that. They def could use it on regular staff positions— I spent years in a full-time, permanent staff position that was funded through a project partner.

The support academics in CS50 are all done by student TAs at the lowest level, or teaching fellows— grad students maybe— for lecturing or more advanced help. The staff are for things like video production, technical infrastructure, etc. There's only so many qualified students willing to do that work and no way you're getting existing faculty to do it. I'll also bet that they have rules about hiring staff to do anything resembling instructional work.

Like I said, though, I don't know what the situation is exactly but I'm pretty familiar with Harvard.

Re:Harvard’s new computer science teacher is a chatbot by lioeters on Sunday June 25, 2023 @11:25AM (#36469206)
> grad students

Why buy the cow, amirite?

Re:Harvard’s new computer science teacher is a chatbot by chefandy on Sunday June 25, 2023 @05:34PM (#36472816)
Yeah though they're unionized now I think. I wonder if anything changed with cs50?
Re:Harvard’s new computer science teacher is a chatbot by chefandy on Monday June 26, 2023 @08:10PM (#36487516)
> I learned that they always have money for stuff but rarely for staff.

(By "they," I mean Harvard, generally. I have no reason to believe CS50 is exempt from this, but I have no special insight, here.)

Re:Harvard’s new computer science teacher is a chatbot by some_random on Sunday June 25, 2023 @11:00AM (#36468871)
Yeah, not exactly helping with my prior of "college is a massive, societal level scam"
Re:Harvard’s new computer science teacher is a chatbot by chefandy on Monday June 26, 2023 @02:15PM (#36483520)
College is great and very useful, it's just heavily abused by our society. I think that a lot of well-meaning people in the 80s and 90s saw that people with college degrees were better off, and confusing correlation with causation, insisted everyone needed to go to college so they could also be better off. Well, now we have way more people with college degrees than we need.

They've taken on debt that their (government-employed) guidance counselors insisted would pay for itself. Decent paying white collar positions for non-degree holders, like administrative assistant, now require them because of lazy recruiting. And income inequality is worse because you can't resolve structural economic issues with credentials.

So you have someone with an expensive English degree from BU answering phones in a CS pool for 35k, and the 'dumb' kids who went to trade school and became electricians have to turn away work at hundreds of dollars per hour because they're in such high demand.

Re:Harvard’s new computer science teacher is a chatbot by devwastaken on Tuesday June 27, 2023 @04:35AM (#36490374)
Harvard isn't about learning. It's about paying the TA. Connections and money.
Re:Harvard’s new computer science teacher is a chatbot by Paul-Craft on Sunday June 25, 2023 @05:18AM (#36466319)
> The weekly help sessions ended up being a million lost students in a giant hall waiting to see one of a few dozen TAs offering uneven pedagogical quality. The assignments were often automatically graded upon submission and you often have some equivalent of a unit test to see if you got it right before submission, so you know if you got it wrong, but not how wrong. Just having a resource for sanity checks and walkthroughs of lesson concepts without trudging to the hall-o-TAs is prolly really useful.

It's been a while since I was an undergrad, and I didn't study CS when I was (I was in math -- we didn't even get the unit test to tell us we were wrong lol). I understand auto-grading like this is very common in lower level CS classes, but wouldn't it be possible to write these "test suites" in such a way that they could provide some hints on exactly what was wrong? I have to imagine, based on my experience as a graduate TA in math, that undergrads as a whole make mistakes that typically fall into certain classes, and that those classes could be reflected or captured in the pattern of which "unit tests" end up failing. It doesn't strike me as terribly difficult to analyze these patterns of failures, after running the course at least once, and come up with useful hints as feedback.

Does nobody do this? Or, perhaps more to the point, does nobody ask their TAs to do this?

> Totally legit philosophical debates aside, I think this is a great use of the technology. It's a very, very well-funded class (bigger dedicated full-time year-round staff than some entire departments,) so I'm sure it's not a half-assed effort.

I agree broadly with this. If you're going to do this experiment, in principle, a class like Harvard CS-50 sounds like the right place to start, for the reasons you've listed.

However, if this chatbot is just an equivalent of ChatGPT with the GPT-4 model and some additional training/fine tuning, I don't think the technology is quite there yet. We've seen a lot of examples already of how GPT-x has a tendency to give "confidently incorrect" answers, and that's the absolute last thing you want to be telling beginners. I've seen answers that could plausibly fool experts, yet be wrong enough as to be useless once you try to verify the details. The danger here is that beginners may not recognize this, and may not be able to verify the details.

That said, I think there is a future for this sort of technology in precisely this type of instructional environment. The "confidently incorrect" problem is something that can be mitigated with fine-tuning, proper prompt engineering, and other techniques, but, to my knowledge, can't be eliminated if the system is essentially a bare text interface into an LLM. I think the work being done with web search is a possible direction here, since I've seen GPT-4 essentially give up and go search the web when pressed for details or when the actual answer would become too complex. For a restricted domain like CS50, maybe even some kind of formally encoded knowledge base would help it (similar to how I mentioned analyzing patterns of failures to provided hints).

What I do know is that the gigantic "hall-o-TAs" is absolutely not a great user experience for undergrad learners, so anything that offers the potential for improvement on that UX is definitely something I'd like to see pursued and developed.

Re:Harvard’s new computer science teacher is a chatbot by rmbyrro on Sunday June 25, 2023 @06:04AM (#36466560)
Providing factual context in the prompts reduces hallucinations to nearly zero in my anecdotal experiences.

For example, I once copy pasted an OSS code base into ChatGPT and all hallucinations went away. I had to limit to the files relevant to my questions, due to the context window limitation, but worked really well.

Re:Harvard’s new computer science teacher is a chatbot by cdperera on Sunday June 25, 2023 @06:09AM (#36466593)
Regarding providing hints, my university does this actually. The automated testing software we use (in house built, actually) allows the lecturer / TA team to define hints in case that test case fails.

Providing hints is a bit tricky, however. Often times, students fail the test case for unexpected reasons, and our hints actually mislead them. I've had students come to me (as a TA) saying that the test case is wrong because they take into account the hint.

Naturally, we could just give better test cases / give a broad disclaimer, which we try to. We do the latter, but the former is tricky, especially when we are trying to come up with novel problems.

Re:Harvard’s new computer science teacher is a chatbot by chongli on Sunday June 25, 2023 @07:48AM (#36467221)
I have been a TA for first-year intro CS on multiple occasions. I have written the test suites that auto-grade student submissions.

The problem with first year CS students is that, unlike math, they’re allowed to have a huge range of possible prior experience in the subject. Some have zero programming experience whatsoever, while others started programming in diapers and competed in numerous programming contests (and even won) before enrolling.

Yes, we do offer an advanced version of the intro course that is far more challenging and would be appropriate for many of those advanced students. But the advanced course is optional so no student can be forced to take it. Many of the advanced students opt to take the standard intro course in the hope that it would boost their grades.

So the course is dealing with (essentially) a strongly bimodal distribution of student ability and the professors don’t want to give the advanced students a free ride. That means the course is actually somewhat challenging for all but the most elite programmers and therefore extremely challenging for total beginners.

What this meant was that Monday office hours (assignments were due Tuesday mornings) typically had several hundred students (out of a total of about 1800) scrambling for help from a dozen TAs. Of course, my school only has a tiny fraction of the funding of Harvard, but there you go.

I doubt that Harvard would want to spend proportionally more on TAs for their course (which would mean more TAs than students), so even though their course may be way more “well-funded” than ours, it’s not at all so relative to endowment.

Re:Harvard’s new computer science teacher is a chatbot by fn-mote on Sunday June 25, 2023 @08:21AM (#36467444)
> So the course is dealing with (essentially) a strongly bimodal distribution of student ability and the professors don’t want to give the advanced students a free ride. That means the course is actually somewhat challenging for all but the most elite programmers and therefore extremely challenging for total beginners.

HN readers: universities that teach a so-called "How to Design Programs" curriculum [1] have discovered a partial solution to this problem. Emphasizing a design process in a functional language removes the advantage that students with experience but not a deep understanding gain from knowing how to cobble things together the way they learned in high school AP CS A.

I'm not saying it's the only route, but it works well and develops important skills that are (ime) not learned by tinkering.

[1]: https://htdp.org [2]: https://course.ccs.neu.edu/cs2500/labs.html

Re:Harvard’s new computer science teacher is a chatbot by chongli on Sunday June 25, 2023 @09:49AM (#36468156)
My university uses that exact approach! I have first hand knowledge with students who refuse to follow the design recipe approach and insist on muddling through the code until they figure it out.

We even hand-mark their design recipes in the course! The students just do them after the fact.

Re:Harvard’s new computer science teacher is a chatbot by joker_minmax on Sunday June 25, 2023 @09:58AM (#36468249)
Couldn't they just distribute the class ability level correctly using a placement test?
Re:Harvard’s new computer science teacher is a chatbot by voakbasda on Sunday June 25, 2023 @10:23AM (#36468436)
The advanced students looking to boost their grade could easily lower their score intentionally, so they get sorted into the beginner group.
Re:Harvard’s new computer science teacher is a chatbot by anticensor on Sunday June 25, 2023 @12:03PM (#36469657)
There is a solution to that too: Have the placement exam counted as a proficiency course in the student grades, like foreign language requirements or scientific prep courses in graduate-level schools for those majored in a different field.
Re:Harvard’s new computer science teacher is a chatbot by chongli on Sunday June 25, 2023 @12:21PM (#36469849)
My school is a public university. They’re not allowed to introduce mandatory placement exams like that. They have to work within the government-set high school curriculum.

They also don’t want to allow the advanced students to “test out” of the intro course, as some schools do, because the advanced students may be highly skilled at programming but totally lacking in introductory CS knowledge.

Re:Harvard’s new computer science teacher is a chatbot by sharma-arjun on Sunday June 25, 2023 @01:21PM (#36470573)
Yeah, it's extremely common for people to be extremely prolific programmers in high school, without having much knowledge of foundational computer science or algorithm concepts.

Source: Am a high school programmer

Re:Harvard’s new computer science teacher is a chatbot by joker_minmax on Monday June 26, 2023 @06:45PM (#36486879)
I went to a public university and they determined which mathematics class you started in by your ACT/SAT or AP test scores. I guess I'm just struggling to see why a programming course would be any different, or why a placement test would be prohibited when they're already using the standardized test scores as a placement test of sorts. Unless things have changed in the last five years.
Re:Harvard’s new computer science teacher is a chatbot by chefandy on Monday June 26, 2023 @08:42PM (#36487728)
(to be clear, I was never a Harvard College undergrad. I was a degree candidate in a Harvard Extension program before transferring, and I worked for Harvard for over two decades, one of them full-time.)

Firstly, a big part of the allure of CS50 is the fact that it's one big ol' nerdy computer party and you're invited!

Secondly, and I'm not trying to sell the Harvard mystique, but considering that Harvard's acceptance rate is 4%, the students handle brick-wall challenges gracefully.

Additionally, they don't necessarily want students that are going to do poorly in CS50 to pursue computer science at Harvard, so I'm sure it also serves as an effective bouncer for all of the students who haven't decided on their concentration deciding that they'll do CS because tech is sexy right now. The lectures look pretty straightforward, but we had to write code involving basic data structures and memory management in C on our paper midterm exams, with no reference, administered in a lecture hall. It's probably tough for kids that are used to acing everything, but it's a lot easier to find out you don't like CS in CS50 than it is to find out after choking on discrete math or a compiler class.

And as my first-year expository writing professor said after assigning like 70 pages of close-reading Kant for a weekly assignment, "Will it be difficult? Yes. Reading Kant is like running in sand. But that's why you decided to go to Harvard."

Re:Harvard’s new computer science teacher is a chatbot by chefandy on Monday June 26, 2023 @08:45PM (#36487746)
(to be clear, I was never a Harvard College undergrad. I was a degree candidate in a Harvard Extension program before transferring, and I worked for Harvard for over two decades, one of them full-time.)