News for hackers. Stuff that matters.
Re:Bloom's 2 Sigma Problem by trane_project on Saturday February 25, 2023 @03:23PM (#34940080)
I am already working on fixing this: https://github.com/trane-project/trane/

No AI needed. Just an old fashioned depth first search through a graph of skills and dependencies.

I made it to help me practice music, but I have been branching out and using it to study math for a few weeks. I find myself saying "just one question more" and then spending another half hour in it.

Still needs more material to be useful to other people but it's a solid experience. I learned and memorized how to play most of the notes in the saxophone with good intonation in about a week, as a complete beginner.

Re:Bloom's 2 Sigma Problem by cjblack on Saturday February 25, 2023 @03:32PM (#34940171)
I really like simple, reliable, tools like this - just watched the demo video. I hadn't heard of mastery learning before, but now I'm thinking about trying to map some Excel lessons towards this and see if I can implement it at work.

I've found that our youngest employees are coming out of school with little-to-no knowledge of how to work spreadsheet software - even those who are otherwise pretty technically proficient.

While programmers tend to eschew tools like Excel, in consulting (or at least the consulting we do) it's critical to being able to understand how a process works before designing an automated solution. Excited to potentially have a way to share that understanding.

Re:Bloom's 2 Sigma Problem by trane_project on Saturday February 25, 2023 @03:40PM (#34940241)
Thanks for the feedback. An excel course should be doable. Currently the most general way to write material is to use this: https://trane-project.github.io/generated_courses/knowledge_... Basically a folder of lessons where the flashcards are pairs of markdown files.

There's also another tool to write simple flashcards and lessons in a json file, run a command, and build all those directories and markdown files for you, but I have not written the documentation yet. Here's an example for a course based on a reworking of Euclid's Elements: https://github.com/trane-project/trane-math/blob/master/cour...

But plain markdown files for now should get you going.

Re:Bloom's 2 Sigma Problem by airstrike on Saturday February 25, 2023 @04:14PM (#34940561)
This looks great, thanks for sharing. I do think it would be more ergonomic to have this served via a website given that it feels CRUD-like with some browsing of exercises, relationships between them, etc.

The CLI approach assumes people are comfortable using the terminal, which unfortunately limits the number of potential users

Re:Bloom's 2 Sigma Problem by trane_project on Saturday February 25, 2023 @04:22PM (#34940615)
Definitely. I have not worked in frontend/ux stuff for over 10 years, and I am working on this by myself, so I have been focusing on the CLI to quickly iterate and prove that it works reasonably well. Part of the reason of trying it out on math was to a find a more technical group of users.

The ideal state is a graphic interface where the exercises are presented on the screen to reduce the friction. Gotta start somewhere.

Re:Bloom's 2 Sigma Problem by compacct27 on Sunday February 26, 2023 @09:03AM (#34945611)
I’ve been coding UI’s for over 8 years now, and I love ed tech adjacent projects like this. Wanna team up on making the UI?
Re:Bloom's 2 Sigma Problem by trane_project on Sunday February 26, 2023 @02:48PM (#34948700)
Sure. I would love some help. I am visiting my family all March (I am typing this from the airport), so I don't know how much time I'll have to work on trane for the next month. But maybe that gives you some time to use it, and read the code and docs.

I am not going to put my personal email here, and I don't see an option to send private messages on hn, although it's probably out there somewhere. Probably in the commits, lol. Worst case, feel free to open an issue on the trane repo.

As a first ask, what do you think of this: https://github.com/1Password/typeshare?

I figured writing the UI in rust is probably not a good idea. The ecosystem for UI is very immature and the language itself is probably overkill. But doing it in typescript/html/css requires you to understand the internal data structures (all the JSON files you see in the courses are just serialized rust objects).

I found that repo and it seems like a promising approach to autogenerate the types and make the interaction easier.

Re:Bloom's 2 Sigma Problem by jackthetab on Saturday February 25, 2023 @05:35PM (#34941150)
I'm more than happy to do a CLI app, but I think this needs a way to run it on a phone (noticed I don't necessarily mean "app" ;-). I can see me doing this on my phone while waiting for the barista to make my coffee.

I'm going to do something similar with Anki. I'll try to compare the two...

Re:Bloom's 2 Sigma Problem by trane_project on Saturday February 25, 2023 @05:49PM (#34941258)
Main difference with anki is that anki explicitly discourages trying to make dependencies between flashcards or decks. I tried to work out something like that into Anki, but as I saw it, dependencies were core to what I had in mind. There's some similar functionality in supermemo, but it's based on a tree, not a graph, so it was inherently limited.

No experience in mobile at all so that'll have to wait, but a GUI/website/app that works on most platforms is something I would like to work out eventually.

Re:Bloom's 2 Sigma Problem by airstrike on Saturday February 25, 2023 @06:43PM (#34941621)
Mobile these days can be React Native which also "builds" for the web and desktop

Next.js makes React less painful

Solito makes Next.js + React Native a possibility today: https://news.ycombinator.com/item?id=30678100

Re:Bloom's 2 Sigma Problem by trane_project on Saturday February 25, 2023 @06:52PM (#34941671)
I know it's possible. But the last time I did any frontend work, jQuery was the state of the art. So I decided that for now my efforts are better spent nailing the core experience and creating more courses.
Re:Bloom's 2 Sigma Problem by airstrike on Saturday February 25, 2023 @10:22PM (#34942747)
I'm not saying you don't know it's possible, just providing you pointers on one possible stack should you decide to go down that path
Re:Bloom's 2 Sigma Problem by trane_project on Sunday February 26, 2023 @02:53PM (#34948740)
My bad. I'll consider it when I decide to work on a GUI.
Re:Bloom's 2 Sigma Problem by wizzwizz4 on Saturday February 25, 2023 @04:34PM (#34940705)
Trane looks good. I've been trying to design a similar system, but for mathematics: the difference there is that there can be multiple routes towards a particular understanding, with different routes better for different people.

Prerequisites for calculus: an understanding of the standard reals OR an understanding of the hyperreals. Yes, real and hyperreal analysis are radically different in many ways, but they're provably equivalent, and they get you to the same answers. Some people find one more intuitive than the other, and some problems are easier in one than the other.

Imo, the biggest issue with mathematics education is that it tries to push you down one, standardised way of thinking about things: if you don't get it, that cripples your ability to think and reason for yourself. The main thing I do in one-to-one mathematics tutoring is finding how the student thinks, and nurturing that (instead of stamping it out and replacing it with a "more formal" approach).

I'd like to see something branchy, but I haven't found a good solution. Do you have any ideas for tackling this?

Re:Bloom's 2 Sigma Problem by trane_project on Saturday February 25, 2023 @05:01PM (#34940901)
I started working on this a few weeks ago: https://github.com/trane-project/trane-math It currently needs a README, but you could take a look at the courses on how I am building the flashcards. It's easy to reference external resources, so that's what I have been doing, rather than trying to create exercises of my own.

I am starting with a very basic olympiad-style book and a book based on Euclid's Elements, because I don't have the understanding required to clearly work out the dependencies of more advanced stuff. And I also would like to start at the beginning to make sure I don't miss anything.

The ideal end state is to have courses that cover all the undergrad and grad math curriculum. I am also curious on whether this could be used by researchers to keep up to date with the latest research on their fields. But all of that is a long way out.

As for your question, there are a couple of ways that Trane could handle multiple paths through similar material.

1. Just have separate curriculums. You could copy the courses, but the second copy has different dependencies, courses/lessons IDs. For example, one could have a series of courses teaching the undergrad MIT math curriculum and another the Harvard curriculum. They might share a lot of the material, but the order will be different.

2. Trane does not lock you into a specific order. There are filters that let you specify which parts of the graph you want to study. You are free to get questions from specific courses and lessons. You can also use the metadata in the courses to say things like "give me questions from all lessons teaching linear algebra" or "give me questions from all courses on real analysis but not from the lessons on set theory". The dependencies between the lessons that match that metadata are still respected. There are a few more options, but you get my point. The dependencies are not set in stone, and there's freedom to jump around and study specific topics.

I actually use option 2 most days. If I want to practice guitar, I just set a filter to give me exercises from the guitar. Similar thing when I want to practice the saxophone.

Re:Bloom's 2 Sigma Problem by wizzwizz4 on Saturday February 25, 2023 @05:48PM (#34941248)
Mathematics doesn't really have dependencies. Everything can be thought of in terms of everything else: you need an in, which is usually arithmetic or geometry (but, for physicists, is often physics – and for programmers, is often concepts related to their favourite programming languages), and then everything can be defined in terms of something else.

However, many people find it easier if you have concrete anchors, rather than making everything an abstract ivory tower. A certain kind of musician might get Fourier series quite quickly by analogy to the behaviour of sound, and then pick up Taylor series by analogy to that; somebody like me might find it easier to translate the most general Fourier series to a Maclaurin series, then compare coefficients. All these approaches would give you an understanding of Fourier, Maclaurin and Taylor series', which then lets you learn things that build upon them.

The "separate courses" approach is doable (and the best I've come up with so far), but it doesn't scale: courses that cover the same things without providing further insight don't get merged together. It can't handle the combinations that, intuitively, I feel like there should be a way to handle. Human teachers can cope with it just fine!

One nitpick on trane-math: running `cargo test` will wipe out the .trane directory, if it exists. That seems not very good; perhaps there should be a way to point trane at a temporary directory, so you can test it better?

Re:Bloom's 2 Sigma Problem by trane_project on Saturday February 25, 2023 @06:05PM (#34941361)
Thanks for pointing that out. I'll fix it, but for now just have the .trane directory in the folder right above.

I guess you are right. The textbooks definitely have dependencies, but that seems to be the very problem that you have with them. Unfortunately, I think other than different curriculums for each way to approach the subject, this is beyond the current scope of Trane. I made this thinking of music at first, which has a more rigid way of approaching the subject that tends to work for a wide set of students. There's no person on earth that will be comfortable studying highly chromatic music without mastering harmonically easier material first.

Re:Bloom's 2 Sigma Problem by wizzwizz4 on Saturday February 25, 2023 @07:18PM (#34941830)
Yeah, trane's design works really well for practical stuff like learning an instrument. It's probably good for language learning, too: perhaps a series of books or texts, each of which builds on the language skills learned from the previous…

It's a good idea to limit the scope of your projects. Jack of all trades is a master of none, and the fact trane exists (and is GPL'd) makes it just that bit easier for me to solve the other problem.

Re:Bloom's 2 Sigma Problem by admsmz on Saturday February 25, 2023 @05:32PM (#34941124)
This sounds very similar to the system behind http://mathacademy.com . You follow a graph of skills and dependencies and because you always have the necessary prerequisites it is always doable.

Their team of mathematicians have created a curriculum from k-12 to a bachelors in mathematics.

Re:Bloom's 2 Sigma Problem by trane_project on Saturday February 25, 2023 @05:43PM (#34941210)
Yeah, this is cool and similar to the end state I have in mind. The main difference is that I am not trying to create the curriculum/exercises. I am just creating flaschards that say "Solve exercise x.y.z in that textbook". Plus the whole free-software angle and being able to share the material as text files.

The math thing is just a side project at the moment to try to see how it works for other fields. I am primarily using this for music. I was hoping something like this existed already, but all the solutions are either very specialized like this one or do not support dependencies as a core feature, like Anki. Had Anki supported something like this, I wouldn't have needed to make my own thing.

Re:Bloom's 2 Sigma Problem by zozbot234 on Saturday February 25, 2023 @05:48PM (#34941249)
It would be nice to simply include support for dependency tracking in existing flashcard software. Unfortunately, the most commonly used flashcard software, viz. Anki, has not been adding such features. I can only assume that the project is very lightly maintained and/or working on paying down technical debt - new innovative features don't seem to be a priority.
Re:Bloom's 2 Sigma Problem by trane_project on Saturday February 25, 2023 @05:53PM (#34941294)
I think ultimately it's a very different approach to their current one, so it might be too difficult to do in the existing codebase. Maybe not, I have not looked at the code. But the public docs and learning philosophy around anki all seemed to discourage it, so I decided to make my own thing.

And yeah, I don't think Anki gets a lot of support. I am an open-source maintainer myself, so I know that story.

Re:Bloom's 2 Sigma Problem by dls2016 on Saturday February 25, 2023 @09:29PM (#34942473)
Agreed, I've also been waiting to see a system with spaced repetition (to help memorize and retain) + dependency graph (to choose what new topics to present). Not sure what the AI value add would be?

I have two children and I was glad that teachers are willing to use new tools (I never asked if they were forced to use them, but I assume not because each teacher seems to use different websites). I'm sure some kids still get bored, but they can let the ones who enjoy math practice at their own pace and provide special guidance to them while still spending the majority of their time helping those who need it.

Re:Bloom's 2 Sigma Problem by soferio on Saturday February 25, 2023 @05:50PM (#34941265)
This looks great. My son uses “beast academy” from Art of Problem Solving (https://beastacademy.com/), which is fantastic. But mathacademy looks like it might be a good competitor at the post year 6 level.
Re:Bloom's 2 Sigma Problem by sn9 on Saturday February 25, 2023 @07:48PM (#34941973)
It would probably be best to use the AoPS books in conjunction with the site as a supplement.
Re:Bloom's 2 Sigma Problem by soferio on Saturday February 25, 2023 @11:43PM (#34943108)
My observation is that the paper books are unnecessary because the online experience on beastacademy is so comprehensive - and your answers are auto checked. YMMV.
Re:Bloom's 2 Sigma Problem by squeaky-clean on Saturday February 25, 2023 @08:13PM (#34942106)
Mathacademy looks interesting but FYI for everyone else, this user only has 5 comments in their history and 4 are advertising mathacademy
Re:Bloom's 2 Sigma Problem by eps on Sunday February 26, 2023 @04:42AM (#34944301)
It's also $49/mo with no preview.
Re:Bloom's 2 Sigma Problem by ykonstant on Sunday February 26, 2023 @06:06AM (#34944654)
What in tarnation. For 100 bucks/month you can get 1-1 mastery learning sessions from math PhDs/postdocs that need money (like me! I am the postdoc!).

If you want to learn some math and are willing to fork over $50 a month, my advice is to find a competent tutor online*, tell them what your goals are, and work with them to produce an evolving, customized mini-course.

Have them pick material suitable for your goals, work through it in-session together so they can spot gaps in your knowledge/technique and make progressive problem sets to cover those gaps, and so on.

*A local tutor would be better, but I presume they will be prohibitively expensive in the average HN user's location.

Re:Bloom's 2 Sigma Problem by codetrotter on Sunday February 26, 2023 @07:23AM (#34944988)
> For 100 bucks/month you can get 1-1 mastery learning sessions from math PhDs/postdocs that need money (like me! I am the postdoc!).

But how many hours of tutoring will 100 bucks/month get me? Maybe 1 hour per week.

Re:Bloom's 2 Sigma Problem by ykonstant on Sunday February 26, 2023 @08:01AM (#34945188)
No, you can't do anything in 1h/week, 3 hours should be the minimum, 4 is ideal, depending on the level of the class (I would ask for more than 100 for 4 hours a week if I am to teach advanced harmonic analysis, for example).
Re:Bloom's 2 Sigma Problem by huhtenberg on Sunday February 26, 2023 @07:43AM (#34945088)
The problem is that finding a postdoc that is also a good _teacher_ is a monumental task of its own.

Compared to an online platform that's being used by a ton of people, has lots of reviews and recommendations and that is, presumably, actively optimized through a feedback loop, it's an easy choice between the two. Also, a platform removes the friction of cancelling the engagement if the need to do so arises for whatever reason.

I mean, if someone I know points at a postdoc and says "this guy is excellent", then their recommendation will prevail. But chances of that happening are next to zero.

Re:Bloom's 2 Sigma Problem by ykonstant on Sunday February 26, 2023 @08:24AM (#34945332)
I have no experience with those platforms, so I don't know how deep the instruction is. After taking, say, a Calculus course there, would you be comfortable doing Spivak's exercises? If yes, that seems great.
Re:Bloom's 2 Sigma Problem by huhtenberg on Sunday February 26, 2023 @12:43PM (#34947482)
This is orthogonal to the instruction quality. What good is the depth of the material if the teacher can't explain it clearly and concisely?
Re:Bloom's 2 Sigma Problem by ecshafer on Saturday February 25, 2023 @10:37PM (#34942811)
Kahn Academy has a similar graph and plenty of gamified experiences, but its a non-profit. $50 a month is expensive.
Re:Bloom's 2 Sigma Problem by throwoutway on Saturday February 25, 2023 @10:37PM (#34942812)
I'm in the market for something like this but $49/mo is absurd, even though it's supposedly discounted from 79/99. Way too high
Re:Bloom's 2 Sigma Problem by jxf on Sunday February 26, 2023 @05:19AM (#34944455)
It's not K-12; it starts at 4th grade, according to the site.
Re:Bloom's 2 Sigma Problem by huhtenberg on Sunday February 26, 2023 @07:37AM (#34945059)
How is it better than the Khan Academy courses?
Re:Bloom's 2 Sigma Problem by hyperturtle on Saturday February 25, 2023 @11:41PM (#34943090)
This is cool! I've had similar ideas, but when it comes to creating the curriculum it was always the time consuming part. If the graph was in something like a wiki, where people could contribute, add linkages, add new lessons, etc., and a system to automatically prune the graph to each student, there could be a globally sourced and maintained "school" for all human knowledge.
Re:Bloom's 2 Sigma Problem by trane_project on Sunday February 26, 2023 @03:56AM (#34944111)
The courses are markdown and JSON files in GitHub so close enough. It does restrict a bit because it assumes some technical proficiency, but there's probably some middle ground (open a GitHub issue to suggest changes and have someone else write the PR, for example).
Re:Bloom's 2 Sigma Problem by tgv on Sunday February 26, 2023 @07:27AM (#34945014)
Sorry, but how is that exactly a solution to the problem in the article? From what I can see, yours is just a (rather user unfriendly) guideline to self-study.
Re:Bloom's 2 Sigma Problem by trane_project on Sunday February 26, 2023 @10:55AM (#34946447)
open .

next

answer

score YOUR_SCORE

next

That's 80% of how to use it. Most of the rest is how to select individual parts of the graph to study. So other than setting it up and writing material for it (which I don't envision most people needing to do), I don't see how that's less user friendly than a website. But I understand how using a CLI is inherently limiting. Not enough time as a solo project so I am not focusing on that for now.

As for how it's a solution, see my answer to the other person that asked the same question.

Re:Bloom's 2 Sigma Problem by andrepd on Sunday February 26, 2023 @07:38AM (#34945061)
The effect is about how individually-tutored students consistently out-perform classroom students. How does a study guide, such as your program, help with that?
Re:Bloom's 2 Sigma Problem by trane_project on Sunday February 26, 2023 @10:46AM (#34946390)
The problem is bridging the gap between the ideal solution (everyone gets an individual tutor) that scales for everyone.

It's not meant to replace teachers, but it reduces the amount of individual teaching needed. In my own case, once I am done with this and can afford the time, I'll get a private music teacher and go through any tough spots I encountered since our last meeting.

Re:Bloom's 2 Sigma Problem by tgv on Sunday February 26, 2023 @04:02PM (#34949423)
Sorry, but you came in way too pretentious. A system like yours may support the student a bit, but it's in no way closing the gap with an individual tutor. Don't say it's a solution. It's an interactive homework list.
Re:Bloom's 2 Sigma Problem by trane_project on Monday February 27, 2023 @12:18AM (#34953278)
I never said I had solved it, only that I was working on solving it. Nowhere in the documentation or in my comments here I intended to give the impression this is anywhere close to done or intended for the general public at this time. There's an entire half of the project intended for instructors that I have barely mentioned publically or had had time to properly sketch out.

Mathacademy is pretty much the same thing from what I can gather and has already been applied very successfully in a school district. Same thing with the DARPA tutor also mentioned here.

All of the principles these tools are based on are already proven by mountains of research. And yet no free software, general solution exists that combines all of them. I know because I resisted working on this for months while looking for it.

And sure, it's an interactive homework list. I say so myself in the readme. One that automatically reviews skills at variable intervals depending on the mastery of the material, interleaves questions from separate topics for increased retention, gradually introduces new topics only after dependent knowledge is sufficiently mastered, balances the difficulties of the questions, and allows students to study specific portions of the graph.

So if we are giving each other advice on how we come across online, I find your comments lacking any criticism I can use to improve the project, and very dismissive of the hundreds of hours I have spent researching and working on the problem with the intention of giving all of my work away for free. If that ever changes and I charge a cent for an incomplete solution, your criticism will be more understandable.