ianbicking
In the more core experience they were super focused on not giving kids answers. I suppose built on a fear of subverting the teachers or classroom environment. But again it became a fixation that took all attention away from teaching itself.
But I figure you put something out in the world and see what happens. Yet I came back to it a year later and it was exactly the same. Nothing had changed. It baffled me then, and still baffles me... I had done enough to know the whole experience was easy to manipulate and yet I couldn't see any evidence of effort. It felt very much like "we tried nothing and it didn't work". Maybe I was missing something, I don't know.
I think there's a few different problems here:
1. I think the author is correct to highlight that underlying motivation, which is so important to teaching. We can focus on a high-minded Constructivist ideal of intrinsic motivation, but motivating students is also what grades are for, and classrooms, and learning with peers, having someone pay attention to your work, and so on. I do think that the AI chatbot is not capable of providing that. But the AI can still ENGAGE with that motivation. I have a feeling Khanmigo never did. I suspect some of their privacy controls also kept it from ever creating a good model of the learner.
2. The entire experience was very focused on answers, on solving problems. And yet it focused on that so reluctantly. The obsession with not giving away the answers had the unfortunate result in not engaging with anything but answers. It felt like it was always teasing at something it would withhold. But questions are cheap, especially rote questions, they shouldn't be treated as precious.
3. This may be unfair, but I infer the makers of Khanmigo did not build it with love and craft, which is unfortunate. I don't know enough to say why. Though there's a weird confidence to the product and to all of Khan Academy that I feel does not bring the necessary humility.
4. The chat experience was asymmetric in the wrong direction. In each turn the AI can and will spew out paragraphs of text, and the student types a couple words. Chatbots DO NOT have to work like this, they operate wonderfully with large chunks of user input. But you have to have the modality to receive it, the openness of experience to elicit that response, and the respect to make use of that response. Speech can be good, but it's a real challenge in a classroom environment (though if you are investing in Khanmigo, you can also invest in the other hardware to make it possible).
5. In some fairness, individual tutors don't represent any pedagogy at all. It's usually just someone making it up as they go. Motivation is a huge aspect, studying with someone sitting next to you, watching you study, will do wonders for keeping you focused! But also a tutor will have a natural theory of mind, hopefully the ability to see a student get the wrong answer and get inside the student's head to model their understanding and find a path to a correct understanding. That's a very challenging task, not beyond GPT-4 entirely but still quite challenging, especially if you are optimizing cost or time to response.
6. A good experience will do a TON of prework. I don't know if Khanmigo does that, if they've reshaped and expanded each educational module into a thorough playbook for the AI. That's where you have a chance to model things like wrong answers (there are actually SEVERAL books enumerating wrong answers in math), and use that to plan out responses.
If Khanmigo is not successful, I think that's on Khanmigo. Still teaching is very challenging. And maybe all of Khan Academy fails in this way: it does not seem to acknowledge the challenge of what they're trying to attempt. They seem far too confident in what they do, and not nearly creative enough.
And then finally both model output and human input become one world frame for the model, and the human adding a "you can do it!" isn't just input but a frame that colors not just the next step for the model, but also all previous steps (since at each step the model is viewing the totality of the transcript).
That this makes sense only makes it all the more absurd
It could work well with e-ink because it's basically not interactive; you don't press and hold and move around to adjust it just right, instead you can confidently get an exact scroll position with one tap.
Watching the report on the OpenAI/Hugging Face incident (https://www.youtube.com/watch?v=87DyyMV0kCY) it's easy to imagine how AI customer support reps could create their own knowledge bases where they trade tricks for how to work the system on behalf of the customer.
Obviously companies will fight this, but I expect it to happen along the way.
And it seems legitimately like an important problem for mathematics (and other areas), one which is deeply academic in the best sense: mathematicians spend much more time transmitting understanding than doing novel math, because they mostly exist in academia and that’s the job.
I'm mostly using my system to make comments on long AI-generated documents (especially design documents). I find it works well to have the AI generate something, and then I read through it, making comments along the way.
You can get pretty far just repeating the things you see... "I'm reading [heading] and [comments]". But I do find some use in selecting content and saying "I don't agree with this" or whatever else.
The result is just an augmented message. It looks like:
<transcript>
Let's see what we've got here.
<selection doc="proposal.md" location="paragraph 3">
The system already...
</selection>
No, I don't like how this is approaching the problem, ...
</transcript>
It makes reviewing feel pretty relaxed and natural. I've played around with similar note taking systems, which I think could be great for studying in school, but haven't had the focus on that particular problem to take it very far.
But I think the best thing really is giving the agent a richer understanding of what the user is experiencing and doing and just creating a rich representation of that. The keywords can be useful, but almost only as checkpoints: a keyword can identify the moment to take the transcript and package it up and deliver it.
One difference perhaps in design motivation: I have really embraced long latency interactions. I use ChatGPT with extended thinking by default, and just suck it up when the answer didn't really require thinking. I deliver 10 points of feedback at once instead of little by little. (Often halfway through I explicitly contradict myself, because I'm thinking out loud and my ideas are developing.) I just don't stress out about latency or feedback, and so low-latency but lower-intelligence interactions don't do it for me (such as ChatGPT's advanced voice mode, or probably Thinking Machine's work). I think this focus is in part a value statement: I'm trying to do higher quality work, not faster work.
If we do enough passes of synthetic or goal-based training of source code generation, where the models are trained to successfully implement things instead of imitating success, then we may see new programming paradigms emerge that were not present in any training data. The "new language" would probably not be a programming language (because we train on generating source FOR a language, not giving it the freedom to generate languages), but could be new patterns within languages.
It makes me think of the Book Of Ember, the possibility of chopping things out very deliberately. Maybe creating something that could wonder at its own existence, discovering well beyond what it could know. And then of course forgetting it immediately, which is also a well-worn trope in speculative fiction.
That is, it's nice to make a pretty stand-alone image, but without tools to maintain consistency and place them in context you can't make a project that is more than just one image, or one video, or a scattered and disconnected sequence of pieces.
I was fairly neutral about the tool for a while, but lately I've been going all-in on Claude Code, using things like rules and subagents.
It's also built to "rerender" the story, for instance rewriting it (slightly) for voice, translate it, or target different reading levels or background. I'm interested in translating stories for language learners in addition to simply translating into other native languages.
I'm also hoping to create some stories that stretch the medium. Perhaps CYOA (though I'm struggling with understanding what a CYOA is good at), though also other multi-perspective stories with reader autonomy in how to read through the story. LLMs make it easier to overproduce content, so you can give the reader flexibility without feeling regret that much of the content will be skipped, or rewrite passages for readers who jump into stories part way through.
Producing quality content is hard, and frankly kind of expensive, which is why I'm focused on finished products instead of interactive experiences. Though I do look forward to some future opportunity to take these rich characters that are grounded in full stories and find other things to do with them.
The thing I actually want to play with is probably some kind of board game that incorporates the printer... ideally with bar/QR codes so the computer can print out money, IOUs, instructions, etc., and have this computer mediation that still gives people physical items to manipulate.
But what LLMs do, in the absence of better instructions, is expect that the user WANTS the most middling innocuous output. Which is very reasonable! It's not a lack of capability; it's a strong capability to fill in the gaps in its instructions.
The person who has a good intuition for design (visual, narrative, conversational) but can't articulate that as instructions will find themselves stuck. And unsurprisingly this is common, because having that vision and being able to communicate that vision to an LLM is not a well practiced skill. Instructing an LLM is a little like instructing a person... but only a little. You have to learn it. And I don't think as LLMs get better that this will magically fix itself, because it's not exactly an error, there's no "right" answer.
Which is to say: I think applying design to one's work with AI is possible, important, and seldom done.
https://www.dustyrobotics.com/compare/fieldprinter-vs-sitepr...
The biggest improvement there is to treat the game engine as the "user", and the player (and their input) is merely one among many things the game engine is managing. But then you also need a game engine that manages lots of the state programmatically, with formal transitions of that state. The LLM can manage the state transitions, but the actual state needs to be more formal.
Doing experiments with LLMs and text adventures was revealing for me in this sense. An obvious thing to consider is using the LLM to parse the text... but if you try this you'll quickly realize that the parsers are mostly limited by what the parser _can parse into_. That is, the representation of a command is so limited that there's not a rich set of alternate inputs that would map to any valid command.
Before LLMs this also struck me in the voice assistant / NLP space, especially "natural language understanding" (NLU). The parsing wasn't great, but the thing-you-parse-into was also incredibly limited. Like you could parse "set an alarm for 8:30" into some template structure. But "no, change that to 8" didn't have a template structure, didn't have any structured representation.
What we've discovered is that the representation that actually fits these concepts is the chat log, or the somewhat magical discernment process of the LLM.
Unlike the point-and-click adventure, the text adventure has poor discoverability. This creates a fog where the player can imagine all kinds of possibilities. But the actual choice points are on the same order of magnitude as the hotspots, verbs, and inventory that define the choice points of a point-and-click adventure.
What I think the text adventure DOES accomplish (and the point-and-click adventure also accomplishes) is giving the player freedom of focus. You can look anywhere. You are usually in some open series of spaces where you can explore at leisure. The text adventure in particular offers a kind of tesseract opportunity, like in the flashback sequence shown in the article.
(Writing this, I am now thinking about a kind of LLM-driven game that discards all pretense of action or puzzles, but instead the player is a ghost free to view their environment, free even to view the internal thoughts of characters, but unable to change anything.)
It's possible to have a more structured substrate to an LLM text adventure, though also a lot of work... I wrote up my own thoughts on an experiment here: https://ianbicking.org/blog/2025/07/intra-llm-text-adventure
The default with LLMs are more collaborative storytelling than what we'd normally call a "game", but I think there's some new game genre waiting to be discovered.
I'm trying to use this to create stories that would be somewhat unreasonable to write otherwise. Branching stories (i.e., CYOA), multiperspective stories, some multimedia. I'm still trying to figure out the narrative structures that might work well.
LLMs can overproduce and write in different directions than is reasonable for a regular author. Though even then I'm finding branching hard to handle.
The big challenges are rhythm, pacing, following an arc. Those have been hard for LLMs all along.
So, prior art! :)
[1] https://www.cs.tufts.edu/comp/150FP/archive/alan-kay/smallta... (page 4)
Searching by embedding is just a way to construct queries, like ILIKE or tsvector. It works pretty nicely, but it's not distinct from SQL given pg_vector/etc.
The more distinctive feature here seems to be some kind of proxy (or monkeypatching?) – is it rewriting prompts on the way out to add memories to the prompt, and creating memories from the incoming responses? That's clever (but I'd never want to deploy that).
From another comment it seems like you are doing an LLM-driven query phase. That's a valid approach in RAG. Maybe these all work together well, but SQL seems like an aside. And it's already how lots of normal RAG or memory systems are built, it doesn't seem particularly unique...?
Maybe the best solution is just an eslint plugin. Like this plugin basically warns for the same thing on another type: https://github.com/bensaufley/eslint-plugin-preact-signals
Overloading .map() does feel a bit too clever here, as it has this major difference from Array.map. I'd rather see it as .mapRemote() or something that immediately sticks out.
I can imagine a RpcPromise.filterRemote(func: (p: RPCPromise) => RPCPromise) that only allows filtering on the truthiness of properties; in that case the types really would save someone from confusion.
I guess if the output type of map was something like:
type MapOutput = RpcPromise | MapOutput[] | Record<string, MapOutput>;
map(func: (p: RpcPromise) => MapOutput)
Though another case occurs to me that might not be caught by any of this:
result = aPromise.map(friend => {...friend, nickname: getNickname(friend.id, userId)})
I'm trying to see if there's something specifically for streaming/generators. I don't think so? Of course you can use callbacks, but you have to implement your own sentinel to mark the end, and other little corner cases. It seems like you can create a callback to an anonymous function, but then the garbage collector probably can't collect that function?
---
I don't see anything about exceptions (though Error objects can be passed through).
---
Looking at array mapping: https://blog.cloudflare.com/capnweb-javascript-rpc-library/#...
I get how it works: remotePromise.map(callback) will invoke the callback to see how it behaves, then make it behave similarly on the server. But it seems awfully fragile... I am assuming something like this would fail (in this case probably silently losing the conditional):
friendsPromise.map(friend => {friend, lastStatus: friend.isBestFriend ? api.getStatus(friend.id) : null})
The array escape is clever and compact: https://blog.cloudflare.com/capnweb-javascript-rpc-library/#...
---
I think the biggest question I have is: how would I apply this to my boring stateless-HTTP server? I can imagine something where there's a worker that's fairly simple and neutral that the browser connects to, and proxies to my server. But then my server can also get callbacks that it can use to connect back to the browser, and put those callbacks (capability?) into a database or something. Then it can connect to a worker (maybe?) and do server-initiated communication. But that's only good for a session. It has to be rebuilt when the browser network connection is interrupted, or if the browser page is reloaded.
I can imagine building that on top of Cap'n Web, but it feels very complicated and I can equally imagine lots of headaches.