News for hackers. Stuff that matters.
Re:How to write good prompts: using spaced repetition to cre... by kd5bjo on Friday January 01, 2021 @01:49AM (#25601403)
I find that writing spaced-repetition prompts in the way the article describes to have too small of a return on investment: Reviewing them is quick, but it takes a lot of effort to craft good ones from a text.

Good authors take care with their prose and use nuance to get their point across, which is often lost in the translation to a few bullet points. This subtlety is often the meat of the text, and the high-level points are the bones that hold it all together: essential, but not where the real value is.

My Anki deck is mostly cloze deletions automatically extracted from ebooks, where I've imported the entire text ~20 lines at a time. I've deliberately set up the review schedule so that it's hard for me to memorize particular phrases: 1 week for the first review and 4 weeks for the second, with a 1 day delay for misses. This forces me to remember what role the passage plays in the wider narrative (argument for non-fiction) to successfully predict the blanked-out word.

With this setup, I'm noticing new connections between different sections of the same book, and between different books, every time a card comes up for review: My understanding of the world has changed since I last saw it, and so I notice different subtleties than I did the first time around.

Instead of viewing them as a chore, I'm now actually excited to do my Anki reviews every morning-- where they used to make me mentally exhausted, they're now a warm-up that gets my brain started for the day.

Re:How to write good prompts: using spaced repetition to cre... by hyperpallium2 on Friday January 01, 2021 @03:58AM (#25601852)
> it takes a lot of effort to craft good ones

Though this engagement itself aids memorization...

I'm not a fan of spaced repetition, instead preferring to understand material and find connections (associations) integrating it within my existing knowledge. This makes it usable, not just recallable. It's holographic, in that I can also recover it from other things I know, like the shape of the missing jigsaw piece, or a theorem or proof.

However, I've come to realize that the very act of engagement to understand is, in itself, aiding memorization.

Re:How to write good prompts: using spaced repetition to cre... by kd5bjo on Friday January 01, 2021 @04:16AM (#25601919)
A key insight for me was when I realized that spaced repetition isn't really about memorization, it's about skill maintenance.

The skill most people use formal spaced-repetition methods for is factual recall, to be sure, but the underlying principles can be applied to any human endeavour. My Anki deck, for example, is designed to exercise reading comprehension. For cooking, I don't try to memorize recipes like the article demonstrates; I do, however, try to revisit and make the dishes in my repertoire semi-regularly to reinforce the operational cues that are impossible to put into words. It's similar for all of my other hobbies: I naturally tend to have a period of intense engagement with them at first that trails off as my confidence builds. I seem to revisit most of them on the same sort of exponential schedule without really trying, as if my subconscious understands when it's time to have a refresher.

Re:How to write good prompts: using spaced repetition to cre... by andymatuschak on Friday January 01, 2021 @12:44PM (#25604978)
(author of parent article here)

This is very interesting! Can you say more about the task you perform when you see these prompts? How do you decide how to "grade" your response? Presumably it's not just whether or not you can remember the deleted word(s)?

Also: are you blanking out individual words? Phrases? Sentences? An example would be interesting if you're willing to share!

Re:How to write good prompts: using spaced repetition to cre... by kd5bjo on Friday January 01, 2021 @01:20PM (#25605299)
Edit: Though my examples below are all fiction, that's half or less of my current reading list. I also have philosophy, research papers, long-form academic books, popular science books, and the like.

-----

For reference, here's the next new card for one of the books I'm reading:

  Bleak House
  Charles Dickens
  CHAPTER III
  ---------------------------------------------------------------------
  coach gave me a terrible start.
  
  It said, "What the de-vil are you crying for?"
  
  I was so frightened that I lost my voice and could only answer in a
  whisper, "Me, sir?" For of course I knew it must have been the
  gentleman in the quantity of wrappings, though he was still looking
  out of his window.
  
  "Yes, you," he said, turning round.
  
  "I didn't know I was crying, sir," I faltered.
  
  "But you are!" said the gentleman. "Look here!" He came quite
  opposite to me from the other corner of the coach, brushed one of his
  [...] furry cuffs across my eyes (but without hurting me), and showed
  me that it was wet.
  
  "There! Now you know you are," he said. "Don't you?"
  
  "Yes, sir," I said.
  
  "And what are you crying for?" said the gentleman, "Don't you want to
  go there?"
(The prior cards include the text up to and including the line starting "But you are!")

My response, having never seen the passage before, was "coat's"; the correct word is "large". The substitution doesn't meaningfully alter the passage: The only thing we know about the gentleman at this point is that he's a stranger whom Esther has just met in the coach, wearing a large coat (cloak?). This is the first time he speaks in the book.

I selected "again" because this is a new card; if this had been a review, I might have deemed it acceptable. The bar for similarity varies depending on the book the passage came from, my mood on any particular day, and how much I care about the contents of the passage. The Rime of the Ancient Mariner, being poetry, doesn't get any latitude: I must remember the word exactly. Some of the tedious battle descriptions in Le Morte d'Arthur get a pass no matter how badly they go wrong.

If a card is causing me trouble, there are a few corrective actions I can take: In some instances I'll edit the card to blank out a more meaningful word than the one randomly selected, which is an easier and more engaging review task. If I can't figure out the blanked word from context, I'll look it up in a dictionary; this happens most often with my second-language texts. And in extreme cases, I have enough coverage of each book that permanently dropping card or two won't cause problems.

The idea for this came from the paper[1] that introduced the concept of Cloze deletions. They were originally envisioned as a readability score. Instead of calculating statistics about the words used, like traditional methods, Taylor proposed to blank out every Nth word and measure how many could be guessed correctly by a reader. Readability and comprehension are two sides of the same coin. They both rely on observing the relationship between text and reader. If used to evaluate the text, this is called readability; if to evaluate the reader, comprehension.

[1] https://www.gwern.net/docs/psychology/writing/1953-taylor.pd...

Re:How to write good prompts: using spaced repetition to cre... by metaleks on Monday January 04, 2021 @04:25AM (#25629167)
I'm intrigued. I've read your comments in this thread, and I can't comprehend how you decide what to cloze on. Is this something that's decided by your script? Do you have multiple clozes per note? Why not cloze on "wrappings"?

If you have a blog post describing your process, I'm sure I, and many others, would love to read it.

Re:How to write good prompts: using spaced repetition to cre... by robador on Friday January 01, 2021 @04:52PM (#25607579)
I have so many questions, I am intrigued! Why are you doing this? Are you reviewing whole books in this fashion, as in the whole text? How long does that take? Do you read those books first? Are you reviewing in order of the order the text was written in? How much time do you spend reviewing?
Re:How to write good prompts: using spaced repetition to cre... by kd5bjo on Friday January 01, 2021 @05:43PM (#25608041)
First, the mechanics: I'm reviewing entire books that I haven't read before; I have a Python script that will split up a text file and produce a CSV file for Anki to import. New cards are set to come up in the original order, but reviews follow Anki's scheduling algorithm. Because I only have 1-2 new cards per book per day, the effect is similar to having several bookmarks: One is the most current, the second trails the first by a week, and the third by a month, etc.

A "normal" length book, like a genre fiction or popular science book, generally produces about 1000 cards like the one above. At a pace of 1 new card per day, that's about a three year commitment. It's a lot easier to have more books going than to increase the pace on any single book; the brain likes variety. I've got about 20 books going at once, which is around an hour of reading every day, including both reviews and new material. Overall, that works out to an average pace of 1 "standard" book every two months.

As for why, it started as an experiment to fix several problems I was having at the same time: My preexisting Anki deck was running dry, but still contained items that I wanted to keep reviewing; I needed a source of low-effort cards to keep the review habit going. I also had a long list of books that I should read someday but that day never seemed to be getting any closer; I decided to force the issue.

And finally, I had been unable to figure out how to make flashcards for literature at all. What series of questions / prompts can you write that captures the essence of something like this:

  It is an ancient Mariner,
  And he stoppeth one of three.
  'By thy long grey beard and glittering eye,
  Now wherefore stopp'st thou me?

  The Bridegroom's doors are opened wide,
  And I am next of kin;
  The guests are met, the feast is set:
  May'st hear the merry din.'

  He holds him with his skinny hand,
  'There was a ship,' quoth he.
  'Hold off! unhand me, grey-beard loon!'
  Eftsoons his hand dropt he.

  He holds him with his glittering eye—
  The Wedding-Guest stood still,
  And listens like a three years' child:
  The Mariner hath his will.

  The Wedding-Guest sat on a stone:
  He cannot choose but hear;
  And thus spake on that ancient man,
  The bright-eyed Mariner.

  (...)
Short of memorizing the poem word for word, it's hard to imagine any set of prompts that would adequately capture this: The events are almost incidental compared to the cadence and sound of the words. Or, using the Bleak House passage from above: If this gentleman becomes a recurring character in the book, I'm bound to learn more about his personality as the story progresses. If he exhibits some small behavior in this scene that foreshadows his character development later, how am I to notice it? If I have simply noted that he and Esther first meet on this trip and never revisit the actual text, I'll not have the opportunity to make that connection.
Re:How to write good prompts: using spaced repetition to cre... by robador on Friday January 01, 2021 @06:51PM (#25608615)
Thank you for your detailed response. I'm really tempted to try this as I'm already in the habit of using anki on a daily basis. Would you be willing to share your Python script?

If I may, how do you ensure you're only getting one or two cards per book? Do you create a deck per book, or is all in one deck and there's some trickery in the review settings you do?

Additionally, it's really interesting to me that there is a non-linear approach for the reviewing, what's the impact on your enjoyment of books? And, does this approach give you better retainment of details?

Finally, is the length of text set, and how did you determine what it should be?

Sorry for the interrogation

Re:How to write good prompts: using spaced repetition to cre... by andymatuschak on Saturday January 02, 2021 @07:41PM (#25618196)
This is quite striking. Thank you for sharing.
Re:How to write good prompts: using spaced repetition to cre... by edylemond on Friday January 01, 2021 @04:58AM (#25602120)
Really interesting how you extract cloze deletions from ebooks! It sounds very similar to what I'm building with Traverse.link, a social spaced repetition app with connected cards. You can import an article (working on ebook support), and it'll turn the paragraphs into cards, to which you can then add cloze deletions and free form prompts.