samuelcole

i did end up open-sourcing the aligner! https://github.com/samuelcole/tale-align

i really loved this description, and i wrote up the technique into a full tale:

https://tale.fyi/@sam/aligning-800-audiobooks-8-000-hours-to...

i actually wrote this up (and narrated it, ha) into a full tale: https://tale.fyi/@sam/aligning-800-audiobooks-8-000-hours-to...

there's a subtle line on the left that should track vaguely where you are in the paragraph, i added a little easter egg yesterday: if you hold the play button the indicator grows to highlight the lines in the text, so that may help

but in general: noted that it's hard to follow along!

just did the explicit 'read from here' line, to set the watermark precisely (it also controls where the play button starts from)

check it out! let me know what you think

got it working! https://tale.fyi/moby-dick

it got confused by some of the more uhh complex passages ;-)

i think i fixed it? let me know

i love that you're using the PWA... i love PWAs, so i added fun stuff like offline support, but i wasn't sure if anyone would find it

shipped! look for a sun/moon icon near the tale.fyi title at the top

yeah right now you have to use browser zoom, but noted!

good news: i already store the word level alignments, i just felt like words lighting up were distracting

i might add this as an option in the future, thanks!

oh, i've disliked passwords for my entire career i think:

1. everyone uses the same string

2. if they don't use the same string, they forget which password goes to which site, which is really frustrating

3. if they do use the same string, someone guesses it and the user feels 'hacked'

4. what's up with magic words proving identity anyways?

i personally feel like your computer should know who you are and then pass along that identity to the website, and passkeys get close to this (which i haven't implemented quite yet here)

good catch, fixed: orientation: 'any' was meant as 'don't lock the phone' and android treated it the exact opposite. removed it

sadly it may take a bit for your phone to pick up the new manifest, an uninstall and reinstall might fix it

i used to run a literary journal in college, so i built that dna into tale as 'lists', so that curators can be followed and they can create lists of their favorite work. i dunno if it creates a gallery type environment, but that's the dream

dark souls: in dark souls as you wander around you can leave little notes around the world, and then the game will surface them to other players, almost randomly. players can vote on notes, which makes them appear more or less often. dark souls is a famously obtuse game, so the notes are genuinely helpful sometimes, and sometimes they are trolling you. but it makes the whole experience less lonely. i wonder if that's what people are missing when they read a book: the book _club_ feeling.

i actually had an out-of-memory bug, which i've since fixed. chewing through it now to see if we can get it

thanks! i really appreciate that you get it: 'whispersync for the open web' is a really great description, i might use it ;-)

it follows your browser/device preference, but noted that a manual override would be a useful setting!

awww thanks, you get it

noted for next time: the great fiction deserves something, we may not

you wrote brat! i 'own' your book (on kindle, ha, so i'm not sure if that counts as real ownership), and it's on my to-read pile

anyways, i'm going to send you an email

i filed tickets for myself for the play button weirdness, thanks for including the browser, it might be related, and the product feedback around sign up (i didn't want to be too pushy, maybe went too far in the other direction)

for your thinking:

global user highlighting is basically half-done: tap any paragraph to leave a note, however the only way to share that note is with links... tale.fyi needs some way to discover other people's notes. did you ever play dark souls?

i also like integrating other forms of criticism from around the web, hmmm

you don't care about user-written fiction? interesting from you, since you know all the mechanics... is it because it's unedited basically? like when does fiction cross from user-generated to commercial? is it curation?

not right now! i'm still deciding what i want to do with this

tbh, the most exciting thing is the aligner, i might extract that into an open source project... that's a good idea

i know, i'm being a bit emotional here, ha

i'm mostly reacting to the default thing that people scroll are reddit, facebook, instagram, ahem hn

which are all non-fiction... i wanted to build something easy to scroll in line at the grocery store that was fiction

well! (i've been waiting for someone to ask ha)

when i first knew i wanted to align audio to text, i reached for aeneas, which was built for this exact problem/product (https://www.readbeyond.it/aeneas/)

however, it drifted by hundreds of seconds deep into chapters and emits no confidence signal, so i couldn't eval it cleanly

so i switched to a neural acoustic model trained with ctc (meta's massively multilingual speech aligner, it ships inside torchaudio): it outputs a probability for each character at every frame of audio, and viterbi decoding finds the most probable path that spells exactly our text, with a per-word probability

that probability is the gate: a book ships read-along only if its median confidence and coverage clear a bar. moby dick's didn't, so it's honestly text-only

added a link to sign in on https://tale.fyi/tell

good catch, thanks

moby dick didn't align properly, there's a confidence gate it didn't pass.

i know, it makes me sad too, i'll keep working on it

this is the list of everything: https://tale.fyi/audiobooks

librivox is all human volunteers, so i think it makes a difference from eleven labs, but i might add a fallback to ai readings for folks who want that

reset feedback noted!

do you think there's space for that? i really wanted to build that too, but there is already goodreads, fable, storygraph, bookly, etc

the aligner already supports french (it's meta's massively multilingual speech model under the hood), so it would all work today, but i started on standard ebooks, which is english-only.

but gutenberg and librivox both have french libraries so i could absolutely do that next!

i've been building this over the last two weeks, it first started as a reader for standard ebooks (https://standardebooks.org/), with clean typography, progress tracking, and nice deep linking (try clicking on a paragraph to create a mark and copy it to your clipboard)

but i think i really unlocked something when i discovered the amazing recordings on https://librivox.org, i've been spending the last few days frying my macbook's gpu to do neural alignment so that you can sync an audiobook with its text counterpart, to make switching between the text and the audio seamless

it's a ctc forced-alignment over the whole audiobook, keyed to the same paragraph anchors i used for progress tracking and linking

i think it's really cool? let me know

It's just a good 'ol download-and-play. I don't expect we need streaming, but bitrate switching would be interesting.