Universal Speech Model

Re:Universal Speech Model by thangalin on Wednesday March 29, 2023 @09:57PM (#35366778)

For my sci-fi story (alpha readers wanted; see profile), I used Whisper to transcribe an interview of a Malawian President. From there, I created a vocabulary comprised of only the president's words, which I used almost exclusively when writing his speech.

The results from Whisper are incredible, with very few mistakes. Though it did get Nelson Mandela's first name wrong (transcribed as Nesson). What's more, Whisper finished transcribing a 60-minute audio stream in 20 minutes on commodity hardware (T1000 G8 NVIDIA GPU). Broadly, here are the steps I used:

* Download and install podman.

* Download and install git.

* Download and install curl.

* Open a command prompt.

* Run the following commands to containerize Whisper:

    git clone https://github.com/lablab-ai/whisper-api-flask whisper
    cd whisper
    mv Dockerfile Containerfile
    podman build --network="host" -t whisper .
    podman run --network="host" -p 5000:5000 whisper

* Download MP3 file (e.g., filename.mp3).

* Run the following command to produce a transcription:

    curl -F "[email protected]" http://localhost:5000/whisper

[ Parent | on HN ]

Re:Universal Speech Model by 101008 on Wednesday March 29, 2023 @10:31PM (#35367115)

Whisper is great. You can get faster results running the tiny model. I used it for podcast transcription and it is much faster and the quality is not worse than the medium model - there are some podcast episodes that the transcription is the same.

[ Parent | on HN ]

Re:Universal Speech Model by alach11 on Wednesday March 29, 2023 @10:48PM (#35367253)

If speed is important, you're much better off using a larger model and whisper.cpp.

[ Parent | on HN ]

Re:Universal Speech Model by pantalaimon on Thursday March 30, 2023 @06:55AM (#35370746)

Wow thank you! That's a nice speedup indeed, with whisper I get

    33,53s user 2,05s system 443% cpu 8,023 total

with the 'tiny.en' model whereas whisper.cpp gives me

    22,71s user 0,12s system 745% cpu 3,062 total

with the 'base.en' model for a 15s audio clip on an i7-3770 (8 threads).

[ Parent | on HN ]

Re:Universal Speech Model by alach11 on Thursday March 30, 2023 @11:25AM (#35374153)

Awesome! Thanks for posting the stats.

In my workflows I've found rare but noticeable quality differences between the model sizes. So when practical I try to use the larger ones.

[ Parent | on HN ]

Re:Universal Speech Model by malborodog on Wednesday March 29, 2023 @10:39PM (#35367179)

why not just run whisper from the command line directly? Why put it into a docker container??

[ Parent | on HN ]

Re:Universal Speech Model by ec109685 on Wednesday March 29, 2023 @10:47PM (#35367251)

Why not keep everything tightly contained?

[ Parent | on HN ]

Re:Universal Speech Model by malborodog on Thursday March 30, 2023 @04:05AM (#35369534)

Hm, I'm on Mac so it takes up a bunch of ram and I'm not used to this workflow. good point though.

[ Parent | on HN ]

Re:Universal Speech Model by ec109685 on Friday March 31, 2023 @01:43AM (#35383079)

Unless you actually use the memory (e.g. allocate it), it won’t impact system performance, but yeah, it definitely is overhead.

[ Parent | on HN ]

Re:Universal Speech Model by pdntspa on Wednesday March 29, 2023 @11:05PM (#35367385)

some people just love making their environments needlessly complicated.

[ Parent | on HN ]

Re:Universal Speech Model by jazzyjackson on Thursday March 30, 2023 @03:03AM (#35369104)

complexity is in the eye of the beholder, some people just get docker enough that it's not a friction

Now installing the dependencies of every git repo I want to try on my host system, that's how an environment becoming needlessly complicated

[ Parent | on HN ]

Re:Universal Speech Model by hodanli on Wednesday March 29, 2023 @10:36PM (#35367154)

thank you for this

[ Parent | on HN ]