Introduction
Whisper is a cutting-edge automatic speech recognition (ASR) system that has been meticulously
trained on a massive 680,000 hours of multilingual and multitask supervised data sourced from the
web. The
exceptional size and diversity of this dataset result in enhanced robustness to various factors such
as
accents, background noise, and technical language. Additionally, the system’s capabilities extend
beyond
transcribing speech in multiple languages to include translation of those languages into English. To
promote
useful applications and encourage further research on robust speech processing, the models and
inference code
have been made open-source.
The Whisper architecture is a simple end-to-end approach, implemented as an encoder-decoder
Transformer. Input
audio is split into 30-second chunks, converted into a log-Mel spectrogram, and then passed into an
encoder. A
decoder is trained to predict the corresponding text caption, intermixed with special tokens that
direct the
single model to perform tasks such as language identification, phrase-level timestamps, multilingual
speech
transcription, and to-English speech translation.
Let’s get our hands dirty!
How to use whisper with command line
If you want to use the Whisper command line interface (CLI) on your local machine, there are a
few prerequisites that you need to install first. Specifically, you’ll need to have Python, PyTorch,
and
FFmpeg installed on your system. Here’s how you can get started:
-
To install Python, head
over to the official Python website using the link below and follow the installation
instructions provided. -
Next, you’ll need to install PyTorch. To
do this, visit the PyTorch website using the link below and obtain the pip install command that
corresponds to your system’s compute platform.
- Finally, you’ll need to install
FFmpeg. To do this, you can visit the FFmpeg website using the link below and follow the
installation instructions provided. -
Alternatively, if you’re a Windows user, you can download and install the Chocolatey package
manager using the
link below. Once installed, use the command “choco install ffmpeg” to install FFmpeg on your
system.
choco install ffmpeg
Now that you have Python, PyTorch, and FFmpeg installed, you’re ready to install Whisper itself.
Simply run
the following command to install the latest version of the package:
pip install -U openai-whisper
Assuming that you have all your audio files in a single folder, here’s how you can use the Whisper
command
line interface to transcribe them:
- First, navigate to the folder that contains all your audio files.
- Open the command prompt in that folder.
- To transcribe a specific audio file, simply run the following command:
Note: First run will take a minute as it has to download the base model
whisper "file name"
Note that by default, Whisper will use the base model to transcribe the audio file. If you want to
use a
different model (tiny, base, small, medium, large ), you can specify it using the –model
argument. For
example, to use the medium model, you would run the following command:
whisper "file name" --model medium
Users can manually enter the language, if known.
whisper "file name" --language English
To see all the available options and commands for Whisper, you can run the following command:
whisper -help
Once file is transcribed, whisper will provide transcribed files in different formats like json,
srt, tsv, txt
and vtt
Find the original audio and the transcribed files in the link below.
Audio file:
Audio
File
Transcribed text file:
Transcribed
text
Transcribed JSON file:
Transcribed
JSON
Why whisper when you can use Whisper? It’s like having a language interpreter for your inner
thoughts – minus
the awkward misunderstandings.
Stay tuned for the next post where we’ll explore how to integrate Whisper into your Python programs.
With
Whisper’s powerful automatic speech recognition capabilities, the possibilities are endless!