yum-archive/TaSTT-Whisper

High-performance GPGPU inference of OpenAI's Whisper automatic speech recognition (ASR) model

git clone https://git.yummers.dev/yum-archive/TaSTT-Whisper

yumBegin work on evaluation frameworkf7d5741

master
branchesmasterall refs →
502 B18 linesraw

Evaluation tool

This directory holds code to evaluate the accuracy of transcriptions.

Example usage (in Powershell):

python3 -m pip install -r requirements.txt
python3 evaluate.py reference_transcript.txt
./setup.ps1
./WhisperCLI.exe \
  --audio_path *.mp3 \
  --model_path *.bin
  • setup.ps1: Downloads whisper checkpoints and audio from librivox. Copies WhisperCLI.exe to CWD.
  • evaluate.py: Computes Levenshtein distance between generated transcription and "correct" transcription.
1## Evaluation tool
2
3This directory holds code to evaluate the accuracy of transcriptions.
4
5Example usage (in Powershell):
6```
7python3 -m pip install -r requirements.txt
8python3 evaluate.py reference_transcript.txt
9./setup.ps1
10./WhisperCLI.exe \
11  --audio_path *.mp3 \
12  --model_path *.bin
13```
14
15* setup.ps1: Downloads whisper checkpoints and audio from librivox. Copies
16  WhisperCLI.exe to CWD.
17* evaluate.py: Computes Levenshtein distance between generated transcription
18  and "correct" transcription.