summaryrefslogtreecommitdiffstats
path: root/Evaluate/README.md
blob: 91038f61f17294ba15f2feeb25bf53fee932e026 (plain)
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
## Evaluation tool

This directory holds code to evaluate the accuracy of transcriptions.

Example usage (in Powershell):
```
python3 -m pip install -r requirements.txt
python3 evaluate.py reference_transcript.txt
./setup.ps1
./WhisperCLI.exe \
  --audio_path *.mp3 \
  --model_path *.bin
```

* setup.ps1: Downloads whisper checkpoints and audio from librivox. Copies
  WhisperCLI.exe to CWD.
* evaluate.py: Computes Levenshtein distance between generated transcription
  and "correct" transcription.