diff options
| author | yum <yum.food.vr@gmail.com> | 2024-07-12 14:48:36 -0700 |
|---|---|---|
| committer | yum <yum.food.vr@gmail.com> | 2024-07-12 14:48:36 -0700 |
| commit | 75069522ffc8863a356d95e509c81612a3703458 (patch) | |
| tree | 8abebad0dc35facf668a5a1777b7a4bcd1b5085c /Scripts/cpp_transcribe.py | |
| parent | fef509ce622641079c08c5c9e602315e8d207868 (diff) | |
Fix translation plugin
Translation needs torch to convert the nllb model, but the latest
version (2.3.1) has an embedded OMP dll which clashes with ctranslate2's
dll. Using the last minor version instead (2.2.2) doesn't clash.
Also propagate the device, quantization, and flash attention settings
to the translator. If you're using GPU, this is a HUUUUGE performance
uplift. Translation is basically instant. The bigger models are now
feasible to use.
Diffstat (limited to 'Scripts/cpp_transcribe.py')
0 files changed, 0 insertions, 0 deletions