yum-archive/TaSTT-Whisper
High-performance GPGPU inference of OpenAI's Whisper automatic speech recognition (ASR) model
git clone https://git.yummers.dev/yum-archive/TaSTT-Whisper
993cfa5
master
1 CPU Tasks 2LoadModel 2.20693 seconds 3RunComplete 3.16174 seconds 4Run 3.07912 seconds 5Callbacks 387.3 microseconds, 4 calls, 96.825 microseconds average 6Spectrogram 16.201 milliseconds, 3 calls, 5.40033 milliseconds average 7Sample 3.3725 milliseconds, 28 calls, 120.446 microseconds average 8Encode 2.07037 seconds 9Decode 1.00834 seconds 10DecodeStep 1.00495 seconds, 28 calls, 35.8911 milliseconds average 11 GPU Tasks 12LoadModel 1.81217 seconds 13Run 2.94117 seconds 14Encode 1.95373 seconds 15EncodeLayer 1.56747 seconds, 24 calls, 65.3115 milliseconds average 16Decode 987.441 milliseconds 17DecodeStep 987.44 milliseconds, 28 calls, 35.2657 milliseconds average 18DecodeLayer 915.401 milliseconds, 672 calls, 1.3622 milliseconds average 19 Compute Shaders 20mulMatTiled 1.68817 seconds, 529 calls, 3.19125 milliseconds average 21mulMatByRowTiled 562.722 milliseconds, 7803 calls, 72.1161 microseconds average 22convolutionMain2Fixed 99.873 milliseconds 23softMaxFixed 84.045 milliseconds, 696 calls, 120.754 microseconds average 24copyTranspose 80.3619 milliseconds, 1392 calls, 57.7313 microseconds average 25fmaRepeat1 71.9629 milliseconds, 2093 calls, 34.3827 microseconds average 26convolutionMain 60.588 milliseconds 27addRepeatScale 53.2349 milliseconds, 1344 calls, 39.6093 microseconds average 28normFixed 34.7651 milliseconds, 2093 calls, 16.6102 microseconds average 29addRepeatEx 31.9206 milliseconds, 2064 calls, 15.4654 microseconds average 30copyConvert 30.7856 milliseconds, 1440 calls, 21.3789 microseconds average 31addRepeatGelu 25.5167 milliseconds, 698 calls, 36.5569 microseconds average 32softMaxLong 25.3214 milliseconds, 28 calls, 904.336 microseconds average 33scaleInPlace 24.1527 milliseconds, 696 calls, 34.7022 microseconds average 34softMax 21.0692 milliseconds, 672 calls, 31.353 microseconds average 35addRepeat 19.8584 milliseconds, 744 calls, 26.6914 microseconds average 36diagMaskInf 12.5615 milliseconds, 672 calls, 18.6927 microseconds average 37convolutionPrep1 6.113 milliseconds, 2 calls, 3.0565 milliseconds average 38convolutionPrep2 1.2294 milliseconds, 2 calls, 614.7 microseconds average 39add 532.9 microseconds 40addRows 178.9 microseconds, 28 calls, 6.38929 microseconds average 41 Memory Usage 42Model 877.966 KB RAM, 1.42785 GB VRAM 43Context 1.9831 MB RAM, 723.782 MB VRAM 44Total 2.84049 MB RAM, 2.13467 GB VRAM