yum-archive/TaSTT-Whisper
High-performance GPGPU inference of OpenAI's Whisper automatic speech recognition (ASR) model
git clone https://git.yummers.dev/yum-archive/TaSTT-Whisper
993cfa5
master
1 CPU Tasks 2LoadModel 827.449 milliseconds 3RunComplete 4.95485 seconds 4Run 4.90459 seconds 5Callbacks 343.6 microseconds, 4 calls, 85.9 microseconds average 6Spectrogram 12.0208 milliseconds, 3 calls, 4.00693 milliseconds average 7Sample 3.798 milliseconds, 28 calls, 135.643 microseconds average 8Encode 3.78211 seconds 9Decode 1.12187 seconds 10DecodeStep 1.11805 seconds, 28 calls, 39.9304 milliseconds average 11 GPU Tasks 12LoadModel 429.525 milliseconds 13Run 4.86319 seconds 14Encode 3.82894 seconds 15EncodeLayer 3.23846 seconds, 24 calls, 134.936 milliseconds average 16Decode 1.03424 seconds 17DecodeStep 1.03083 seconds, 28 calls, 36.8153 milliseconds average 18DecodeLayer 934.97 milliseconds, 672 calls, 1.39132 milliseconds average 19 Compute Shaders 20mulMatTiledEx 2.59201 seconds, 240 calls, 10.8 milliseconds average 21mulMatTiled 733.345 milliseconds, 289 calls, 2.53753 milliseconds average 22mulMatByRowTiled 492.505 milliseconds, 6507 calls, 75.6884 microseconds average 23mulMatByRowTiledEx 226.315 milliseconds, 1296 calls, 174.626 microseconds average 24softMaxFixed 169.603 milliseconds, 696 calls, 243.683 microseconds average 25convolutionMain2Fixed 130.131 milliseconds 26matReshapePanels 85.7723 milliseconds, 145 calls, 591.533 microseconds average 27addRepeatGelu 52.8833 milliseconds, 698 calls, 75.764 microseconds average 28copyConvert 49.8477 milliseconds, 1440 calls, 34.6165 microseconds average 29scaleInPlace 47.7803 milliseconds, 696 calls, 68.6499 microseconds average 30normFixed 44.0434 milliseconds, 2093 calls, 21.0432 microseconds average 31fmaRepeat1 38.6945 milliseconds, 2093 calls, 18.4876 microseconds average 32copyTranspose 36.6512 milliseconds, 1392 calls, 26.3299 microseconds average 33addRepeatEx 33.6887 milliseconds, 2064 calls, 16.322 microseconds average 34addRepeat 32.9016 milliseconds, 744 calls, 44.2226 microseconds average 35convolutionMain 32.8426 milliseconds 36softMaxLong 19.8753 milliseconds, 28 calls, 709.832 microseconds average 37addRepeatScale 15.8724 milliseconds, 1344 calls, 11.8098 microseconds average 38softMax 5.5277 milliseconds, 672 calls, 8.22574 microseconds average 39diagMaskInf 5.1549 milliseconds, 672 calls, 7.67098 microseconds average 40convolutionPrep2 3.9464 milliseconds, 2 calls, 1.9732 milliseconds average 41convolutionPrep1 3.4569 milliseconds, 2 calls, 1.72845 milliseconds average 42add 722.7 microseconds 43addRows 80 microseconds, 28 calls, 2.85714 microseconds average 44 Memory Usage 45Model 877.966 KB RAM, 1.42785 GB VRAM 46Context 1.9831 MB RAM, 771.235 MB VRAM 47Total 2.84049 MB RAM, 2.18101 GB VRAM