yum-archive/TaSTT-Whisper
High-performance GPGPU inference of OpenAI's Whisper automatic speech recognition (ASR) model
git clone https://git.yummers.dev/yum-archive/TaSTT-Whisper
993cfa5
master
1 CPU Tasks 2LoadModel 1.79203 seconds 3RunComplete 8.79853 seconds 4Run 8.71884 seconds 5Callbacks 626.8 microseconds, 4 calls, 156.7 microseconds average 6Spectrogram 17.3373 milliseconds, 3 calls, 5.7791 milliseconds average 7Sample 5.449 milliseconds, 28 calls, 194.607 microseconds average 8Encode 7.29966 seconds 9Decode 1.41824 seconds 10DecodeStep 1.41276 seconds, 28 calls, 50.4557 milliseconds average 11 GPU Tasks 12LoadModel 930.123 milliseconds 13Run 8.64946 seconds 14Encode 7.34021 seconds 15EncodeLayer 6.40759 seconds, 24 calls, 266.983 milliseconds average 16Decode 1.30925 seconds 17DecodeStep 1.30389 seconds, 28 calls, 46.5676 milliseconds average 18DecodeLayer 1.19422 seconds, 672 calls, 1.77711 milliseconds average 19 Compute Shaders 20mulMatTiledEx 4.91773 seconds, 240 calls, 20.4906 milliseconds average 21mulMatTiled 1.47531 seconds, 289 calls, 5.10489 milliseconds average 22mulMatByRowTiled 627.1 milliseconds, 6507 calls, 96.3731 microseconds average 23softMaxFixed 268.285 milliseconds, 696 calls, 385.467 microseconds average 24convolutionMain2Fixed 266.261 milliseconds 25mulMatByRowTiledEx 241.609 milliseconds, 1296 calls, 186.427 microseconds average 26matReshapePanels 156.683 milliseconds, 145 calls, 1.08057 milliseconds average 27convolutionMain 102.091 milliseconds 28copyConvert 77.6113 milliseconds, 1440 calls, 53.8967 microseconds average 29addRepeatGelu 71.5118 milliseconds, 698 calls, 102.452 microseconds average 30copyTranspose 63.3929 milliseconds, 1392 calls, 45.5409 microseconds average 31normFixed 60.9615 milliseconds, 2093 calls, 29.1264 microseconds average 32scaleInPlace 59.9341 milliseconds, 696 calls, 86.1122 microseconds average 33fmaRepeat1 56.3539 milliseconds, 2093 calls, 26.9249 microseconds average 34addRepeatEx 51.8785 milliseconds, 2064 calls, 25.1349 microseconds average 35addRepeat 48.1192 milliseconds, 744 calls, 64.6763 microseconds average 36softMaxLong 28.3411 milliseconds, 28 calls, 1.01218 milliseconds average 37addRepeatScale 21.3646 milliseconds, 1344 calls, 15.8963 microseconds average 38softMax 10.198 milliseconds, 672 calls, 15.1756 microseconds average 39convolutionPrep1 9.1072 milliseconds, 2 calls, 4.5536 milliseconds average 40diagMaskInf 8.3764 milliseconds, 672 calls, 12.4649 microseconds average 41convolutionPrep2 7.4623 milliseconds, 2 calls, 3.73115 milliseconds average 42add 2.3886 milliseconds 43addRows 97.6 microseconds, 28 calls, 3.48571 microseconds average 44 Memory Usage 45Model 877.966 KB RAM, 1.42785 GB VRAM 46Context 1.9831 MB RAM, 771.235 MB VRAM 47Total 2.84049 MB RAM, 2.18101 GB VRAM