yum-archive/TaSTT-Whisper
High-performance GPGPU inference of OpenAI's Whisper automatic speech recognition (ASR) model
git clone https://git.yummers.dev/yum-archive/TaSTT-Whisper
993cfa5
master
1 CPU Tasks 2LoadModel 1.57347 seconds 3RunComplete 9.46787 seconds 4Run 9.40671 seconds 5Callbacks 292.1 microseconds, 4 calls, 73.025 microseconds average 6Spectrogram 12.2725 milliseconds, 3 calls, 4.09083 milliseconds average 7Sample 3.5692 milliseconds, 27 calls, 132.193 microseconds average 8Encode 7.26322 seconds 9Decode 2.14291 seconds 10DecodeStep 2.13933 seconds, 27 calls, 79.2343 milliseconds average 11 GPU Tasks 12LoadModel 991.351 milliseconds 13Run 9.36883 seconds 14Encode 7.35144 seconds 15EncodeLayer 6.25822 seconds, 32 calls, 195.569 milliseconds average 16Decode 2.01739 seconds 17DecodeStep 2.01737 seconds, 27 calls, 74.7173 milliseconds average 18DecodeLayer 1.88943 seconds, 864 calls, 2.18684 milliseconds average 19 Compute Shaders 20mulMatTiledEx 5.37127 seconds, 320 calls, 16.7852 milliseconds average 21mulMatTiled 1.17596 seconds, 385 calls, 3.05444 milliseconds average 22mulMatByRowTiled 878.04 milliseconds, 8346 calls, 105.205 microseconds average 23mulMatByRowTiledEx 460.074 milliseconds, 1664 calls, 276.486 microseconds average 24softMaxFixed 288.221 milliseconds, 896 calls, 321.675 microseconds average 25convolutionMain2Fixed 201.063 milliseconds 26norm 141.073 milliseconds, 2684 calls, 52.5606 microseconds average 27matReshapePanels 138.851 milliseconds, 193 calls, 719.436 microseconds average 28addRepeatGelu 89.1783 milliseconds, 898 calls, 99.3077 microseconds average 29copyConvert 83.2232 milliseconds, 1856 calls, 44.8401 microseconds average 30scaleInPlace 77.8363 milliseconds, 896 calls, 86.8709 microseconds average 31fmaRepeat1 77.8123 milliseconds, 2684 calls, 28.9912 microseconds average 32addRepeatEx 76.9018 milliseconds, 2656 calls, 28.954 microseconds average 33addRepeat 66.8479 milliseconds, 960 calls, 69.6332 microseconds average 34copyTranspose 62.5101 milliseconds, 1792 calls, 34.8829 microseconds average 35addRepeatScale 40.5807 milliseconds, 1728 calls, 23.4842 microseconds average 36convolutionMain 39.8186 milliseconds 37softMaxLong 32.0594 milliseconds, 27 calls, 1.18739 milliseconds average 38softMax 15.9281 milliseconds, 864 calls, 18.4353 microseconds average 39diagMaskInf 12.6164 milliseconds, 864 calls, 14.6023 microseconds average 40convolutionPrep1 5.4486 milliseconds, 2 calls, 2.7243 milliseconds average 41convolutionPrep2 4.0996 milliseconds, 2 calls, 2.0498 milliseconds average 42add 883.4 microseconds 43addRows 73.4 microseconds, 27 calls, 2.71852 microseconds average 44 Memory Usage 45Model 892.591 KB RAM, 2.8815 GB VRAM 46Context 1.98376 MB RAM, 1.13447 GB VRAM 47Total 2.85543 MB RAM, 4.01597 GB VRAM