yum-archive/TaSTT-Whisper
High-performance GPGPU inference of OpenAI's Whisper automatic speech recognition (ASR) model
git clone https://git.yummers.dev/yum-archive/TaSTT-Whisper
993cfa5
master
1 CPU Tasks 2LoadModel 939.886 milliseconds 3RunComplete 48.7479 seconds 4Run 48.6305 seconds 5Callbacks 10.5582 milliseconds, 37 calls, 285.357 microseconds average 6Spectrogram 280.966 milliseconds, 42 calls, 6.68965 milliseconds average 7Sample 65.5797 milliseconds, 511 calls, 128.336 microseconds average 8Encode 19.0653 seconds, 10 calls, 1.90653 seconds average 9Decode 29.5369 seconds, 10 calls, 2.95369 seconds average 10DecodeStep 29.4709 seconds, 511 calls, 57.6731 milliseconds average 11 GPU Tasks 12LoadModel 586.498 milliseconds 13Run 48.4589 seconds 14Encode 19.2258 seconds, 10 calls, 1.92258 seconds average 15EncodeLayer 15.7109 seconds, 240 calls, 65.4622 milliseconds average 16Decode 29.233 seconds, 10 calls, 2.9233 seconds average 17DecodeStep 29.233 seconds, 511 calls, 57.2074 milliseconds average 18DecodeLayer 27.6558 seconds, 12264 calls, 2.25504 milliseconds average 19 Compute Shaders 20mulMatTiled 19.1596 seconds, 5290 calls, 3.62186 milliseconds average 21mulMatByRowTiled 12.681 seconds, 144789 calls, 87.5829 microseconds average 22fmaRepeat1 3.10945 seconds, 37793 calls, 82.2758 microseconds average 23copyTranspose 2.83737 seconds, 25008 calls, 113.458 microseconds average 24softMaxFixed 1.6294 seconds, 12504 calls, 130.31 microseconds average 25addRepeatScale 1.54396 seconds, 24528 calls, 62.9467 microseconds average 26addRepeatEx 1.06992 seconds, 37272 calls, 28.7056 microseconds average 27normFixed 1.06753 seconds, 37793 calls, 28.2467 microseconds average 28copyConvert 994.495 milliseconds, 25488 calls, 39.0182 microseconds average 29convolutionMain2Fixed 954.715 milliseconds, 10 calls, 95.4715 milliseconds average 30softMax 742.126 milliseconds, 12264 calls, 60.5126 microseconds average 31addRepeatGelu 506.056 milliseconds, 12524 calls, 40.4069 microseconds average 32softMaxLong 491.226 milliseconds, 511 calls, 961.303 microseconds average 33scaleInPlace 438.956 milliseconds, 12504 calls, 35.1053 microseconds average 34addRepeat 403.997 milliseconds, 12984 calls, 31.1149 microseconds average 35diagMaskInf 366.713 milliseconds, 12264 calls, 29.9016 microseconds average 36convolutionMain 276.364 milliseconds, 10 calls, 27.6364 milliseconds average 37convolutionPrep1 44.9126 milliseconds, 20 calls, 2.24563 milliseconds average 38convolutionPrep2 20.0013 milliseconds, 20 calls, 1.00006 milliseconds average 39addRows 7.2369 milliseconds, 511 calls, 14.1622 microseconds average 40add 2.453 milliseconds, 10 calls, 245.3 microseconds average 41 Memory Usage 42Model 877.966 KB RAM, 1.42785 GB VRAM 43Context 91.0716 MB RAM, 785.219 MB VRAM 44Total 91.929 MB RAM, 2.19467 GB VRAM