TaSTT-Whisper.git, branch master High-performance GPGPU inference of OpenAI's Whisper automatic speech recognition (ASR) model https://git.yummers.dev/TaSTT-Whisper.git/atom?h=master 2023-04-05T00:41:30+00:00 Fix audio normalization 2023-04-05T00:41:30+00:00 yum yum.food.vr@gmail.com 2023-04-05T00:40:59+00:00 urn:sha1:59297502afb8f61c1216c6d57d6cc18ab5b9f467 Normalization was putting audio onto range [0, 255], while it should have been on range [0, 1]. * Add AudioBuffer::save() to enable debugging audio issues. begin work disabling vad 2023-03-17T11:11:18+00:00 yum yum.food.vr@gmail.com 2023-03-17T11:11:18+00:00 urn:sha1:aaa0188da81056748ef8ffcd5ad86d6f4bffa6bd Fix beam search previous window conditioning 2023-03-08T04:22:56+00:00 yum yum.food.vr@gmail.com 2023-03-08T04:16:37+00:00 urn:sha1:5e30b2366a4a320f59ed7e0bfcfe72f5f8c9d108 Not all contexts had `prev_prompt`, causing most beams to misbehave. Use logprobs, fix beam candidate selection 2023-03-04T04:42:10+00:00 yum yum.food.vr@gmail.com 2023-03-04T04:10:32+00:00 urn:sha1:a74ee78dbb79c551851dc182090e7d4292b1e80c Incorrect sort condition resulted in worst 5 beams being picked instead of best 5. Use log probabilities for joint probability calculation instead of linear probabilities. Long beams would have probabilities converge exponentially towards zero; now they converge linearly towards -INFINITY. Using both transcripts in Evaluation/setup.ps1, I see a small edit distance regression (~5%) using beam search vs. greedy. Begin work on evaluation framework 2023-03-03T23:43:24+00:00 yum yum.food.vr@gmail.com 2023-03-03T23:43:24+00:00 urn:sha1:f7d5741e5c069d759f8412bd40b279e1d7abac4c Need a way to verify that beam search is actually working better than greedy. Finish beam search rough draft 2023-03-03T04:33:22+00:00 yum yum.food.vr@gmail.com 2023-03-03T04:32:49+00:00 urn:sha1:d743645ba27cc85d36fe6820cd9d21f0fc4a11f2 Seems to work. Doesn't crash. Lots of room for optimization and cleanup. Continue work on beam search 2023-03-03T01:43:26+00:00 yum yum.food.vr@gmail.com 2023-03-03T01:43:26+00:00 urn:sha1:dcd7f3b60e3b9ad8df83d444f8bc67091b411529 Define ContextImpl::Context, wrapping all the data used in decoding. Using a vector of these is much simpler than using N vectors of all the random stuff we need. Begin work on beam search decoding 2023-02-28T02:03:02+00:00 yum yum.food.vr@gmail.com 2023-02-28T02:03:02+00:00 urn:sha1:4f967dbdb7972ec52039bd3e3ce3e1e4cbcf6545 * ContextImpl.h puts prompts, previous prompts, probabilities, and probability IDs into vectors of size 1 or N_BEAMS, depending on the decoding strategy. * Extend sampleBest and friends to return top N tokens, instead of just the top 1 token. Add retainDuration option to CaptureParams 2023-02-27T04:09:15+00:00 yum yum.food.vr@gmail.com 2023-02-27T03:42:33+00:00 urn:sha1:1136acfc365f357d2df13a263714e8ae0614c4f9 This allows users to retain a suffix of the PCM buffer after a VAD segmentation event, reducing some instances of words being lost at the start of the next VAD window. Normalize audio before sending to transcription layer 2023-02-27T04:08:45+00:00 yum yum.food.vr@gmail.com 2023-02-27T03:12:01+00:00 urn:sha1:02c2605454288f7c86023ae700366acf08cd2206 Helps in cases where the speaker is speaking softly, or their mic gain is set low.