Free self-hosted STT for VRChat. https://git.yummers.dev/TaSTT.git/atom?h=v0.13.0 2023-06-30T07:15:50+00:00 2023-06-30T07:15:50+00:00 yum-food 114886918+yum-food@users.noreply.github.com 2023-06-30T07:15:50+00:00 urn:sha1:ab1d4499e1b3198b387a1e2d7333a93f694cdfae Set GPU device index in whisper model 2023-06-30T00:15:30+00:00 jsopn github@jsopn.com 2023-06-30T00:15:30+00:00 urn:sha1:95927745b3d3b2ac566f1cfd634a40e760ed29cb 2023-06-29T06:50:34+00:00 yum yum.food.vr@gmail.com 2023-06-29T06:50:34+00:00 urn:sha1:3b10d3ab3073af2ed716d1607bb92394bb8817fc onAudioFramesAvailable would bail out if audio_state.audio_paused is set, preventing frames from being dropped. This would cause transcriptions to get repeated sometimes. Now that frame dropping code always runs. Also adjust the code structure of the keyboard/VR input handlers to be more similar. 2023-06-29T05:18:18+00:00 yum yum.food.vr@gmail.com 2023-06-29T05:11:46+00:00 urn:sha1:b1efbf5ce1ebd584796d4a57cf9c7b6517f91fac Audio data is stored in chunks of frames, not in individual frames. When I commit a transcript, I want to get rid of the portion of the audio data responsible for that particular transcript. I have code that does this, but it was dropping a slice of the list assuming that each sample is stored individually. Extra fun: Because we have to decimate mic frames, we have to convert between whisper frames and mic frames to drop the correct amount of audio data. 2023-06-29T04:24:56+00:00 yum yum.food.vr@gmail.com 2023-06-29T04:24:56+00:00 urn:sha1:bdaeb1911297d7901a12e3ac51b38c3463789279 Add toggle to UI to enable a profanity filter. It replaces vowels in bad words with asterisks. Bugfix: filters now apply to OBS 2023-06-29T03:35:10+00:00 yum yum.food.vr@gmail.com 2023-06-29T03:35:10+00:00 urn:sha1:ff7eb3c212195af71cd0ce4a3cd0c9a081d6ebda Most transcription output is now gone by default. Users can enable a more verbose output by toggling `Enable debug mode`. Bugfix: Toggling off transcription would reset audio state, frequently resulting in the loss of the last few words spoken. 2023-06-27T23:01:16+00:00 yum yum.food.vr@gmail.com 2023-06-27T23:01:16+00:00 urn:sha1:6638993e313773ba6ca8bdb6d7690b798d41f0d4 Recap: In the STT there's an algorithm that tries to determine when a transcript is "stable" enough to commit. If that is too loose, then accuracy suffers; if too strict, then the audio buffer eventually fills. To mitigate the problem, I check whether the last N transcripts are within some edit distance (Levenshtein edit distance) of each other. The fuzzy matching lets us forgive small instabilities, like differences in uppercase/lowercase or punctuation, while rejecting large instabilities. The default value of 8 seems to be in the sweet spot of accuracy & performance, but it will likely be tuned in the future. 2023-06-27T22:38:42+00:00 yum yum.food.vr@gmail.com 2023-06-27T22:35:30+00:00 urn:sha1:241813a5af11093c6b86e70ada729788c1f0dee6 ... instead of simple equality. TODO: add UI for threshold. Bugfix: Frame::onAppStop() joins the OBS app thread. 2023-06-27T22:16:06+00:00 yum yum.food.vr@gmail.com 2023-06-27T22:16:06+00:00 urn:sha1:cf75998dab6db1b1d21ca06bde18a56b5e896937 This is useful when streaming. Occasionally the STT can get into a bad state, and manually segmenting clears it up. However doing so would clear your accumulated transcript, which isn't always desired. Add ability to preserve the transcript. A small wrinkle: the new commit logic requires N consecutive identical windows before committing. To make this feature play nicely with it, I had to forcibly commit any preview text that hasn't yet been committed. Failing to do this would usually cause short utterances / the most recently said stuff to get wiped out. 2023-06-27T08:50:40+00:00 yum yum.food.vr@gmail.com 2023-06-27T08:47:53+00:00 urn:sha1:8ff761153b7de23a9556f2af179ab3bf4b9849a5 Should improve legibility. * Update README