Free self-hosted STT for VRChat. https://git.yummers.dev/TaSTT.git/atom?h=v0.13.2 2023-07-25T00:52:19+00:00 2023-07-25T00:52:19+00:00 yum yum.food.vr@gmail.com 2023-07-25T00:52:19+00:00 urn:sha1:2074749d4263991d835b19257bf1510dcaf55211 Useful for projects with multiple avatars with different animators. 2023-07-25T00:27:21+00:00 yum yum.food.vr@gmail.com 2023-07-25T00:27:21+00:00 urn:sha1:e64c71199c39fb1637bb2fd410cf27b9f4575b12 The paths you enter in the Unity panel (animator, menu, params, and assets folder) are saved in the app config, but were not populated correctly on app restart or pane redraw. Now they are. 2023-07-04T02:36:13+00:00 yum yum.food.vr@gmail.com 2023-07-04T01:44:43+00:00 urn:sha1:76ae7c28ea6224b2c919122d5dc71bcc00a0ecaa Create a simple server with 3 endpoints: * /create_session: Create a session and return its identifier. * /set_transcript: Update a session's transcript. * /get_transcript: Fetch a session's transcript. Right now the session ID provides authentication *and* authorization. There is no public/private ID so you have to trust whoever you share your ID with. IDs are long and generated by the server, so it should be somewhat secure against low-effort hacking. Other updates: * Drop whisper_requirements.txt - no longer needed. * Vendor curl to make it easier to interact with the server. TODO: * Fuzz test the server. 2023-07-01T02:46:17+00:00 yum yum.food.vr@gmail.com 2023-07-01T02:44:27+00:00 urn:sha1:4f3131b4a36d8e1557edb31d3754a431717dab7b Circle goes red when speaking, grey when done. Ideally it would be in the top right portion of the browser source, but this is a good start. Also, hard-cap transcripts to 4096 chars. This prevents the STT from lagging during long sessions. 2023-06-29T05:18:18+00:00 yum yum.food.vr@gmail.com 2023-06-29T05:11:46+00:00 urn:sha1:b1efbf5ce1ebd584796d4a57cf9c7b6517f91fac Audio data is stored in chunks of frames, not in individual frames. When I commit a transcript, I want to get rid of the portion of the audio data responsible for that particular transcript. I have code that does this, but it was dropping a slice of the list assuming that each sample is stored individually. Extra fun: Because we have to decimate mic frames, we have to convert between whisper frames and mic frames to drop the correct amount of audio data. 2023-06-29T04:24:56+00:00 yum yum.food.vr@gmail.com 2023-06-29T04:24:56+00:00 urn:sha1:bdaeb1911297d7901a12e3ac51b38c3463789279 Add toggle to UI to enable a profanity filter. It replaces vowels in bad words with asterisks. Bugfix: filters now apply to OBS 2023-06-29T03:35:10+00:00 yum yum.food.vr@gmail.com 2023-06-29T03:35:10+00:00 urn:sha1:ff7eb3c212195af71cd0ce4a3cd0c9a081d6ebda Most transcription output is now gone by default. Users can enable a more verbose output by toggling `Enable debug mode`. Bugfix: Toggling off transcription would reset audio state, frequently resulting in the loss of the last few words spoken. 2023-06-27T23:01:16+00:00 yum yum.food.vr@gmail.com 2023-06-27T23:01:16+00:00 urn:sha1:6638993e313773ba6ca8bdb6d7690b798d41f0d4 Recap: In the STT there's an algorithm that tries to determine when a transcript is "stable" enough to commit. If that is too loose, then accuracy suffers; if too strict, then the audio buffer eventually fills. To mitigate the problem, I check whether the last N transcripts are within some edit distance (Levenshtein edit distance) of each other. The fuzzy matching lets us forgive small instabilities, like differences in uppercase/lowercase or punctuation, while rejecting large instabilities. The default value of 8 seems to be in the sweet spot of accuracy & performance, but it will likely be tuned in the future. 2023-06-27T22:38:42+00:00 yum yum.food.vr@gmail.com 2023-06-27T22:35:30+00:00 urn:sha1:241813a5af11093c6b86e70ada729788c1f0dee6 ... instead of simple equality. TODO: add UI for threshold. Bugfix: Frame::onAppStop() joins the OBS app thread. 2023-06-27T22:16:06+00:00 yum yum.food.vr@gmail.com 2023-06-27T22:16:06+00:00 urn:sha1:cf75998dab6db1b1d21ca06bde18a56b5e896937 This is useful when streaming. Occasionally the STT can get into a bad state, and manually segmenting clears it up. However doing so would clear your accumulated transcript, which isn't always desired. Add ability to preserve the transcript. A small wrinkle: the new commit logic requires N consecutive identical windows before committing. To make this feature play nicely with it, I had to forcibly commit any preview text that hasn't yet been committed. Failing to do this would usually cause short utterances / the most recently said stuff to get wiped out.