summaryrefslogtreecommitdiffstats
Commit message (Collapse)AuthorAge
* Bugfix: shader now respects bytes per charv0.3.1yum2023-01-22
|
* Enable using built-in chatboxv0.3yum2023-01-22
| | | | | | | | | | | | | VRChat exposes a built-in chatbox which can be seen by anyone who has it enabled. This was not the case when I started this project: the chatbox would only be visible to friends. Since this is clearly useful, enabling the STT on public models, let's enable sending data to it. Caveats: * The built-in chatbox has anti-spam tech which limits us to updating about once every 2 seconds. The custom chatbox has no such limitation and is thus typically much faster.
* GUI: Save Unity input fields across app restartsyum2023-01-22
| | | | | | | | | | | | | I found that I tend to regenerate the animator on the same avatar a lot, requiring me to re-enter the same paths and parameters over and over again. Persist them across restarts. * Refactor Config classes * Use safe `get_if` instead of the exception-throwing `operator>>` when deserializing from YAML * Begin sketching out Log singleton * Put Quote() and Unquote() into their own little lib; they shouldn't hide inside PythonApp
* GUI: Persist transcription app configv0.2yum2023-01-06
| | | | | | | | | | | | | | | The configuration of the transcription app, such as the number of rows and columns in the text box, now persists across app restarts. I found that I would have to change from the defaults to my preferred config every time I started up in VR, which was annoying. Now we just start with the config that was set last time. * Add dependency on rapidyaml (MIT) * Serialize transcription config to file under Resources/ * Add Config class to wrap serializing/deserializing * Update build instructions * Simplify StartApp() API, taking Config struct instead of a ton of arguments
* Bugfix: user-provided paths may now contain spacesyum2023-01-04
| | | | | | | | | | | | | | | | Previously, paths containing spaces would be interpreted by python's argument parser as multiple separate arguments, causing it to fail. Now we escape paths inside PythonWrapper using std::quoted(). * Improve PII filtering. Python output would contain multiple path separators (like C:\\Users\\foo\\), defeating the PII regex. * Silence compiler warning in PII filter. * Document usability improvements. * Transcription layer exponential backoff goes to ~infinity when paused. This is a hack, since we really don't need to transcribe at all when paused, but it lets us keep the code simple. Good enough until the next rewrite. * Shader only samples background when necessary. * Limit matchStrings() print()s to DEBUG mode
* Portability bugfixesyum2023-01-01
| | | | | * Expose option to run transcription engine on CPU instead of GPU * Use embedded git when setting up the Python virtual environment
* Embed git in packageyum2023-01-01
| | | | | | | package.ps1 fetches PortableGit and embeds it in the package. This eliminates all but one runtime dependency (MSVC++ Redistributable). * Move Python into a new FOSS folder.
* Statically link binaryyum2023-01-01
| | | | Update build instructions.
* Add Discord to README.mdyum2023-01-01
| | | | This seems like the best way to support users.
* Tweak paging logicv0.1yum2022-12-31
| | | | | | Re-paging anything on screen N causes screens N+1...infinity to completely re-page. This fixes cases where we go back and draw something at the bottom of the board, and it never gets overwritten.
* Bugfix: regions truncate correctly at page boundariesyum2022-12-30
| | | | | | | | Boards whose size is an even multiple of CHARS_PER_SYNC would lose the entire last region. * Attempt to fix runaway memory usage of GUI text frames, but this needs more work
* GUI: Update chars per sync defaultyum2022-12-30
| | | | The defaults now reflect what I typically use.
* GUI: Expose transcription window durationyum2022-12-30
| | | | | Users can pick longer transcription durations for accuracy-critical tasks, or shorter durations for latency-critical tasks.
* Bugfix: regenerated FX layers now work on uploaded avatarsyum2022-12-30
| | | | | | | | | | | | | VRChat won't update the FX layer associated with an avatar unless its GUID changes. Delete the GUID file when overwriting our generated FX layer to work around this. * Change paging behavior: when a region is updated, we re-page everything that comes after it. This fixes the issue where we go back to update something, then jump back to the current screen, leaving some random chunk of text somewhere on the board. * Reduce transcription time from 28s to 10s. I'm going to expose this to the user since there's a fundamental latency/stability tradeoff here.
* Fine-tune transcriptionyum2022-12-30
| | | | | | | | Bump up recording window to 28 seconds. This helps a lot with long-form transcription tasks, s.a. transcribing an audiobook. We should expose this as a parameter, since at 10s the transcription delay is typically 300ms, while at 28s it's typically 1.1-1.2s.
* GUI: Users can now control board dimensionsyum2022-12-29
| | | | | | | | Users can now control how many letters wide and tall the board is. Tested at 4x48, 5x60, 10x120, and 20x240. At 20x240, Unity freezes and does not make forward progress. Perhaps creating 4800 float parameters isn't a truly scalable interface.
* Add Scripts/generate_shader.pyyum2022-12-29
| | | | | | | | | | | | | | Now it's possible to generate shaders with a custom number of rows, columns, and bytes per character. All edits to the shader should go through TaSTT_template.shader. To generate a new shader from the template: $ ./Scripts/generate_shader.py \ --bytes_per_char 2 \ --rows 1 \ --cols 12 --shader_template $(pwd)/Shaders/TaSTT_template.shader \ --shader_path $(pwd)/Shaders/TaSTT.shader
* GUI: preview number of parameter bits the config will useyum2022-12-29
| | | | | Users can now see the number of avatar parameter bits they'll use prior to committing.
* First letter no longer disappearsyum2022-12-29
| | | | | | | | | An off-by-one issue in numRegions() would result in one extra layer trying to drive a letter in the last region, which would wrap back around to the 0th character slot (cell). * GUI explicitly logs when it's done generating avatar stuff * OSC layer no longer tries to update cells which don't exist
* Users can disable local beepyum2022-12-29
| | | | | | | | | | | | | | | The transcription engine beeps when you start/stop transcribing so you know that it's listening. Users can now disable this. * add help text to all input fields in GUI * make TaSTT generated file textctrls readonly, since I haven't tested them being reassigned * document idea to configure unity & transcription apps with config files * controller input thread no longer crashes if steamvr isn't running, it just slowly spins and waits * when you stop transcribing, the transcription engine re-transcribes a few times. I think this should improve end-of-transcription tail latencies * transcribe.py now prints out its args
* Encapsulate paging & text wrapping logicyum2022-12-27
| | | | | | | | Define proper interfaces for these things. Simplify osc_ctrl, temporarily dropping support for emotes (they were broken anyway). * Bugfix: Japanese no longer crashes transcribe.py, but it still doesn't show up in the wxTextCtrl
* Bugfix: transcribe panel respects chars per sync etc.yum2022-12-26
| | | | | The transcribe panel was grabbing data from the unity panel, causing the bytes per char / chars per sync parameters to be ignored.
* Bugfix: don't use last region if it's partialyum2022-12-25
| | | | | | | | | | Because we allow users to customize the # of sync params, the board is no longer divided into regions of uniform size. When the last region is a different size than the rest, we simply omit it from paging. This is a hack but it's easy to reason about. Of course the entire paging stack should be rewritten, but not today.
* Touch up TaSTT.shaderyum2022-12-25
| | | | | | | Add a new shader to make the box a little prettier. * Reduce material slots required from 2 to 1 * Add rounding to edge of box
* Make transcription sleeps interruptibleyum2022-12-24
| | | | | This reduces the expected delay to wake up the board & start transcribing from 750 milliseconds to 2.5 milliseconds.
* GUI: expose chars per sync, bytes per charyum2022-12-24
| | | | | | | | | | | | Users can now control how many characters they send per sync event, as well as the number of bytes used to represent each character. This gives them the power to pick between faster paging and fewer sync params. International users must use 2 bytes per char (at least for now). * package.ps1: don't distribute the gigantic TTF files, just the bitmaps
* Document encoding optimizationyum2022-12-22
| | | | | | | | | | | | | | | | By sending encoded words rather than letters, we could speed up English paging rate by 2.5x over an optimized implementation Word-encoded implementation: 16 bits per word (capped at 64k possible words). Optimized char-based imlementation: (5.7 chars per word) * (7 bits per char) == 39.9 bits per word 2.5x slower than word encoding. Today's char-based implementation: (5.7 chars per word) * (16 bits per char) == 91.2 bits per word 5.7x slower than word encoding.
* Update README.mdv0.0yum2022-12-22
|
* Quick hack: don't exponentially back off when unpausedyum2022-12-22
| | | | | This fixed some slowness I was seeing when waking up the STT. The right fix is to add interruptible sleeps. Let's fix this soon.
* Don't delete TaSTT_Generatedyum2022-12-21
| | | | | This makes incremental workflows much more efficient, since you don't have to reassign the FX controller, params, and menu.
* Add shader togglesyum2022-12-21
| | | | | | * Fix shader background rendering * Add ability to control margin size * Add ability to disable speech indicator
* GUI: Add better logging interfaceyum2022-12-21
| | | | | | | | Create printf-like interface for writing to wxTextCtrl objects. Also mask out PII. I wanted a way to not dox myself when recording demos, but I wound up making a second user on my PC to serve the same purpose. Maybe I'll delete the code later idk.
* Control tweak: introduce long/short hold behavioryum2022-12-20
| | | | | | | | | | | | The typical use pattern is now possible without entering radial. Leaving mounted to the world for a long time is no longer possible. Maybe I need an override param? Left joystick controls: * Short press toggle 1: show board, lock to hand, start transcribing * Short press toggle 2: lock to world, stop transcribing * Long press: hide board, stop transcribing
* Bugfix: animators may now include Unicode charactersyum2022-12-20
| | | | Completed first end-to-end test on a third party avatar :)
* Check in `World Constraint.prefab`yum2022-12-20
| | | | Can simply drag this into hierarchy & update reset target.
* GUI: "Finish" avatar generation workflowyum2022-12-20
| | | | | | | | | | | GUI now generates parameters & menu. Still need to handle write defaults. * Add capability to append to avatar parameters & menu * Install canned Unity assets, shaders, and fonts in avatar folder * Check in materials for ease of use * Bugfix: correctly label menu/parameters file pickers
* GUI can now generate animatoryum2022-12-20
| | | | Still need to generate params & merge menus. Getting close....
* GUI: Begin work generating animatoryum2022-12-20
| | | | The GUI can now generate guid.map and animations.
* GUI: Fix transcription outputyum2022-12-19
| | | | | | | | | | | Output now shows up in the textbox in ~real time. We do this by disabling Python's output buffering. This has a performance impact, but it should be negligible. * Fix crash when setting up python environment * UI tweak: text displays now expand with window * Fix how we merge transcribe.py; usually don't have to resort to SIGKILL, which loses stdout/stderr.
* GUI: Improve error loggingyum2022-12-19
| | | | | PythonWrapper correctly captures wxProcess stdout & stderr in sync and async execution modes.
* GUI: Sketch out Unity panelyum2022-12-19
| | | | | | | Now there are two panels: one to run transcription, one to generate avatar assets. Also, getting mics & python version can no longer crash the app.
* Now it's possible to build the app from Powershellyum2022-12-18
| | | | No more WSL dependencies!
* Add resource file headeryum2022-12-18
|
* Add ability to select modelyum2022-12-18
| | | | | | | * icon now works when pinned to taskbar * add model selection * add script to dump mic devices * whisper models now download into the virtual environment
* GUI: Add mic, language selectionyum2022-12-18
| | | | | | | | Users can now select their mic & spoken language in the GUI. * pyaudio now samples at the mic rate, fixing an issue where frames would drop. We downsample in the callback by dropping frames. * add Sounds folder to package
* GUI: Add ability to start & stop transcription engineyum2022-12-17
|
* Finish python virtual envyum2022-12-17
| | | | | | | | | GUI can now download all TaSTT dependencies and install them into a virtual environment. * Add buttons to check embedded python version & install dependencies * Add class to wrap interacting with embedded Python * Put all TaSTT python scripts into a folder
* Check in `future` packageyum2022-12-17
| | | | | | | | | | | I hit some issues installing Whisper and had to embed this package. I haven't taken the time to deeply understand what's going on. I think that embedded Python follows different rules about resolving module paths than regular system Python. Basically, `future`'s setup.py has a line like `import src`, where `src` is a module inside future (like `future/src/__init__.py`). This doesn't work unless we put that directory on the search path.
* Downgrade to Python 3.10.9yum2022-12-17
| | | | Whisper needs Python < 3.11.
* Document embedded venv hackyum2022-12-16
| | | | | | | Check in pip & modify embedded python to install to Lib and Lib/site-packages. Experimentally, packages may be installed with pip and do reside in Lib/site-packages. Hard to tell if this is also touching files outside the venv.
generated by cgit v1.2.3 (git 2.39.1) at 2026-06-01 21:24:47 +0000