| Commit message (Collapse) | Author | Age |
|---|
| |
|
|
|
|
|
|
|
|
|
|
|
| |
VRChat exposes a built-in chatbox which can be seen by anyone who has
it enabled. This was not the case when I started this project: the
chatbox would only be visible to friends. Since this is clearly useful,
enabling the STT on public models, let's enable sending data to it.
Caveats:
* The built-in chatbox has anti-spam tech which limits us to updating
about once every 2 seconds. The custom chatbox has no such limitation
and is thus typically much faster.
|
| |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
Previously, paths containing spaces would be interpreted by python's argument
parser as multiple separate arguments, causing it to fail. Now we escape paths
inside PythonWrapper using std::quoted().
* Improve PII filtering. Python output would contain multiple path separators
(like C:\\Users\\foo\\), defeating the PII regex.
* Silence compiler warning in PII filter.
* Document usability improvements.
* Transcription layer exponential backoff goes to ~infinity when paused.
This is a hack, since we really don't need to transcribe at all when paused,
but it lets us keep the code simple. Good enough until the next rewrite.
* Shader only samples background when necessary.
* Limit matchStrings() print()s to DEBUG mode
|
| |
|
|
|
| |
* Expose option to run transcription engine on CPU instead of GPU
* Use embedded git when setting up the Python virtual environment
|
| |
|
|
|
|
| |
Re-paging anything on screen N causes screens N+1...infinity to
completely re-page. This fixes cases where we go back and draw something
at the bottom of the board, and it never gets overwritten.
|
| |
|
|
|
|
|
|
| |
Boards whose size is an even multiple of CHARS_PER_SYNC would lose the
entire last region.
* Attempt to fix runaway memory usage of GUI text frames, but this needs
more work
|
| |
|
|
|
| |
Users can pick longer transcription durations for accuracy-critical
tasks, or shorter durations for latency-critical tasks.
|
| |
|
|
|
|
|
|
|
|
|
|
|
| |
VRChat won't update the FX layer associated with an avatar unless its
GUID changes. Delete the GUID file when overwriting our generated FX
layer to work around this.
* Change paging behavior: when a region is updated, we re-page everything
that comes after it. This fixes the issue where we go back to update
something, then jump back to the current screen, leaving some random
chunk of text somewhere on the board.
* Reduce transcription time from 28s to 10s. I'm going to expose this to
the user since there's a fundamental latency/stability tradeoff here.
|
| |
|
|
|
|
|
|
| |
Bump up recording window to 28 seconds. This helps a lot with long-form
transcription tasks, s.a. transcribing an audiobook.
We should expose this as a parameter, since at 10s the transcription delay is
typically 300ms, while at 28s it's typically 1.1-1.2s.
|
| |
|
|
|
|
|
|
| |
Users can now control how many letters wide and tall the board is.
Tested at 4x48, 5x60, 10x120, and 20x240. At 20x240, Unity freezes and
does not make forward progress. Perhaps creating 4800 float parameters
isn't a truly scalable interface.
|
| |
|
|
|
|
|
|
|
|
|
|
|
|
| |
Now it's possible to generate shaders with a custom number of rows, columns,
and bytes per character.
All edits to the shader should go through TaSTT_template.shader. To generate
a new shader from the template:
$ ./Scripts/generate_shader.py \
--bytes_per_char 2 \
--rows 1 \
--cols 12
--shader_template $(pwd)/Shaders/TaSTT_template.shader \
--shader_path $(pwd)/Shaders/TaSTT.shader
|
| |
|
|
|
| |
Users can now see the number of avatar parameter bits they'll use
prior to committing.
|
| |
|
|
|
|
|
|
|
| |
An off-by-one issue in numRegions() would result in one extra layer
trying to drive a letter in the last region, which would wrap back
around to the 0th character slot (cell).
* GUI explicitly logs when it's done generating avatar stuff
* OSC layer no longer tries to update cells which don't exist
|
| |
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
The transcription engine beeps when you start/stop transcribing so you know
that it's listening. Users can now disable this.
* add help text to all input fields in GUI
* make TaSTT generated file textctrls readonly, since I haven't tested
them being reassigned
* document idea to configure unity & transcription apps with config files
* controller input thread no longer crashes if steamvr isn't running, it just
slowly spins and waits
* when you stop transcribing, the transcription engine re-transcribes a few
times. I think this should improve end-of-transcription tail latencies
* transcribe.py now prints out its args
|
| |
|
|
|
|
|
|
| |
Define proper interfaces for these things. Simplify osc_ctrl,
temporarily dropping support for emotes (they were broken anyway).
* Bugfix: Japanese no longer crashes transcribe.py, but it still doesn't
show up in the wxTextCtrl
|
| |
|
|
|
|
|
|
|
|
| |
Because we allow users to customize the # of sync params, the board is
no longer divided into regions of uniform size. When the last region is
a different size than the rest, we simply omit it from paging.
This is a hack but it's easy to reason about.
Of course the entire paging stack should be rewritten, but not today.
|
| |
|
|
|
| |
This reduces the expected delay to wake up the board & start
transcribing from 750 milliseconds to 2.5 milliseconds.
|
| |
|
|
|
|
|
|
|
|
|
|
| |
Users can now control how many characters they send per sync event, as
well as the number of bytes used to represent each character.
This gives them the power to pick between faster paging and fewer sync
params.
International users must use 2 bytes per char (at least for now).
* package.ps1: don't distribute the gigantic TTF files, just the bitmaps
|
| |
|
|
|
| |
This fixed some slowness I was seeing when waking up the STT. The right
fix is to add interruptible sleeps. Let's fix this soon.
|
| |
|
|
|
|
|
|
|
|
|
|
| |
The typical use pattern is now possible without entering radial.
Leaving mounted to the world for a long time is no longer possible.
Maybe I need an override param?
Left joystick controls:
* Short press toggle 1: show board, lock to hand, start transcribing
* Short press toggle 2: lock to world, stop transcribing
* Long press: hide board, stop transcribing
|
| |
|
|
| |
Completed first end-to-end test on a third party avatar :)
|
| |
|
|
|
|
|
|
|
|
|
| |
GUI now generates parameters & menu.
Still need to handle write defaults.
* Add capability to append to avatar parameters & menu
* Install canned Unity assets, shaders, and fonts in avatar folder
* Check in materials for ease of use
* Bugfix: correctly label menu/parameters file pickers
|
| |
|
|
| |
Still need to generate params & merge menus. Getting close....
|
| |
|
|
| |
The GUI can now generate guid.map and animations.
|
| |
|
|
|
|
|
| |
* icon now works when pinned to taskbar
* add model selection
* add script to dump mic devices
* whisper models now download into the virtual environment
|
| |
|
|
|
|
|
|
| |
Users can now select their mic & spoken language in the GUI.
* pyaudio now samples at the mic rate, fixing an issue where frames
would drop. We downsample in the callback by dropping frames.
* add Sounds folder to package
|
|
|
GUI can now download all TaSTT dependencies and install them into a
virtual environment.
* Add buttons to check embedded python version & install dependencies
* Add class to wrap interacting with embedded Python
* Put all TaSTT python scripts into a folder
|
