summaryrefslogtreecommitdiffstats
Commit message (Collapse)AuthorAge
* Add example FX layer with write defaults offyum2022-10-20
| | | | | Simple hands animator that doesn't rely on write defaults. Use libunity gen_off_anims utility to generate per-animation reset animations.
* Add "off" animation generatoryum2022-10-20
| | | | | | | | Add utility to programmatically generate "off" animations. Scans every animation in the project, checks if it sets anything nonzero, and if so, generates a copy of it which sets everything to zero. This is useful for transitioning away from write defaults.
* Add preliminary support for negative anchorsyum2022-10-20
| | | | | | | | Some animators generate negative anchors. Casting to u64 doesn't produce an anchor with a valid prefix, so idk what it is. Use the class ID from the little !u! bit instead of deriving it from the anchor. Some things probably don't work yet.
* Quiet down transcribe.pyyum2022-10-20
| | | | | | Also adjust continuous transcription algorithm to use leftmost minimum instead of rightmost. This prevents some cases where we generate longer and longer text.
* Add continuous transcription modeyum2022-10-17
| | | | | | | | | | | | | | | | | Algorithm: * look at last 20 chars of last committed transcription * scan new transcription using 10-char sliding window * find spot where distance is minimized * stitch two messages together Thus we're able to maintain a continuously growing transcription without having to feed the AI more than 30 seconds of data at a time. Seems to work reasonably well in bench tests. Also fix silence detection. AI exposes a probability that nothing was said. Hand-pick a probability of 0.1. Sometimes the AI still goes sicko mode with this setting but going higher occasionally results in no transcription.
* Update backlogyum2022-10-16
|
* Add dev cheatsheetyum2022-10-16
| | | | Scratch doc containing commands I've been using a lot
* Programmatically set noop animationyum2022-10-16
| | | | | Overwrite any animation containing an unknown GUID to the tastt noop animation. This seems to help the reset layer function properly.
* Semi-fix gesture reset layeryum2022-10-16
| | | | | | | | | | | | Now we only overwrite gesture parameters if there's no active gesture. This makes gesturing smoother, since we're not overwriting gesture params twice on every frame. Gestures don't reliably reset. I think I need to add the noop animation across the entire animator. No idea what's really causing it. Also factor out code for generating transitions that have parameter conditions. Support exists for boolean and integer equality conditions.
* Fix a couple unity/YAML bugsyum2022-10-16
| | | | | | | | | * Unity needs empty Mappings to be indicated with {} or it will assume they're a Sequence * Unity doesn't like it when we reassign the default animation layer's MonoBehaviour ID, so hack around this by simply reusing the existing MonoBehaviour's ID * Use MulticoreUnityParser everywhere
* Add multicore YAML parseryum2022-10-16
| | | | | | | | Divide YAML stream into `nproc` chunks and parse each sub-stream in a process. We can't use threads because of the python global interpreter lock, but processes work pretty well. Parsing my 1.2M line / 43k document YAML goes from 65 seconds to 13.
* Add libunity.addTransitionyum2022-10-15
| | | | | * Implement basic board toggle using new transition logic * Metadata can now restore from file
* Transcribe.py now pagesyum2022-10-15
| | | | | | | | | Messages longer than a board will automatically write over the top. TODO * Real cell-based message diffing * Cumulative transcription * this would completely mitigate the effects of trim events
* Further improve transcribe.py responsivenessyum2022-10-15
| | | | | | | Add a third heuristic. If the transcription is relatively long and the first bit differs from the previous transcription, immediately overwrite. Because the transcription is long, it's a bit less likely to be a complete mistranscription.
* Tweak transcribe.pyyum2022-10-15
| | | | | | | | | | | | | | Slightly improve temporal stability and responsiveness at the cost of limiting to a 30 second recording. Before committing to a transcription, wait for two consecutive transcriptions such that they are identical, or the former is a prefix of the latter. This helps with temporal stability by eliminating most one-off wildly inaccurate transcriptions. Also make osc_ctrl.sendMessageLazy a little lazier, limiting it to 2 consecutive non-empty cells per call. This allows us to recover from mistranscriptions faster.
* Fix animations: renamed prefab from CustomSTT to TaSTTyum2022-10-15
| | | | | | | | | Also: * Check in toggle on/off animations * Add toggle parameter * libunity bug: getUniqueId() was calling allocateId() incorrectly * Remove osc_ctrl `client` global * Fix transcribe.py text encoding
* libunity: can now add layers, params, and animationsyum2022-10-15
| | | | Write defaults fix is now complete
* Begin fixWriteDefaults logicyum2022-10-15
| | | | | | | | | * Generate an animation which zeroes out everything which uses write defaults * Disable write defaults on every animation for which we do this * Add copy() method to Mapping and Sequence * Because of the `parent` pointer, copy.deepcopy() doesn't really work on this data structure.
* Add libunity CLIyum2022-10-15
| | | | | | * Add guid scanning method * Generate mapping from guid to filename * Mapping may be saved & restored from disk
* reimplement animator merging in yaml parseryum2022-10-14
| | | | | | | Object IDs are allocated optimally now, but it's a bit slower due to long parse times. Also fix minor bug in generate_fx.py.
* Add simple yaml parser (WIP)yum2022-10-13
| | | | | | | | Add parser for Unity's malformed YAML. This should make it easier to manipulate animators. It probably doesn't quite work yet, and certainly needs some usability features.
* Add ability to merge FX controllersyum2022-10-12
| | | | | | TODO * write default detection/correction * real cmdline interface
* Add ability to leave board in worldyum2022-10-11
| | | | | | | | | * Add VRLabs' World Constraint as a submodule * Add animations for world constraint * Add toggles for board * Add libunity.py (no content yet) * Support >30s transcription * Add board FBX
* WIP add lock world animationsyum2022-10-04
|
* Add osc_ctrl.ResizeBoardyum2022-10-04
| | | | | It's a little buggy; it likes to overwrite cells on the board. No idea why.
* WIP update resize animationsyum2022-10-04
|
* Board is now blank by defaultyum2022-10-03
| | | | Also update README.
* Update READMEyum2022-10-03
|
* Introduce STT proof-of-conceptyum2022-10-03
| | | | | | | | | | | | | | | | | Using OpenAI's whisper neural network, we can do local STT. Translation quality is good, system resource usage is minimal (1 GB VRAM), latency is much lower than cloud-based translation. * Add transcribe.py * Creates 3 threads: * One saves mic audio to a buffer * One passes mic audio to the STT * One sends the transcribed text to the board * Main thread listens for input. Press enter to start a new message. * Add osc_ctrl.sendMessageLazy, a simple diff-based message sending utility. * A little complexity: it only sends 1 empty cell per call, allowing us to quickly say new things without having to wait for the whole buffer to clear.
* Add LICENSEyum2022-10-02
| | | | | | * Update README with contribution instructions & design details. * Add text-to-text demo gif * Document known Unity landmines in generate.sh.
* Add 4th layer of indexingyum2022-10-02
| | | | | | | | | | | | | | * Double board size from 6x16 to 8x22 * Reduce parameter bits used (thanks to extra layer of indexing) * Rename template.anim to template.anim.txt to prevent Unity from constantly rewriting it * osc_ctrl.encodeMessage now pads the message so that all empty space is overwritten * Delete osc_ctrl.sendMessageCellContinuous. Now that we use a single 'Enable' bit, this idea is sidelined. * We can probably achieve the same effect by making TaSTT.shader a little more clever. For example, if we pass it the current cell number, it could render a time-based 'fade-in' effect which simulates smooth streaming.
* Use a single Enable parameter instead of one per layeryum2022-10-02
| | | | Even more reliable now.
* Paging now works for other players at 40 characters per secondyum2022-10-02
| | | | | | | * Shorten animations to 1 frame * Eliminate fx internal transition delays * These were causing the shader parameters to interpolate, causing the inconsistent / flickering letters I was seeing
* Add 'Do Nothing' animationyum2022-10-02
| | | | | | | | | | | | | | | Per the VRC docs, state behaviors may not execute if the total length of time in the state is < 0.02 seconds. Adding a 2-frame 'Do Nothing' animation to the top of every layer seems to help with stability. *shrug* More cleanup: * Generate a unique return-home transition for each terminal state instead of reusing the same one. * Use globally unique state names in animator. * All animations are at least 2 frames long.
* Add parameters to resize board (likely broken)yum2022-10-02
| | | | | | | | | | | | ... and a bunch of bugfixes: * Shader is now transparent * Simplify shader row/column calculation * Add punctuation to texture * Fix generate.sh * Add lorum_ipsum.txt * Fix how long text is scrolled * Simplify encoding logic in osc_ctrl.py
* Add board expand/shrink animationsyum2022-10-01
|
* Add line wrapping and support for arbitrarily long messagesyum2022-09-30
| | | | | | | | | | | Add trivial line wrapping algorithm. Words are only added to a line if they don't put it over the column limit, and only broken if they alone exceed the column limit. Extend board size to 16x6, using 145 bits of parameter memory. Add simple generate.sh script, which generates everything needed to use the text-to-text board.
* Redo FX layeryum2022-09-30
| | | | | | | | | | | | | | | | Apparently the same avatar parameter can only be updated so quickly before VRChat starts dropping messages. So now we divide the board into "groups" of 8 characters. Each group can be updated relatively slowly, but all groups can be updated in parallel. Thus we can update the board group-by-group, pausing between each group. * Fix shader bugs - now there are Row05 parameters, and row00 refers to the topmost row instead of the bottom-most. * Remove outdated layer/group names files * Extend osc_ctrl.py to support encoding & sending messages * Add generate_params.py to handle creating TaSTT_params.asset * Add generate_utils.py for common code generation facilities & parameters.
* begin unity-native portyum2022-09-30
|
* Add README.mdyum2022-09-29
|
* FIRST WORKING PROTOTYPE!yum2022-09-29
| | | | | | | | | Can't get much faster than 0.1 seconds per character with the current design. Still, a good first step! * Simplify parameters: only use 3 8-bit ints + 1 boolean. * Rewrite FX generator according to new params. * Rewrite osc_ctrl.py to test in-game display.
* delete SetLetters.csyum2022-09-29
| | | | | | | | | | | | | | | | | | | | | | | | Doesn't work in game. Also change # of characters per slot to 80, down from 128. Also realize that VRChat supports 256 BITS of parameter, not 256 BYTES. Next design idea: * 3 8-bit parameters: letter, row, col * 1 boolean parameter: active * one animation for each slot/letter combo, as usual * one fx layer like this: if !active: do nothing if row == 0: if col == 0: if letter == 0: play row00_col00_letter00 animation * because write defaults are off, we should be able to "save" letters by simply setting active = false * thus we don't need to simultaneously address the entire board, saving memory
* add 'hello world' osc controlleryum2022-09-29
| | | | | | simply sends numbers to a parameter's osc address of course, nothing is showing up in game. More debugging is needed.
* Generate 128 animations per cell (up from 60)yum2022-09-28
| | | | Also generate the full 6 rows instead of just 3
* big code dumpyum2022-09-28
| | | | | | | | | | | | | | | | | | | | | Summary: added generator for animations and FX layer. * add generate_animations.sh * generates an animation for every cell (14*6=72) and every letter (60); 72*60 = 4212 total animations * add generate_fx.py * seems to work in-game * drives every parameter needed * add {group,cell}_names.txt * name of every group & cell parameter * the STT has 72 individual character slots called cells. They are grouped into 4-character groups to save on parameter bandwidth. Thus each slot may be indexed with 8 bits. * add SetLetters.cs * this thingy looks at each group parameter and uses it to set each cell parameter. The generated fx layer hooks the script in. * check in SDK-generated assets as references until code generators are complete
* initial commityum2022-09-27
* initial checkin of shader * 6 rows and 14 cols * currently using Noto Sans Mono font * AFAIK free as in beer & freedom
generated by cgit v1.2.3 (git 2.39.1) at 2026-06-19 15:39:07 +0000