| Commit message (Collapse) | Author | Age |
|---|
| |
|
|
|
|
| |
I find it kind of annoying when people wave around a big chatbox so I
added the option to have the chatbox be locked in worldspace whenever
it's visible. This defaults to on and can be disabled.
|
| | |
|
| |
|
|
|
|
|
|
|
|
|
|
|
| |
It now waits up to 10 seconds for a graceful exit and falls back on
the equivalent of a SIGKILL. The caller is assumed to have signaled to the
process through `in_cb` that an exit is desired.
Also:
* Fix graceful exit path of transcribe_v2.py.
* Add toggle to enable/disable preview text. It is enabled by default.
* Constrain transcription temperature to 0.0. This keeps latency more
predictable at the cost of some accuracy.
|
| |
|
|
| |
This makes them more reliable.
|
| |
|
|
|
|
| |
The non-text OSC messages were paging in too close to the text OSC
messages, breaking the whole system. Now the non-text OSC messages bump
back the time at which text OSC messages can begin being sent.
|
| |
|
|
|
|
|
|
|
|
|
|
| |
Also:
* DiskStream starts returning silence when out of data instead of just
stopping.
* Filter out Whisper segments with high `no_speech_prob` and low
`avg_logprob`.
* Add `saveAudio` function, useful for debugging.
* Tune vad silence cutoff to 250 ms. This is pretty accurate in
benchmarks.
|
| |
|
|
|
|
|
| |
Also parameterize `min_silence_duration_ms` in AudioSegmenter. I suspect
that for conversational speech, segmenting closer to 500 ms (rather than
the 2000ms default) is a better tradeoff between accuracy and
compute efficiency.
|
| |
|
|
| |
No longer needed.
|
| |
|
|
|
|
|
|
| |
FuzzyRepeatCommitter was approximating this behavior in the
best-performing configuration, so switch to it in earnest.
This committer simply commits audio once we detect a long enough gap in
speech. That's it!
|
| |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
This logic is highly IO bound *and* latency critical so it makes sense to put
it into its own thread.
Also:
* Collector::drop* methods return the dropped audio. Committer includes
that audio in commits. Transcription thread holds onto it. When the
user segments their speech with a button press, the transcription
thread sends the entire combined audio of all commits over to Whisper
to be transcribed. This allows us to recover from errors introduced
by segmentation.
* Remove unused animator params
* Fix issue where clearing the board doesn't completely reset STT state
TODO:
* Coalescing does not occur for in-place updates. It should.
|
| |
|
|
|
|
|
|
| |
Also:
* Enable SO_REUSEADDR on browser src socket
* Temporarily add evaluation dependencies to requirements.txt
* Fix browser src. It's now looking for a prefix that the python app
actually uses.
|
| |
|
|
|
|
|
|
|
|
|
|
| |
Four threads:
* Main thread
* Transcription (mic -> collector -> whisper -> committer -> pager)
* VR input
* Keyboard input
Also:
* add OscPager class to encapsulate all OSC interactions.
* bump `last_n_must_match` from 2 to 3 to reduce hallucinations
|
| |
|
|
| |
This has a slight positive effect on my benchmark.
|
| |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
Try adding two filters on top of the usual AudioCollector:
* Minimum length preservation: never report fewer than N seconds worth
of audio data. Pad with silence as needed.
* Volume normalizing: normalize audio volume.
Using my benchmark of 30-second audio clips from 3 speakers (lower is
better):
length enf + norm = 87.118
nothing = 90.917
norm = 94.538
length = 111.402
Both together are a slight improvement, but independently degrade the
result by a lot. I also observed more hallucinations in a conversational
pattern when using them vs. not. So I'll phase them out.
I'm still curious about *compression* as opposed to normalization.
|
|
|
A set of proper interfaces is called for. See #dev-update-spam in
discord for drawing of design.
Also add code to mechanically optimize committer parameters using an
audio file. Not perfectly repeatable since it depends on the performance
characteristics of the machine, but prob better than what we had before
(nothing).
|
