browser audio detection phase 3

This commit is contained in:
Joren de Graaf
2026-10-02 21:34:38 +02:00
parent 5c0ca7bc47
commit 20bdabdd8f
50 changed files with 24755 additions and 144 deletions
+18
View File
@@ -0,0 +1,18 @@
# Browser YAMNet model
The browser GraphModel uses the [official TensorFlow YAMNet implementation](https://github.com/tensorflow/models/tree/master/research/audioset/yamnet) and [published weights](https://storage.googleapis.com/audioset/yamnet.h5). The official 521-class AudioSet map is included beside it. Runtime assets live in `fireworks_detect/static/browser-yamnet/`; browsers fetch them only from BoomCast. `VERSION` records the pinned source revision and TensorFlow version.
To regenerate, use a dedicated conversion environment. The conversion script downloads the pinned source and weights, exports a SavedModel, converts it to a TensorFlow.js GraphModel, and writes the class map and version file. It supplies scoped empty modules for the converter's unused TF Decision Forests, TF Hub, and JAX imports; YAMNet does not use those features. The following package set was used for this artifact on Python 3.13:
```sh
mkdir -p .yamnet-build-tmp
python3 -m venv .yamnet-convert
TMPDIR="$PWD/.yamnet-build-tmp" .yamnet-convert/bin/pip install --no-cache-dir tensorflow-cpu tensorflowjs tensorflow-hub tf-keras
TMPDIR="$PWD/.yamnet-build-tmp" .yamnet-convert/bin/pip install --no-cache-dir --no-deps tensorflowjs==4.22.0
TF_USE_LEGACY_KERAS=1 TMPDIR="$PWD/.yamnet-build-tmp" .yamnet-convert/bin/python scripts/build_browser_yamnet.py
node scripts/verify_browser_yamnet.mjs
npm run build
python manage.py collectstatic --noinput
```
The verifier runs an actual TensorFlow.js inference and checks the 521-class score output. Keep the class map aligned with the model weights. The browser's relevance and speech thresholds are triage settings in `browser_policy.py`; the server classifier remains authoritative. Compare changed model outputs with representative clips before enabling the beta after an upgrade.