Tip
Pre-built Installers & Nightly Binaries: Looking to quickly install the VST3, AU, or Standalone plugins without building from source? Download pre-built installers and packages from the latest completed run of the Release Plugins GitHub Action Workflow (under the Artifacts section).
NetBender (nn_tilde_bending) is a performative neural audio synthesis and live circuit-bending engine evolved from IRCAM's nn~, started as a fork of nn_tilde_bending and inspired by modern runtimes such as (neural_tilde). It transforms deep learning audio models such as RAVE, Stable Audio Open VAE, and Meta EnCodec—into expressive, tactile instruments via real-time weight mutation, layer-selective trace isolation, momentary short-circuiting, thermal drift, resonant cross-talk, latent space terrain navigation, and safety circuit breakers.
Because TorchScript (torch.jit) is deprecated in PyTorch 2.x+, NetBender uses a unified, runtime-agnostic IBackend architecture (BackendFactory) across all frontends (JUCE VST3/AU/Standalone, Max/MSP, and PureData). Simply load any .pte, .onnx, or .ts model file and NetBender automatically dispatches to the appropriate engine:
| Extension | Backend Class | Hardware Acceleration / Delegates | Weight-Bending Mechanism |
|---|---|---|---|
.pte |
ExecuTorchBackend |
CoreML (Apple Neural Engine / GPU), MPS (Metal), XNNPACK (ARM NEON / x86 AVX), Portable | Input-Promoted Parameters (torch.func.functional_call) + unified NBND v2 .bin sidecar |
.onnx |
OnnxBackend |
CoreML Execution Provider (macOS @gpu 1 / GPU toggle), CPU multi-threaded |
Promoted Graph Initializers (onnx_bend_prep.py) + zero-copy Ort::Value bindings & NBND v2 .bin sidecar |
.ts |
TorchBackend |
MPS (macOS), CUDA (Linux/Windows), CPU (LibTorch fallback) | Direct in-place at::Tensor parameter mutation + shadow baseline cloning |
Compiled edge runtimes (ExecuTorch .pte and ONNX Runtime .onnx) achieve their speedups by freezing static weights inside hardware-compiled subgraphs. To preserve zero-allocation, sample-accurate weight bending and eliminate sidecar file clutter, NetBender unifies all model configuration, ratios, latency offsets, and baseline weights into a single NBND v2 binary sidecar (<model>.bin):
- Embedded JSON Metadata: The
.binfile header embeds the model metadata (sample rate, compression ratio, channels, methods,history_samples,output_offset,latent_scale, and delegate settings), removing the need for separate.jsonsidecar files. - Contiguous Binary Tensors: Promoted layer names, dimensions, and float32 weights are stored contiguously directly after the metadata header.
- Instant Zero-Copy Mapping: At runtime, NetBender reads the metadata and memory-maps the baseline weights with zero serialization overhead, binding them directly into ExecuTorch and ONNX Runtime execution inputs.
All multi-byte numeric fields are encoded in little-endian byte order:
+-----------------------+-----------------------+-----------------------+-----------------------+
| 'N' | 'B' | 'N' | 'D' (0x4E 0x42 0x4E 0x44: Magic 4B) |
+-----------------------------------------------+-----------------------------------------------+
| Version: uint32 (e.g. 0x00000002) | JSON Length L: uint32 |
+-----------------------------------------------+-----------------------------------------------+
| Embedded UTF-8 JSON Metadata String (L bytes) ... |
+-----------------------------------------------+-----------------------------------------------+
| Number of Tensors T: uint32 | |
+-----------------------------------------------+-----------------------------------------------+
| [Tensor Record 0]: |
| - Name Length K: uint32 |
| - Name: char[K] (UTF-8 / ASCII) |
| - Num Dimensions D: uint32 |
| - Shape: int64[D] (D * 8 bytes) |
| - Num Elements E: uint64 (8 bytes) |
| - Weights: float32[E] (E * 4 bytes) |
+-----------------------------------------------------------------------------------------------+
| [Tensor Record 1 ... T-1] (repeated contiguously) |
+-----------------------------------------------------------------------------------------------+
| Section | Field | Type | Description |
|---|---|---|---|
| Header | magic |
char[4] |
Format identifier: 'N', 'B', 'N', 'D' (0x4E424E44). |
version |
uint32 |
Container version (2 = unified JSON metadata + tensor payloads; 1 = tensor payload only). |
|
json_len |
uint32 |
Byte length |
|
json_payload |
char[L] |
UTF-8 JSON string encoding sample_rate, ratio, latent_dim, history_samples, output_offset, latent_scale, decode_latent_steps, delegate, methods, etc. |
|
num_tensors |
uint32 |
Total count |
|
| Tensor Records | name_len |
uint32 |
Length |
| (Repeated $T$ times) | name |
char[K] |
Layer identifier name (e.g. "generator.conv_in.weight"). |
num_dims |
uint32 |
Number of dimensions |
|
shape |
int64[D] |
Array of [128, 64, 7]). |
|
num_elements |
uint64 |
Total number of elements |
|
weights |
float32[E] |
IEEE 754 32-bit floating point tensor weights ( |
NetBender provides native frontends for DAWs, standalone desktop performance, and modular patching environments:
| Target | Description | Output / Installation |
|---|---|---|
plugin |
JUCE VST3 & AudioUnit (AU) (NetBender) |
Bundled dylibs, signed with SIGN_ID, installed to ~/Library/Audio/Plug-Ins/VST3 and ~/Library/Audio/Plug-Ins/Components (macOS). |
standalone |
JUCE Standalone Application (NetBender) |
Desktop app with direct audio/MIDI I/O and full interactive circuit-bending UI. |
maxmsp |
Max/MSP Externals (netBender~, mc.netBender~, mcs.netBender~, netBender.info) |
Built as .mxo / .mxe64 bundles for Max 8+. |
puredata |
PureData External (netBender~, with legacy nn~ alias) |
Built as netBender~.pd_darwin / netBender~.pd_linux / netBender~.dll. |
all (default) |
Builds all enabled frontend targets | Builds all configured targets. |
NetBender includes a unified Python CLI (python_tools/export_cli.py) and standalone utilities (python_tools/migrate_ts_to_pte.py, python_tools/rave_ts_to_onnx.py, python_tools/stable_audio_ts_to_onnx.py) to export models from checkpoints or migrate existing .ts models to .pte (ExecuTorch) and .onnx (ONNX Runtime).
Always install Python conversion dependencies inside .venv (CMake also auto-discovers ExecuTorch's C++ headers and delegate libraries directly from .venv):
python3 -m venv .venv
.venv/bin/pip install -r requirements.txtExecuTorch exports support two bend promotion modes (--bend-mode) and four hardware delegates (--delegate {coreml,mps,xnnpack,portable}):
-
--bend-mode delegate(Default, Recommended for CoreML / XNNPACK): Promotes 1D/channelwise normalization (norm,weight_g,$\gamma$ ,$\beta$ ), biases, and activation parameters (snake$\alpha/\beta$ , channel scales) to dynamic inputs while keeping heavy convolution kernels static inside the delegated CoreML/XNNPACK subgraph. This gives you full Apple Neural Engine / GPU acceleration and expressive, stable real-time circuit bending. -
--bend-mode full(Recommended forxnnpack/portablewhen bending every kernel): Promotes all floating-point weights (including 1D/2D convolution matrices) to dynamic graph inputs. -
--bend-mode none: Exports a pure static.ptegraph with no promoted weight inputs (maximum raw inference speed).
# Example converts reference models to CoreML-accelerated .pte files (GPU / Apple Neural Engine):
.venv/bin/python -m python_tools.migrate_ts_to_pte models/darbouka.ts \
-o models/darbouka_coreml.pte --delegate coreml --bend-mode delegate
.venv/bin/python -m python_tools.migrate_ts_to_pte models/speech.ts \
-o models/speech_coreml.pte --delegate coreml --bend-mode delegate
.venv/bin/python -m python_tools.migrate_ts_to_pte models/encodec_bendable.ts \
-o models/encodec_coreml.pte --delegate coreml --bend-mode delegate
.venv/bin/python -m python_tools.migrate_ts_to_pte models/stable_audio_1_vae.ts \
-o models/stable_audio_1_vae_coreml.pte --delegate coreml --bend-mode delegate
# Converts reference models to XNNPACK-accelerated .pte files (CPU SIMD):
.venv/bin/python -m python_tools.migrate_ts_to_pte models/darbouka.ts \
-o models/darbouka_xnnpack.pte --delegate xnnpack --bend-mode delegate
.venv/bin/python -m python_tools.migrate_ts_to_pte models/speech.ts \
-o models/speech_xnnpack.pte --delegate xnnpack --bend-mode delegate
.venv/bin/python -m python_tools.migrate_ts_to_pte models/encodec_bendable.ts \
-o models/encodec_xnnpack.pte --delegate xnnpack --bend-mode delegate
.venv/bin/python -m python_tools.migrate_ts_to_pte models/stable_audio_1_vae.ts \
-o models/stable_audio_1_vae_xnnpack.pte --delegate xnnpack --bend-mode delegate
# Export Stable Audio 3 SAME-S directly to XNNPACK, CoreML, or Portable .pte:
.venv/bin/python python_tools/exporters/same_s_exporter.py \
-i models/checkpoints/stable_audio_3_same_s.safetensors \
-o models/stable_audio_3_same_s_xnnpack.pte --format pte --delegate xnnpack
.venv/bin/python python_tools/exporters/same_s_exporter.py \
-i models/checkpoints/stable_audio_3_same_s.safetensors \
-o models/stable_audio_3_same_s_coreml.pte --format pte --delegate coreml
# Or use the unified export_cli wrapper / custom layer filters:
.venv/bin/python -m python_tools.export_cli migrate_pte \
-i models/stable_audio_1_vae.ts \
-o models/stable_audio_1_vae_coreml.pte \
--delegate coreml \
--bend-mode delegate \
--filter snake norm biasNote
How .ts .pte Lowering Works:
migrate_ts_to_pte.py first translates the TorchScript inlined_graph IR directly into an eager PyTorch nn.Module (TorchScriptGraphEagerModule) and exports it via torch.export.export() without touching ONNX (used by encodec_bendable.ts and stable_audio_1_vae.ts). If a model contains non-exportable complex/FFT ops (such as RAVE's filtered-noise synthesizer in darbouka.ts and speech.ts), it automatically falls back to NetBender's RAVE graph sanitizer to prune the non-exportable branch while preserving the tonal waveform and stateful streaming history window (history_samples).
# Convert RAVE TorchScript (.ts) models to bendable ONNX (exports forward, encode, decode + .bin sidecar):
.venv/bin/python -m python_tools.export_cli rave_to_onnx \
-i models/darbouka.ts \
-o models/darbouka_bendable.onnx \
--buffer-size 2048
.venv/bin/python -m python_tools.export_cli rave_to_onnx \
-i models/speech.ts \
-o models/speech_bendable.onnx \
--buffer-size 2048
# Convert Stable Audio Open 1.0 (v1 VAE) (.ts) or EnCodec (.ts) to bendable ONNX:
.venv/bin/python -m python_tools.export_cli stable_audio_to_onnx \
-i models/stable_audio_1_vae.ts \
-o models/stable_audio_1_vae_bendable.onnx \
--buffer-size 8192
.venv/bin/python -m python_tools.export_cli stable_audio_to_onnx \
-i models/encodec_bendable.ts \
-o models/encodec_bendable.onnx \
--buffer-size 2048 --channels 1
# Export Stable Audio 3 SAME-S (v3) directly from safetensors:
# Options: --format {ts, onnx, pte, both, all} and --delegate {xnnpack, coreml, portable}
.venv/bin/python python_tools/exporters/same_s_exporter.py \
-i models/checkpoints/stable_audio_3_same_s.safetensors \
-o models/stable_audio_3_same_s.ts --format both
# Or export all formats (.ts, .onnx, and .pte) in one run:
.venv/bin/python python_tools/exporters/same_s_exporter.py \
-i models/checkpoints/stable_audio_3_same_s.safetensors \
--format all --delegate xnnpack
# Export directly from Hugging Face or local checkpoints (Stable Audio, EnCodec, RAVE):
.venv/bin/python -m python_tools.export_cli stable_audio --onnx -o models/sa_vae.onnx
.venv/bin/python -m python_tools.export_cli encodec --onnx --output models/encodec_24k.onnx
.venv/bin/python -m python_tools.export_cli rave -i path/to/rave.ckpt -o models/my_rave.onnx --onnxIf you already have a standard .onnx model, you can promote its weights and generate its NBND .bin sidecar with onnx_bend_prep.py:
# Inspect an ONNX model's inputs, outputs, and initializers:
python python_tools/onnx_bend_prep.py -i model.onnx --inspect
# Promote all float initializers (or filter specific layers) and generate <model>_bendable.onnx + .bin:
python python_tools/onnx_bend_prep.py -i model.onnx -o model_bendable.onnx --filter norm bias snakeIn Python scripts, you can export any custom torch.nn.Module to a bendable .pte bundle using export_bendable_pte from python_tools/pte_bend_prep.py:
import torch
from python_tools.pte_bend_prep import export_bendable_pte
sample_audio = torch.zeros(1, 1, 2048)
export_bendable_pte(
module=my_audio_model,
sample_audio_input=sample_audio,
output_pte_path="models/custom_effect.pte",
delegate="coreml", # "coreml", "mps", "xnnpack", or "portable"
bend_mode="delegate", # "delegate", "full", or "none"
extra_metadata={"sr": 48000, "in_dim": 1, "out_dim": 1, "higher_ratio": 2048},
)-
Loading Models:
- Click LOAD MODEL in the top bar and select any
.pte,.onnx, or.tsfile. - The status bar displays the active model name, active backend badge (
[ExecuTorch (coreml)],[ONNX Runtime], or[TorchScript]), and latency/ratio info.
- Click LOAD MODEL in the top bar and select any
-
Interactive Weight Canvas & Triple-Display Architecture (
1D/2D/Latents):- 1D Multi-Trace Oscilloscope: Draw, invert, quantize, or scramble weights directly on the selected layer's parameter curve.
- 2D Filter Matrix Heatmap: Inspect and bend 2D convolution kernels and projection matrices with cross-channel trace bridges.
-
Latent Topographic Vector Pad: Navigate the autoencoder bottleneck (
encode$\rightarrow$ Latent Terrain Hook$\rightarrow$ decode) using Random Fourier Feature orbits (FourierOrbit), latent slew limiting (LatentSlew), and step quantization (Quantize). Works identically across.pte,.onnx, and.tsmodels without any runtime dependency on LibTorch.
-
Trace Isolation & Macro Controls:
- Isolate specific architectural groups (Norm / Gain (
$\gamma, \beta$ ), Early Transients, Internal Latents, Output Heads) to bend dynamics, pitch/formants, or texture independently. - Trigger momentary SHORT [!] bursts with Attack/Hold/Release envelopes or audio sidechain transient detection.
- Enable Thermal Drift for continuous, bounded Ornstein-Uhlenbeck parameter evolution.
- Isolate specific architectural groups (Norm / Gain (
-
Saving Bent Models:
- Click SAVE MODEL to snapshot your current bent weights into a new
.pte/.onnx(with updatedNBND.binsidecar) or.tsfile.
- Click SAVE MODEL to snapshot your current bent weights into a new
All Max/MSP (netBender~, mc.netBender~, mcs.netBender~, netBender.info) and PureData (netBender~, also aliased to nn~ for backward compatibility) externals share the same multi-backend engine as the JUCE plugin.
netBender~ <model_path> [method] [buffer_size]
-
<model_path>: Path or filename of a.pte,.onnx, or.tsmodel (if you omit the extension,netBender~automatically searches for<name>.pte,<name>.onnx, and<name>.tsin order). -
[method]: Synthesis method to run (forwardby default;encode,decode, orpriorwhen companion graphs/methods are available). -
[buffer_size]: Internal circular buffer size in samples (must be$\ge$ model'shigher_ratio;0disables the worker thread).
Examples:
[netBender~ vintage_bendable.onnx forward 2048][netBender~ stable_audio_1_vae.pte forward 8192 @gpu 1][mc.netBender~ rave_model.pte decode 2048]
Connect a message box to the left inlet of netBender~ and route the rightmost info outlet to inspect and mutate network layers in real time:
| Message | Arguments | Description |
|---|---|---|
layers |
(none) | Outputs layers <layer_0> <layer_1> ... from the info outlet listing all bendable parameter tensors. |
get_weights |
<layer_name> |
Outputs layer <w_0> <w_1> ... containing the current float weights of <layer_name>. |
set_weights |
<layer_name> <f1> <f2> ... |
Replaces the weights of <layer_name> in real time (zero-allocation in .pte / .onnx). |
reset_weights |
<layer_name> |
Restores <layer_name> to its original baseline weights from the NBND sidecar / snapshot. |
reset_all_weights |
(none) | Restores all bendable layers in the model to their original baseline weights. |
reload |
(none) | Reloads the model and resets all state buffers and weights from disk. |
load |
<model_path> |
Dynamically loads a new .pte, .onnx, or .ts model. |
method |
<method_name> |
Switches active synthesis method (forward, encode, decode, prior). |
dump |
(none) | Prints model path, active backend (ExecuTorch, ONNX Runtime, TorchScript), dimensions, ratios, methods, and attributes to the console. |
enable |
0 or 1 |
Enables or bypasses audio processing. |
gpu / @gpu |
0 or 1 |
Enables hardware acceleration (CoreML Execution Provider for .onnx,MPS/CUDA for .ts, or built-in delegate for .pte). |
- CMake (v3.15+) and a C++20 compiler (Xcode Command Line Tools on macOS, GCC 10+ on Linux, Visual Studio 2022 on Windows).
- ONNX Runtime (
src/cmake/add_onnxruntime.cmakeautomatically downloads the matching ONNX Runtime release ifort_diris not specified). - ExecuTorch: Automatically discovered from the project's Python virtual environment (
.venv) oncepip install -r requirements.txt(executorch>=1.0.0) has been installed, or via-DEXECUTORCH_ROOT=/path/to/executorch. - LibTorch: Download from pytorch.org into
./libtorchor pass-DCMAKE_PREFIX_PATH=/path/to/libtorch.
cmake -S src -B build \
-DCMAKE_PREFIX_PATH="$(pwd)/libtorch" \
-DCMAKE_BUILD_TYPE=Release# Build the JUCE VST3 and AU plugins:
cmake --build build --target plugin
# Build the JUCE Standalone desktop app:
cmake --build build --target standalone
# Build the Max/MSP externals (netBender~, mc.netBender~, mcs.netBender~, netBender.info):
cmake --build build --target maxmsp
# Build the PureData external (netBender~):
cmake --build build --target puredata
# Build all enabled targets:
cmake --build build --target all| Option | Default | Description |
|---|---|---|
-DBUILD_JUCE_PLUGIN=ON|OFF |
ON |
Build JUCE VST3, AU, and Standalone plugin targets. |
-DBUILD_MAXMSP=ON|OFF |
ON |
Build Max/MSP externals (netBender~, mc.netBender~, mcs.netBender~, netBender.info). |
-DBUILD_PUREDATA=ON|OFF |
ON |
Build PureData external (netBender~). |
-DENABLE_EXECUTORCH_BACKEND=ON|OFF |
ON |
Enable ExecuTorch .pte backend discovery and linking (EXECUTORCH_ROOT). |
-DBUNDLE_DEPENDENCIES=ON|OFF |
OFF |
Bundle and relocate shared libraries into support/ for redistributable packages. |
-DSIGN_ID="..." |
"-" |
macOS codesigning identity (can also be placed in .env at the repo root). |
The C++ test harness (build/tests/test_backends) verifies loading, streaming forward passes, decode gen, prior synthesis, NaN/Inf/fuse sentry protection, and real-time weight bending across .ts, .onnx, and .pte models:
# Run the full CPU test suite (.ts CPU, .onnx CPU, and *_xnnpack.pte) -> test_report_cpu.md:
./build/tests/test_backends
# Run the full GPU / Apple Neural Engine test suite (.ts MPS, .onnx CoreML EP, and *_coreml.pte) -> test_report_gpu.md:
./build/tests/test_backends --gpu
# Or test specific model files directly:
./build/tests/test_backends --gpu models/stable_audio_1_vae_coreml.pte models/encodec_coreml.pteNetBender models configure their streaming buffer, receptive field discard, and latent generation parameters via unified NBND v2 .bin embedded metadata and ONNX model metadata properties (netbender.*):
| Property | Default | Description |
|---|---|---|
history_samples |
0 |
Number of past audio samples kept in the streaming rolling buffer to provide context for non-causal convolutions. Eliminates boundary jump clicks (e.g. 4096 for Stable Audio Open, 28672 for RAVE Speech, 16384 for SAME-S). |
output_offset |
0 |
Initial transient / receptive-field latency samples discarded from synthesized output so the audio stream begins at steady state without silence gaps. |
latent_scale |
1.0 |
Ornstein-Uhlenbeck random walk standard deviation scale for decode gen synthesis, matched to the decoder's expected latent variance (e.g. 5.0 for EnCodec SEANet, 2.5 for RAVE Speech, 1.0 for unit-normal VAEs). |
decode_latent_steps |
1 |
Number of latent frames expected by the companion decode method. |