Skip to content

[BUGFIX] Honor Hardtanh min_val and max_val - #344

Open
andrenepomuceno wants to merge 1 commit into
sdatkinson:mainfrom
andrenepomuceno:fix/hardtanh-bounds
Open

andrenepomuceno wants to merge 1 commit into
sdatkinson:mainfrom
andrenepomuceno:fix/hardtanh-bounds

Conversation

@andrenepomuceno

Copy link
Copy Markdown

Fixes #342

Problem

The trainer exports torch.nn.Hardtanh with its bounds, {"type": "Hardtanh", "min_val": ..., "max_val": ...} (nam/models/_activations.py, ~L253–258 in neural-amp-modeler 0.13). ActivationConfig::from_json ignored both keys, and Hardtanh always clamped to [-1, 1], so a model trained with other bounds rendered differently from the trainer.

Change

This follows the existing LeakyReLU pattern: a shared singleton for the default, and a dedicated instance when parameters are given.

  • ActivationConfig::from_json reads the optional min_val / max_val for Hardtanh. These are the existing ActivationConfig fields that LeakyHardtanh already uses.
  • ActivationHardTanh gets a (min_val, max_val) constructor, and hard_tanh gets a matching overload. The defaults are unchanged (-1, 1).
  • get_activation(config) returns a parameterized ActivationHardTanh when either bound is set (a missing bound defaults to ±1). Otherwise it returns the "Hardtanh" singleton as before.

A bare "Hardtanh" string, or an object without bounds, behaves exactly as before. No other path special-cases Hardtanh: the fast-tanh and LUT swaps only touch Tanh/Sigmoid/SiLU, and the A2 fast path only matches LeakyReLU.

Tests

  • test_hardtanh_config: sets the bounds on ActivationConfig directly and checks clamping at both bounds and pass-through inside them.
  • test_from_json_hardtanh_bounds: parses the trainer's export format and checks the parsed bounds and the resulting activation.
  • test_from_json_hardtanh_default: "Hardtanh" and {"type": "Hardtanh"} leave the bounds unset, resolve to the singleton, and clamp to [-1, 1].
  • test_hardtanh_bounds_apply_realtime_safe: applying a bounded Hardtanh allocates nothing (allocation tracking).

Cross-check: a randomly initialised neural-amp-modeler 0.13 WaveNet with inactive layer1x1 and a bounded Hardtanh, rendered with tools/render, matches the PyTorch forward pass (pad_start=True, 4096 samples) to a max abs error of 7.5e-8 with this PR and #341 (inactive layer1x1 residual) applied, down from 1.9e-3 on main.

run_tests passes in both the default and -DNAM_USE_INLINE_GEMM builds (g++ 11, Linux). Before the fix, the new tests fail.

Note

Like LeakyHardtanh, this does not check that min_val < max_val. PyTorch asserts it at construction, so a trainer-exported file always satisfies it.

The trainer exports torch.nn.Hardtanh as {"type": "Hardtanh", "min_val": ...,
"max_val": ...}, but the bounds were ignored and the activation always clamped
to [-1, 1]. Parse the optional bounds and build a parameterized Hardtanh when
they are present. Bare "Hardtanh" still uses the [-1, 1] singleton.

Fixes sdatkinson#342
dfernandes83 added a commit to dfernandes83/NeuralAmpModelerCore that referenced this pull request Oct 6, 2026
…s() larger than max buffer), sdatkinson#343 (keep activation in the residual when layer1x1 is inactive), sdatkinson#344 (honor Hardtanh min_val/max_val)
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

[BUG] Hardtanh ignores min_val/max_val from the .nam config

1 participant