docs(model): attribute adopted code in the Nemotron implementation

Modalities is MIT licensed; Megatron-LM, Megatron-Bridge and state-spaces/mamba
are Apache 2.0 (NVIDIA / Tri Dao, Albert Gu) and TorchTitan is BSD 3-Clause.
Preserving those notices in the derived files is a license obligation, so each
source file that adapts upstream code now carries a header naming the upstream
file, what was taken from it, the copyright holder and the license.

Marked as adapted from Megatron-LM:
* mamba2_mixer.py - packed [z, x, B, C, dt] projection layout, conv1d/A_log/D/
  dt_bias shapes and initialization distributions
* layer_pattern.py - the layer pattern symbols
* nemotron_layers.py - single-operator pre-norm residual layer structure
* nemotron_layer_specs.py - the declarative ModuleSpec/builder pattern
* nemotron_model.py - hybrid model structure
* router.py - sigmoid scoring, selection-only expert bias, renormalization
* moe.py - the load-balancing loss formula and its sequence-level variant
* load_balancing.py - the sign-based expert bias update rule
* experts.py, nemotron_mlp.py - the squared ReLU activation

Marked as adapted from state-spaces/mamba:
* ssd.py - the chunk-parallel SSD block decomposition (ssd_minimal_discrete /
  segsum) and the GatedRMSNorm semantics

Marked as adapted from TorchTitan (following the convention already used in
model_factory.py and trainer.py):
* experts.py - stacked per-expert weights and the torch._grouped_mm call
* moe.py - dispatch/combine structure, expert-bias and token-count buffers
* router.py - router interface shape
* load_balancing.py - the optimizer step pre-hook approach
* nemotron_stages_generator.py - pipeline split-point structure

Marked as adapted from Meta's Llama:
* nemotron_attention.py - grouped-query key/value head repetition

The YAML configs note that their hyperparameter values are adopted from the
Megatron-Bridge recipe; no code was taken from Megatron-Bridge.

docs/components/nemotron.md gains an Attribution section tabulating all of the
above in one place, and explicitly lists the files with no third-party
derivation (norms.py, nemotron_model_factory.py, nemotron_mfu.py, and the
partitioning algorithm in nemotron_stages_generator.py).

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>

Author: le1nux

config_files/training/config_nemotron3_nano_30b_a3b_fsdp2.yaml

@@ -1,7 +1,9 @@
 # Nemotron-3 Nano 30B-A3B pretraining configuration.
 #
 # Architecture from the model report (arXiv:2512.20848, Table 1 / Figure 2) and cross-checked
-# against the Megatron-Bridge recipe `recipes/nemotronh/h100/nemotron_3_nano.py`:
+# against the Megatron-Bridge recipe `src/megatron/bridge/recipes/nemotronh/h100/nemotron_3_nano.py`
+# (Copyright (c) 2026, NVIDIA CORPORATION, licensed under the Apache License, Version 2.0), from
+# which the hyperparameter values below are adopted:
 #
 #   52 layers   pattern MEMEM*EMEMEM*EMEMEM*EMEMEM*EMEMEM*EMEMEMEM*EMEMEMEME
 #               -> 23 Mamba-2, 23 MoE, 6 attention

docs/components/nemotron.md

@@ -177,3 +177,58 @@ Decaying the SSM dynamics parameters or the router gate destabilizes training.
 | `stages_generator` | `nemotron_stages_generator` | [NemotronStagesGenerator](../../src/modalities/models/nemotron/nemotron_stages_generator.py) |
 | `mfu_calculator` | `nemotron` | [NemotronMFUCalculator](../../src/modalities/utils/nemotron_mfu.py) |
 | `model_initialization` | `composed` (`model_type: nemotron`) | [ComposedInitializationRoutines](../../src/modalities/nn/model_initialization/composed_initialization.py) |
+
+## Attribution
+
+Modalities is MIT licensed. The files below adapt code from third-party projects and carry the
+corresponding notices in their file headers, as required by those licenses.
+
+**NVIDIA Megatron-LM** — Copyright (c) NVIDIA CORPORATION, Apache License 2.0.
+Portions additionally Copyright (c) 2024, Tri Dao, Albert Gu.
+
+| File | What was adapted | Upstream |
+|------|------------------|----------|
+| [mamba2_mixer.py](../../src/modalities/models/components/mamba2/mamba2_mixer.py) | Packed `[z, x, B, C, dt]` projection layout; `conv1d`/`A_log`/`D`/`dt_bias` shapes and init distributions | `megatron/core/ssm/mamba_mixer.py` |
+| [layer_pattern.py](../../src/modalities/models/nemotron/layer_pattern.py) | Layer pattern symbols (`M`, `*`, `E`, `-`) | `megatron/core/models/hybrid/hybrid_layer_allocation.py::Symbols` |
+| [nemotron_layers.py](../../src/modalities/models/nemotron/nemotron_layers.py) | Single-operator pre-norm residual layer structure | `megatron/core/ssm/mamba_layer.py`, `.../hybrid/hybrid_block.py` |
+| [nemotron_layer_specs.py](../../src/modalities/models/nemotron/nemotron_layer_specs.py) | Declarative layer-spec/builder pattern | `megatron/core/transformer/spec_utils.py::ModuleSpec`, `.../hybrid/hybrid_layer_specs.py` |
+| [nemotron_model.py](../../src/modalities/models/nemotron/nemotron_model.py) | Hybrid model structure (pattern-driven stack) | `megatron/core/models/hybrid/hybrid_model.py` |
+| [router.py](../../src/modalities/models/components/moe/router.py) | Sigmoid scoring, selection-only expert bias, top-k renormalization | `megatron/core/transformer/moe/moe_utils.py::topk_routing_with_score_function` |
+| [moe.py](../../src/modalities/models/components/moe/moe.py) | Load-balancing loss formula and its sequence-level variant | `.../moe_utils.py::switch_load_balancing_loss_func` |
+| [load_balancing.py](../../src/modalities/models/components/moe/load_balancing.py) | Sign-based expert bias update rule | `.../moe_utils.py::get_updated_expert_bias` |
+| [experts.py](../../src/modalities/models/components/moe/experts.py), [nemotron_mlp.py](../../src/modalities/models/nemotron/nemotron_mlp.py) | Squared ReLU activation | `megatron/core/activations.py` |
+
+**state-spaces/mamba** — Copyright (c) 2024, Tri Dao, Albert Gu, Apache License 2.0.
+
+| File | What was adapted | Upstream |
+|------|------------------|----------|
+| [ssd.py](../../src/modalities/models/components/mamba2/ssd.py) | Chunk-parallel SSD block decomposition (`ssd_minimal_discrete` / `segsum`); `GatedRMSNorm` semantics | `ssd_minimal_discrete`, `mamba_ssm.ops.triton.layernorm_gated.rmsnorm_fn` |
+
+**Meta TorchTitan** — BSD 3-Clause License. (Modalities already adapts from TorchTitan elsewhere.)
+
+| File | What was adapted | Upstream |
+|------|------------------|----------|
+| [experts.py](../../src/modalities/models/components/moe/experts.py) | Stacked per-expert weight layout; `torch._grouped_mm` over expert-sorted tokens | `torchtitan/models/common/moe.py::GroupedExperts` |
+| [moe.py](../../src/modalities/models/components/moe/moe.py) | Dispatch/combine structure; expert-bias and token-count buffers | `.../moe.py::MoE` |
+| [router.py](../../src/modalities/models/components/moe/router.py) | Router interface shape | `.../moe.py::TokenChoiceTopKRouter` |
+| [load_balancing.py](../../src/modalities/models/components/moe/load_balancing.py) | Applying the bias update as an optimizer step pre-hook | `.../moe.py` |
+| [nemotron_stages_generator.py](../../src/modalities/models/nemotron/nemotron_stages_generator.py) | Pipeline split-point structure (via Modalities' own `StagesGenerator`) | TorchTitan pipeline utilities |
+
+**Meta Llama** — [facebookresearch/llama](https://github.com/facebookresearch/llama).
+
+| File | What was adapted | Upstream |
+|------|------------------|----------|
+| [nemotron_attention.py](../../src/modalities/models/nemotron/nemotron_attention.py) | Grouped-query key/value head repetition (`_repeat_kv`), via Modalities' own GPT-2 | `llama/model.py` |
+
+**NVIDIA Megatron-Bridge** — Copyright (c) 2026, NVIDIA CORPORATION, Apache License 2.0.
+The hyperparameter values in
+[config_nemotron3_nano_30b_a3b_fsdp2.yaml](../../config_files/training/config_nemotron3_nano_30b_a3b_fsdp2.yaml)
+and the two Nemotron test configs are adopted from
+`src/megatron/bridge/recipes/nemotronh/h100/nemotron_3_nano.py` and cross-checked against the model
+report. No code was taken from Megatron-Bridge; it served as the configuration reference, and
+`src/megatron/bridge/models/nemotronh/nemotron_h_bridge.py` documents the HF parameter mapping that
+guided the module naming.
+
+Files with **no** third-party derivation (written for Modalities, structured after its existing
+GPT-2 components): `norms.py`, `nemotron_model_factory.py`, `nemotron_mfu.py`, and the
+partitioning algorithm in `nemotron_stages_generator.py`.

src/modalities/models/components/mamba2/mamba2_mixer.py

@@ -1,3 +1,9 @@
+# Portions of this file are adapted from NVIDIA's Megatron-LM
+# (megatron/core/ssm/mamba_mixer.py): the packed [z, x, B, C, dt] input-projection layout, the
+# conv1d / A_log / D / dt_bias parameter shapes, and their initialization distributions.
+# Copyright (c) 2024, NVIDIA CORPORATION. Copyright (c) 2024, Tri Dao, Albert Gu.
+# Licensed under the Apache License, Version 2.0.
+
 """Mamba-2 mixer, the sequence-mixing operator of hybrid Mamba-Transformer models.
 
 The layout follows the Megatron-LM reference implementation

src/modalities/models/components/mamba2/ssd.py

@@ -1,3 +1,9 @@
+# Portions of this file are adapted from the Mamba-2 reference implementation in
+# state-spaces/mamba (https://github.com/state-spaces/mamba): the chunk-parallel SSD block
+# decomposition follows `ssd_minimal_discrete` / `segsum`, and GatedRMSNorm reproduces the
+# semantics of `mamba_ssm.ops.triton.layernorm_gated.rmsnorm_fn` with norm_before_gate=False.
+# Copyright (c) 2024, Tri Dao, Albert Gu. Licensed under the Apache License, Version 2.0.
+
 """Pure-PyTorch Mamba-2 primitives (state space dual / SSD).
 
 This module provides a dependency-free, CPU-runnable and ``torch.compile``-friendly

src/modalities/models/components/moe/experts.py

@@ -1,3 +1,11 @@
+# Some portions of this implementation are inspired, adapted, or refactored from Meta's
+# open-source project TorchTitan (torchtitan/models/common/moe.py::GroupedExperts),
+# licensed under the BSD 3-Clause License. Specifically the stacked per-expert weight
+# layout and the torch._grouped_mm call over expert-sorted tokens.
+#
+# The squared ReLU activation follows NVIDIA's Megatron-LM (megatron/core/activations.py),
+# Copyright (c) NVIDIA CORPORATION. Licensed under the Apache License, Version 2.0.
+
 """Grouped expert feed-forward networks for mixture-of-experts layers.
 
 All experts of a layer are stored in two stacked weight tensors so that the whole layer can be

src/modalities/models/components/moe/load_balancing.py

@@ -1,3 +1,11 @@
+# Portions of this file are adapted from NVIDIA's Megatron-LM
+# (megatron/core/transformer/moe/moe_utils.py::get_updated_expert_bias): the sign-based
+# per-expert bias update rule.
+# Copyright (c) 2025, NVIDIA CORPORATION. Licensed under the Apache License, Version 2.0.
+#
+# Applying the update as an optimizer step pre-hook follows Meta's open-source project TorchTitan
+# (torchtitan/models/common/moe.py), licensed under the BSD 3-Clause License.
+
 """Auxiliary-loss-free load balancing for mixture-of-experts layers.
 
 Implements the update rule of "Auxiliary-Loss-Free Load Balancing Strategy for Mixture-of-Experts"

src/modalities/models/components/moe/moe.py

@@ -1,3 +1,12 @@
+# Portions of this file are adapted from NVIDIA's Megatron-LM
+# (megatron/core/transformer/moe/moe_utils.py::switch_load_balancing_loss_func): the
+# load-balancing loss formula E * sum_i(f_i * P_i) and its sequence-level variant.
+# Copyright (c) 2025, NVIDIA CORPORATION. Licensed under the Apache License, Version 2.0.
+#
+# The dispatch/combine structure and the expert-bias / token-count buffers are inspired by Meta's
+# open-source project TorchTitan (torchtitan/models/common/moe.py::MoE), licensed under the
+# BSD 3-Clause License.
+
 """Mixture-of-experts feed-forward layer.
 
 The layer combines a :class:`TopKRouter`, a :class:`GroupedExperts` stack and an optional always-on

src/modalities/models/components/moe/router.py

@@ -1,3 +1,11 @@
+# Portions of this file are adapted from NVIDIA's Megatron-LM
+# (megatron/core/transformer/moe/moe_utils.py::topk_routing_with_score_function): the sigmoid
+# scoring path, the selection-only expert bias, and the top-k renormalization.
+# Copyright (c) 2025, NVIDIA CORPORATION. Licensed under the Apache License, Version 2.0.
+#
+# The router interface is additionally inspired by Meta's open-source project TorchTitan
+# (torchtitan/models/common/moe.py::TokenChoiceTopKRouter), licensed under the BSD 3-Clause License.
+
 """Top-k mixture-of-experts router.
 
 Implements the routing scheme used by Nemotron-3 Nano: sigmoid gating with an additive

src/modalities/models/nemotron/layer_pattern.py

@@ -1,3 +1,8 @@
+# The layer-pattern symbols are adopted from NVIDIA's Megatron-LM
+# (megatron/core/models/hybrid/hybrid_layer_allocation.py::Symbols) so that pattern strings are
+# portable between the two frameworks.
+# Copyright (c) 2024-2026, NVIDIA CORPORATION. Licensed under the Apache License, Version 2.0.
+
 """Layer pattern parsing for hybrid Mamba-Transformer models.
 
 A hybrid model such as Nemotron-3 Nano is described by a pattern string in which every

src/modalities/models/nemotron/nemotron_attention.py

@@ -1,3 +1,7 @@
+# The grouped-query key/value head repetition follows Meta's Llama implementation
+# (https://github.com/facebookresearch/llama), as already adapted in
+# modalities/models/gpt2/gpt2_model.py::CausalSelfAttention._repeat_kv.
+
 """Grouped-query causal self-attention with a head dimension decoupled from the model dimension.
 
 This cannot reuse the GPT2 attention module: GPT2 derives the head dimension as