FP8 RL Training and Rollout Weight Synchronization by YJHMITWEB · Pull Request #1898 · NovaSky-AI/SkyRL

YJHMITWEB · 2026-07-14T19:11:40Z

This PR adds end-to-end FP8 training and rollout support. It covers persistent Megatron FP8 parameters, serialized blockwise weight synchronization, mixed-dtype transport, and FP8 vLLM execution.

Changes

Change	Explanation
Serialized blockwise FP8 sync	`fp8_weight_sync_mode=serialized_blockwise` converts eligible weights to E4M3 and sends matching FP32 `weight_scale_inv` tensors in vLLM checkpoint format.
FP8 vLLM execution	Builds the rollout model with `quantization=fp8`, `load_format=dummy`, and a matching 128 x 128 block quantization configuration; the first full sync replaces the dummy weights.
Qwen3.5 mapping	Handles dense attention and MLP linears, GDN projections, shared experts, and routed MoE experts while keeping incompatible GDN and vision projections in BF16.
Mixed-dtype transport	Preserves mixed-dtype logical chunks over NCCL and partitions CUDA IPC buffers by their actual tensor dtype.
Persistent FP8 parameters	Keeps eligible Megatron primary parameters in E4M3, reducing the required HBM in Megatron.
Exact optimizer initialization	Initializes FP32 masters and hybrid CPU copies from unquantized checkpoint shards before the first optimizer step.
FP8 parameter gather	Exposes `fp8_param_gather`, which refreshes persistent FP8 parameters from updated FP32 masters.
FP8-safe packed sequences	Uses one TP/CP-aware alignment rule in collation, packed-sequence preprocessing, and MoE replay.

Scale Support

Without persistent fp8_param, serialized FP8 synchronization supports both FP32 and power-of-two block scales. Hopper supports both modes and defaults to FP32 scales; the current Blackwell path requires power-of-two scales.

Persistent fp8_param is supported only with FP32 scales. The main reason is that at a 1e-6 learning rate, 99.97-99.999% of sampled FP32 master values changed per update, no power-of-two block scale changed. The configuration therefore rejects power-of-two scales with fp8_param. Because Blackwell currently requires power-of-two scales, persistent FP8 parameters are not enabled there.

Configuration

<role> denotes policy or ref. All FP8 features are disabled by default, so existing BF16 configurations retain their current behavior.

Args	Default	Validated FP8 value	Behavior
`generator.inference_engine.fp8_weight_sync_mode`	`null`	`serialized_blockwise`	Sends rollout weights as E4M3 tensors with matching FP32 block scales.
`generator.inference_engine.engine_init_kwargs.quantization`	unset	`fp8`	Enables FP8 vLLM execution.
`trainer.<role>.megatron_config.transformer_config_kwargs.fp8`	unset	`e4m3`	Enables Transformer Engine FP8 compute for that Megatron model.
`trainer.<role>.megatron_config.transformer_config_kwargs.fp8_recipe`	unset	`blockwise`	Selects the Transformer Engine blockwise FP8 recipe.
`trainer.<role>.megatron_config.transformer_config_kwargs.fp8_param`	unset (`false`)	`true`	Stores Megatron primary parameters persistently in E4M3.
`trainer.policy.megatron_config.ddp_config.fp8_param_gather`	`false`	`true` with policy `fp8_param`	Refreshes policy FP8 parameters from updated FP32 optimizer masters after each step.

Environment variable	Runtime default	Validated value (Hopper)	Behavior
`NVTE_FP8_BLOCK_SCALING_FP32_SCALES`	`1` when FP8 is active	`1`	`1` uses FP32 block scales; `0` uses power-of-two scales. Persistent `fp8_param` requires `1`.
`NVTE_FP8_BLOCK_AMAX_EPSILON`	`0`	`1e-4`	Applies the same non-negative block-amax floor to Transformer Engine and serialized rollout quantization.
`VLLM_USE_DEEP_GEMM_E8M0`	`0` with serialized FP32 scales	`0`	Prevents vLLM from converting FP32 scales to E8M0 power-of-two scales.

serialized_blockwise requires Megatron and full-weight synchronization; adapter-only Megatron LoRA is rejected. On Blackwell, use power-of-two scales and keep fp8_param disabled.

Performance

vLLM Rollout Throughput

On Qwen/Qwen3.5-9B-Base, NVIDIA H100 80GB, TP1, vLLM 0.23.0, using DAPO recipe dataset.

Concurrent requests	BF16 output tok/s	FP8 output tok/s	Gain
1	140.5	198.6	+41.4%
16	1,940.3	2,599.4	+34.0%
32	3,418.9	4,361.0	+27.6%
128	7,897.7	8,989.2	+13.8%
256	9,223.3	10,791.0	+17.0%

This gain comes from FP8 vLLM execution and does not require persistent FP8 Megatron parameters.

End-to-End RL Step Timing

The end-to-end comparison uses Qwen/Qwen3.5-9B-Base on eight NVIDIA H100 80GB GPUs with Megatron TP2, eight TP1 vLLM engines, and a training batch size of 32.

Phase	BF16 mean (s)	FP8 mean (s)	FP8 time change	Speedup
Rollout generation	96.51	72.25	-25.1%	1.34x
Log-probability and reward forward passes	109.87	106.31	-3.2%	1.03x
Policy training	46.50	50.47	+8.5%	0.92x
Policy-to-vLLM weight sync	10.03	10.26	+2.3%	0.98x
End-to-end step	268.90	245.49	-8.7%	1.10x

Persistent FP8 Policy HBM

Megatron primary storage	HBM/GPU	Change at highest sample
BF16	59,676-63,638 MiB	Baseline
Persistent E4M3	56,850-57,810 MiB	-5,828 MiB (-5.69 GiB, -9.2%)

Quality Checks

The following WANDB runs log the metrics. All FP8 runs aligns well with the BF16 baseline.
Qwen3.5-4B runs: https://wandb.ai/sky-posttraining-uc-berkeley/qwen35_4b_dapo_mainclean_fp8align_fastbatch_bf16_fp8_20260630?nw=nwuserjinghanyao1

Qwen3.5-9B runs: https://wandb.ai/sky-posttraining-uc-berkeley/qwen35_9b_h100_tp2_bf16_fp8_prready_20260709?nw=nwuserjinghanyao1

Qwen3.5-35B-A3B runs: https://wandb.ai/sky-posttraining-uc-berkeley/qwen35_35b_a3b_fp8_amaxeps_20260705?nw=nwuserjinghanyao1

Reproduce key configs

H100:

export NVTE_FP8_BLOCK_SCALING_FP32_SCALES=1
export NVTE_FP8_BLOCK_AMAX_EPSILON=1e-4
export VLLM_USE_DEEP_GEMM_E8M0=0

trainer.algorithm.kl_loss_coef=0.00 trainer.algorithm.use_kl_in_reward=false \
trainer.policy.model.path=Qwen/Qwen3.5-9B-trainer.placement.colocate_all=true \
trainer.strategy=megatron \
trainer.placement.policy_num_nodes=1 \
trainer.placement.policy_num_gpus_per_node=8 \
trainer.placement.ref_num_gpus_per_node=8 \
trainer.policy.megatron_config.tensor_model_parallel_size=2 \
trainer.policy.megatron_config.pipeline_model_parallel_size=1 \
trainer.policy.megatron_config.context_parallel_size=1 \
trainer.policy.megatron_config.expert_model_parallel_size=1 \
trainer.policy.megatron_config.expert_tensor_parallel_size=1 \
trainer.ref.megatron_config.tensor_model_parallel_size=2 \
trainer.ref.megatron_config.pipeline_model_parallel_size=1 \
trainer.ref.megatron_config.context_parallel_size=1 \
trainer.ref.megatron_config.expert_model_parallel_size=1 \
trainer.ref.megatron_config.expert_tensor_parallel_size=1 \
trainer.epochs=20 trainer.max_training_steps=400 \
trainer.policy.megatron_config.ddp_config.overlap_grad_reduce=false \
trainer.ref.megatron_config.ddp_config.overlap_grad_reduce=false \
trainer.policy.megatron_config.transformer_config_kwargs.fp8_param=true \
trainer.ref.megatron_config.transformer_config_kwargs.fp8_param=true \
trainer.policy.megatron_config.ddp_config.fp8_param_gather=true \
trainer.eval_batch_size=512 \
trainer.train_batch_size=32 \
trainer.policy_mini_batch_size=32 \
trainer.micro_forward_batch_size_per_gpu=1 \
trainer.micro_train_batch_size_per_gpu=1 \
trainer.max_prompt_length=2048 \
trainer.policy.optimizer_config.lr=1e-6 \
trainer.policy.megatron_config.transformer_config_kwargs.fp8=e4m3 \
trainer.ref.megatron_config.transformer_config_kwargs.fp8=e4m3 \
trainer.policy.megatron_config.transformer_config_kwargs.fp8_recipe=blockwise \
trainer.ref.megatron_config.transformer_config_kwargs.fp8_recipe=blockwise \
trainer.policy.megatron_config.transformer_config_kwargs.fp8_amax_compute_algo=most_recent \
trainer.ref.megatron_config.transformer_config_kwargs.fp8_amax_compute_algo=most_recent \
trainer.policy.megatron_config.transformer_config_kwargs.tp_only_amax_red=false \
trainer.ref.megatron_config.transformer_config_kwargs.tp_only_amax_red=false \
trainer.micro_train_batch_size_per_gpu=2 \
trainer.micro_forward_batch_size_per_gpu=2 
generator.sampling_params.max_generate_length=8192 \
generator.n_samples_per_prompt=8 \
generator.eval_n_samples_per_prompt=16 \
generator.inference_engine.max_num_seqs=32 \
generator.eval_sampling_params.max_generate_length=8192 trainer.algorithm.use_kl_loss=false \
generator.inference_engine.num_engines=8 \
generator.inference_engine.tensor_parallel_size=1 \
generator.inference_engine.data_parallel_size=1 \
generator.inference_engine.expert_parallel_size=1 \
generator.inference_engine.engine_init_kwargs.quantization=fp8 \
generator.inference_engine.fp8_weight_sync_mode=serialized_blockwise \
generator.inference_engine.engine_init_kwargs.kv_cache_dtype=fp8_e4m3 \

B200 (only show the difference):

export FLA_TILELANG=0                          # B200/sm_100: TileLang GDN kernels crash; use triton
export NVTE_FP8_BLOCK_SCALING_FP32_SCALES=0    # B200: pow-2 (MXFP8) block scales — mandatory
export NVTE_FP8_BLOCK_AMAX_EPSILON=1e-4 

trainer.policy.model.path=Qwen/Qwen3.5-35B-A3B \
trainer.policy.megatron_config.tensor_model_parallel_size=1 \
trainer.policy.megatron_config.expert_model_parallel_size=8 \
trainer.policy.megatron_config.expert_tensor_parallel_size=1 \
trainer.ref.megatron_config.tensor_model_parallel_size=1 \
trainer.ref.megatron_config.expert_model_parallel_size=8 \
trainer.policy.megatron_config.transformer_config_kwargs.fp8_param=false \
trainer.ref.megatron_config.transformer_config_kwargs.fp8_param=false\

Note

High Risk
Changes weight sync, vLLM loading, FP8 packing alignment, and optimizer initialization on the critical train↔rollout path; misconfiguration is partly guarded by validation but numerical or MoE-loader mismatches could still break rollouts silently without tests on your exact stack.

Overview
Adds end-to-end FP8 for Megatron RL: TE FP8 GEMMs, optional persistent fp8_param, and fp8_weight_sync_mode=blockwise so rollouts receive trainer FP8 payloads and block scales instead of re-quantizing BF16.

Serialized FP8 weight sync introduces a weight_sync/fp8 stack (blockwise quantize, vLLM checkpoint wire format, per-model specs starting with Qwen3.5). The Megatron extractor expands weights into mixed fp8/fp32/bf16 chunks with derives_metadata_from_chunks; NCCL sends one update per chunk, CUDA IPC splits packs by dtype, and delta / sharded_rdt are rejected at validation and send time. vLLM startup is auto-configured (quantization=fp8, load_format=dummy, quantization_config + ignored layers); NewInferenceWorkerWrap loads batched MoE FP8 tensors via FusedMoE.

Training correctness: fp8_recipe (blockwise vs mxfp8) drives TP/CP/FP8 sequence alignment in packing, collators, and workers; fp8_recipe=auto resolves on workers (Hopper blockwise, Blackwell mxfp8) with validation for mxfp8 + no fp8_param. FP8 param uses deferred bridge load, exact FP32 masters from unquantized shards, and requires fp8_param_gather. Driver validation and Ray runtime env pin NVTE_FP8_BLOCK_SCALING_FP32_SCALES / VLLM_USE_DEEP_GEMM_E8M0 to match the scale contract.

Docs and examples/train/fp8/ scripts cover Hopper blockwise (with optional fp8_param) and Blackwell MXFP8 DAPO runs for Qwen3.5.

^{Reviewed by Cursor Bugbot for commit c127af6. Bugbot is set up for automated code reviews on this repo. Configure here.}

gemini-code-assist

Code Review

This pull request introduces blockwise FP8 serialization for rollout weight synchronization (specifically targeting Qwen3.5 layouts), adds chunking to vocab_parallel_entropy to bound temporary memory usage, and implements a colocated worker memory barrier with hard eviction to manage HBM footprint. It also updates sequence packing and alignment logic for FP8/TP/CP layouts and supports persistent FP8 parameter training. The review feedback highlights two critical issues: first, the bytes_per_token calculation for vocab entropy chunking does not account for batch dimensions, which could lead to OOMs; second, accessing megatron_config directly in prepare_runtime_environment will crash FSDP training runs due to an AttributeError, requiring defensive guards.

erictang000

Reviewed as a distributed-systems pass (correctness / concurrency / distributed-sync / numerical-stability / memory / perf). FP8 quantization numerics are sound (fuzz-verified), the lossy-master-seed window is correctly closed, and the BF16 path is preserved. Four issues below: one crash-on-valid-input (High) and three latent distributed hazards (Medium) that surface on specific hardware/config combos. Line-anchored comments follow.

Also worth confirming against the pinned Megatron/Bridge (couldn't verify here): (1) fp8_param.py:152 calls _copy_model_params_to_main_params(state_dict=...) but stock Megatron's method takes no args and copies from live (quantized) params — assumes a patched Bridge version; (2) the hybrid path re-seeds param_to_fp32_param/CPU copies but not optimizer.state[param]['master_param'], which megatron_strategy.py's own patch treats as authoritative; (3) the 3D fused-MoE weight_loader fast path in new_inference_worker_wrap.py assumes vLLM accepts a batched 3D tensor in one call.

erictang000 · 2026-07-21T01:26:23Z

@jinghanyao1-hub ai code review above, you can ignore for now, still taking a deeper pass, will resolve the above if i find it's not reasonable

erictang000 · 2026-07-24T01:48:58Z

high level comment on file structure:

can we reorganize to create a new folder workers/megatron/quantization/ that contains the _fp8_block_amax_epsilon_patch.py (maybe rename to just amax_epsilon_patch.py, and fp8_param.py? We can also move fake_int4_qat.py into that folder. So something like:

workers/megatron/quantization/
    __init__.py
    amax_epsilon_patch.py    
    fp8_param.py      
    fake_int4_qat.py

and then can we split the serialized_fp8.py file into a separate folder/components inside weight_sync? The structure should be something like this

weight_sync/fp8/
    __init__.py            
    quantize.py           # blockwise_cast_to_fp8, batched_blockwise_cast_to_fp8, scale helpers
    vllm_format.py        # SERIALIZED_BLOCKWISE_FP8, wire prefix, scale_name_for_weight,
                          #   get_serialized_fp8_quantization_config, iter_serialized_fp8_tensors
    models/qwen35.py      # _QWEN35_* tables, is_qwen35_config, ignored layers, MoE spec

erictang000 · 2026-07-24T01:51:56Z

another high level comment:

can we add a gpu ci test in the style of test_megatron_models but for full fp8? This will be added to H100 CI.

Essentially this test:

SkyRL/tests/backends/skyrl_train/gpu/gpu_ci/megatron/test_megatron_models.py

Line 271 in da778a5

pytest.param(

but with fp8 quantization on both vllm and megatron.

This will help us maintain full fp8 as a tracked feature across transformer engine/megatron/vllm updates. We want to test vllm/megatron init, weight sync, and logprob diff. We can start with qwen3.5-0.8b and qwen3.5-35B-a3B tests with blockwise quantization only.

erictang000

left some comments:

let's design a more extensible/maintainable solution for model specific quantization mappings/logic and try to keep instances of using a model name to a single config/abstraction
i'm a little confused by the fp8 param init logic but i might be missing something
ideally we can find areas where we could upstream logic to megatron-bridge rather than maintaining it ourselves for things like quantizing weights to fp8 for transfer
gpu ci tests for both regular full fp8 and fp8 params for hopper (+ both moe and dense tests for full fp8, so 3 total gpu ci tests)

otherwise the weight sync changes for cuda ipc/nccl look pretty clean, looking forward to seeing the PR with updates!

Review reorg for NovaSky-AI#1898: move _fp8_block_amax_epsilon_patch.py (renamed to amax_epsilon_patch.py), fp8_param.py, and fake_int4_qat.py into a new workers/megatron/quantization/ package and update import sites.

Review reorg for NovaSky-AI#1898: break weight_sync/serialized_fp8.py into weight_sync/fp8/quantize.py (blockwise casts and scale helpers), weight_sync/fp8/vllm_format.py (wire format, quantization config, tensor iterators), and weight_sync/fp8/models/qwen35.py (Qwen3.5 layer mapping), with the public API re-exported from weight_sync/fp8/__init__.py. _normalize_block_size becomes normalize_block_size now that it is shared across modules. No functional changes.

Answers PR NovaSky-AI#1898 review question: the env var is SkyRL's, the amax_epsilon field is TE's (no upstream knob); needed only for Float8BlockScaling with power-of-2 block scales (Blackwell emulation), where zero-token MoE experts produce amax=0 blocks whose scale_inv overflows the grad norm. FP32-scale Hopper and native MXFP8BlockScaling do not exhibit it (verified empirically).

…ric code Review item (PR NovaSky-AI#1898): the sender-side quantization policy, vLLM ignored-layer derivation, and batched-MoE wire mapping were Qwen3.5-specific free functions imported by generic code. ModelFp8Spec (weight_sync/fp8/ models/base.py) groups them behind one interface, resolved once per checkpoint via resolve_fp8_spec(hf_config): - matches / should_quantize / ignored_layers / moe_expert_spec follow the reviewer's sketch; MoeExpertSpec/MoeProjection additionally carry the vLLM fused-loader targets (vllm_param, shard_id) and the split dimension. - qwen35.py becomes a registered QWEN35_FP8_SPEC; unsupported layouts resolve to None and are rejected with the registered-spec list. - The vLLM worker extension derives _BATCHED_MOE_TARGETS from the registered specs (batched_moe_wire_targets) instead of re-encoding the fused-loader mapping - sender and receiver now share one source of truth. - SerializedFp8Config carries the resolved spec; MegatronWeightExtractor resolves it from hf_config at construction. - inference_servers/utils.py and the fp8 package API no longer name Qwen. User-configurable quantized-layer lists stay out of scope per review note.

Documents generator.inference_engine.fp8_weight_sync_mode and the Megatron-side FP8 surface it composes with (transformer_config_kwargs.fp8 / fp8_recipe incl. the "auto" per-architecture default / fp8_param, ddp_config.fp8_param_gather), plus the auto-managed block-scale env contract. Addresses PR NovaSky-AI#1898 review item on config documentation.

Review reorg for NovaSky-AI#1898: move _fp8_block_amax_epsilon_patch.py (renamed to amax_epsilon_patch.py), fp8_param.py, and fake_int4_qat.py into a new workers/megatron/quantization/ package and update import sites.

Review reorg for NovaSky-AI#1898: break weight_sync/serialized_fp8.py into weight_sync/fp8/quantize.py (blockwise casts and scale helpers), weight_sync/fp8/vllm_format.py (wire format, quantization config, tensor iterators), and weight_sync/fp8/models/qwen35.py (Qwen3.5 layer mapping), with the public API re-exported from weight_sync/fp8/__init__.py. _normalize_block_size becomes normalize_block_size now that it is shared across modules. No functional changes.

Answers PR NovaSky-AI#1898 review question: the env var is SkyRL's, the amax_epsilon field is TE's (no upstream knob); needed only for Float8BlockScaling with power-of-2 block scales (Blackwell emulation), where zero-token MoE experts produce amax=0 blocks whose scale_inv overflows the grad norm. FP32-scale Hopper and native MXFP8BlockScaling do not exhibit it (verified empirically).

…ric code Review item (PR NovaSky-AI#1898): the sender-side quantization policy, vLLM ignored-layer derivation, and batched-MoE wire mapping were Qwen3.5-specific free functions imported by generic code. ModelFp8Spec (weight_sync/fp8/ models/base.py) groups them behind one interface, resolved once per checkpoint via resolve_fp8_spec(hf_config): - matches / should_quantize / ignored_layers / moe_expert_spec follow the reviewer's sketch; MoeExpertSpec/MoeProjection additionally carry the vLLM fused-loader targets (vllm_param, shard_id) and the split dimension. - qwen35.py becomes a registered QWEN35_FP8_SPEC; unsupported layouts resolve to None and are rejected with the registered-spec list. - The vLLM worker extension derives _BATCHED_MOE_TARGETS from the registered specs (batched_moe_wire_targets) instead of re-encoding the fused-loader mapping - sender and receiver now share one source of truth. - SerializedFp8Config carries the resolved spec; MegatronWeightExtractor resolves it from hf_config at construction. - inference_servers/utils.py and the fp8 package API no longer name Qwen. User-configurable quantized-layer lists stay out of scope per review note.

Documents generator.inference_engine.fp8_weight_sync_mode and the Megatron-side FP8 surface it composes with (transformer_config_kwargs.fp8 / fp8_recipe incl. the "auto" per-architecture default / fp8_param, ddp_config.fp8_param_gather), plus the auto-managed block-scale env contract. Addresses PR NovaSky-AI#1898 review item on config documentation.

cursor

Stale Bugbot comment from a previous run.

erictang000 · 2026-08-31T21:31:36Z

h100 ci: https://console.anyscale.com/cld_hxkifz7xa22mwicp21nzkds1lw/prj_4b6c498rypyq6g7yhk6vzgjevt/jobs/prodjob_2uyc79satm1xe69hmn1cltsb5r?job-tab=overview&job-logs-section-tabs=application_logs

was all passing but ran into

=========================== short test summary info ============================
FAILED tests/backends/skyrl_train/gpu/gpu_ci/megatron/test_router_replay.py::test_forward_backward[tp2_pp2_ep2] - SystemExit: 1
==== 1 failed, 7 passed, 4 deselected, 2075 warnings in 4324.38s (1:12:04) =====

timeout

…rl-weight-sync

A `labeled` event starts a run of every workflow triggered on `labeled`, whatever the label was -- applying three labels to #1898 produced 15 runs, of which 3 did work. The other 12 skip, which is correct, but they published their skip under the same check name as the real run. GitHub keeps only the newest check of a given name, so whether a live suite showed as running came down to whether its own run happened to be created last. Two of the three running suites reported "skipped" on the PR. Name the job by whether it is going to run, so a skip lands on its own check context and can only ever supersede another skip. The running branch is the job id -- the existing context name -- so any branch protection pointing at it is unaffected.

…rl-weight-sync

erictang000 · 2026-09-01T00:18:34Z

h100 ci: https://github.com/NovaSky-AI/SkyRL/actions/runs/33453865623
megatron gpu ci: https://github.com/NovaSky-AI/SkyRL/actions/runs/33453869556
megatron models gpu ci: https://github.com/NovaSky-AI/SkyRL/actions/runs/33453865429/job/99689523705

…rl-weight-sync

cursor

Cursor Bugbot has reviewed your changes using default effort and found 1 potential issue.

^{Reviewed by Cursor Bugbot for commit 98f58a0. Configure here.}

…rl-weight-sync

erictang000 · 2026-09-05T16:02:56Z

https://wandb.ai/sky-posttraining-uc-berkeley/skyrl_fp8

verified 100 steps of DAPO still matching on scripts

…LELANG on Blackwell Qwen3.5 loads through the VL bridge, which packs sequences in its own forward and is rejected with SkyRL sample packing; set language_model_only on the policy, ref and inference engine like the other Qwen3.5 examples. The Blackwell scripts also export FLA_TILELANG=0: fla's TileLang GDN packed backward aborts on B200. Document both in the README. Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com> Signed-off-by: Eric Tang <etang@anyscale.com>

gemini-code-assist Bot reviewed Jul 14, 2026

View reviewed changes

Comment thread skyrl/backends/skyrl_train/distributed/megatron/model_utils.py Outdated

Comment thread skyrl/train/utils/utils.py Outdated

YJHMITWEB force-pushed the fp8-rl-weight-sync branch from 9113224 to 5c463c2 Compare July 14, 2026 19:24

jinghanyao1-hub self-assigned this Jul 14, 2026

YJHMITWEB force-pushed the fp8-rl-weight-sync branch 4 times, most recently from 2b1a0a3 to 787a4c5 Compare July 17, 2026 00:26

erictang000 self-assigned this Jul 20, 2026

erictang000 reviewed Jul 21, 2026

View reviewed changes

Comment thread skyrl/train/dataset/collators.py Outdated

Comment thread skyrl/backends/skyrl_train/workers/megatron/quantization/fp8_param.py

Comment thread skyrl/backends/skyrl_train/weight_sync/cuda_ipc_strategy.py

Comment thread skyrl/train/utils/utils.py

YJHMITWEB force-pushed the fp8-rl-weight-sync branch 2 times, most recently from 8460420 to 4a5cfa5 Compare July 21, 2026 22:48

erictang000 mentioned this pull request Jul 22, 2026

MXFP8 for experts in MoE #1937

Draft

kailash109 mentioned this pull request Jul 23, 2026

Persistent MXFP8 for experts layers in MoE #1940

Open

erictang000 mentioned this pull request Jul 24, 2026

SkyRL quantization (FP8, MXFP8, NVFP4) tracker #1943

Open

10 tasks

erictang000 reviewed Jul 24, 2026

View reviewed changes

kailash109 mentioned this pull request Jul 28, 2026

[feat] 1/n Unified Quantization Framework #1953

Open

jinghanyao1-hub force-pushed the fp8-rl-weight-sync branch from 85d056e to c60c8d1 Compare July 28, 2026 20:31

erictang000 added run_train_megatron_gpu_ci_models run_h100_gpu_ci Run H100 GPU CI labels Aug 31, 2026

cursor Bot reviewed Aug 31, 2026

View reviewed changes

Comment thread skyrl/train/utils/utils.py

Comment thread skyrl/train/utils/utils.py

erictang000 added 2 commits August 31, 2026 23:40

address comments

f9196ff

extend h100 ci timemout

d347d3c

erictang000 added run_train_megatron_gpu_ci run_train_megatron_gpu_ci_models run_h100_gpu_ci Run H100 GPU CI and removed run_train_megatron_gpu_ci run_train_megatron_gpu_ci_models run_h100_gpu_ci Run H100 GPU CI labels Aug 31, 2026

dyurk-lila mentioned this pull request Aug 31, 2026

perf(train): make sequence packing alignment-aware #2108

Open

erictang000 added 2 commits August 31, 2026 23:56

Merge branch 'main' of https://github.com/erictang000/SkyRL into fp8-…

c48f004

…rl-weight-sync

Merge branch 'main' of https://github.com/erictang000/SkyRL into fp8-…

2150375

…rl-weight-sync

erictang000 mentioned this pull request Sep 1, 2026

[ci] Keep skipped label-gated runs from masking live GPU CI #2110

Merged

Merge branch 'main' of https://github.com/erictang000/SkyRL into fp8-…

c0798ee

…rl-weight-sync

erictang000 removed the run_train_megatron_gpu_ci label Sep 1, 2026

Merge branch 'main' of https://github.com/erictang000/SkyRL into fp8-…

98f58a0

…rl-weight-sync

cursor Bot reviewed Sep 5, 2026

View reviewed changes

Comment thread skyrl/backends/skyrl_train/weight_sync/broadcast_strategy.py

erictang000 added 2 commits September 5, 2026 02:23

address comment and lint

fbed91d

x

079efbb

erictang000 mentioned this pull request Sep 5, 2026

[ci skip] bump h100 ci timeout #2168

Merged

erictang000 added 2 commits September 5, 2026 03:10

Merge branch 'main' of https://github.com/erictang000/SkyRL into fp8-…

870b521

…rl-weight-sync

fix h100 tests

587768b

erictang000 mentioned this pull request Sep 5, 2026

[megatron] pick fla's GDN backend per GPU architecture instead of forcing TileLang #2170

Draft

Uh oh!

Conversation

YJHMITWEB commented Jul 14, 2026 • edited by cursor Bot Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Changes

Scale Support

Configuration

Performance

vLLM Rollout Throughput

End-to-End RL Step Timing

Persistent FP8 Policy HBM

Quality Checks

Reproduce key configs

Uh oh!

gemini-code-assist Bot left a comment

Choose a reason for hiding this comment

Code Review

Uh oh!

Uh oh!

Uh oh!

erictang000 left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

erictang000 commented Jul 21, 2026

Uh oh!

erictang000 commented Jul 24, 2026

Uh oh!

erictang000 commented Jul 24, 2026

Uh oh!

erictang000 left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

cursor Bot left a comment • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

erictang000 commented Aug 31, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

erictang000 commented Sep 1, 2026

Uh oh!

cursor Bot left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

erictang000 commented Sep 5, 2026

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants

YJHMITWEB commented Jul 14, 2026 •

edited by cursor Bot

Loading

cursor Bot left a comment •

edited

Loading

erictang000 commented Aug 31, 2026 •

edited

Loading