Qwen/Qwen3.8-2.4T-A95B

2.4T-parameter hybrid-attention MoE (~95B active) with linear attention on 69 of 92 layers, 512 routed experts, a built-in MTP draft head, 262K native context window and extensible to 1M context

NVFP4 W4A4 on 8 Blackwell GPUs: 304 output tok/s per user with MTP-3, up to 4300 total tok/s per GPU at DEP16

View on HuggingFaceView on ModelScope