# [gfx1250] FlyDSL MoE heuristic-fallback kernel fails to lower: `LLVM ERROR: Do not know how to expand this operator's operand!` (`raw.ptr.buffer.load.lds`)

[zufayu]

Summary

On gfx1250 (MI455), running DeepSeek-V4 (fp8) through SGLang, aiter's fused_moe
finds no tuned FlyDSL config for the MoE shape and takes the heuristic FlyDSL
fallback. The FlyDSL compiler then emits llvm.amdgcn.raw.ptr.buffer.load.lds, which the
gfx1250 LLVM backend cannot lower, aborting the process during CUDA-graph capture:

LLVM ERROR: Do not know how to expand this operator's operand!
Fatal Python error: Aborted

FlyDSL is the performant MoE path we want to use on gfx1250, so we're asking for a real
fix rather than staying on the older workaround aiter (see "Workaround" below).

Environment

• GPU: AMD MI455 / gfx1250, ROCm 7.14
• Model: DeepSeek-V4-Flash, fp8 (e4m3), tp=1
• SGLang 0.5.18 (79491a58), image henryx/xsgl:v0.5.18-rocm7_14-mi455-20260823-pr32754-trial1
• flydsl 0.3.1

aiter versions — crashes vs works:

aiter	Result
current main fe4b72cd4 (2026-08-23)	❌ crash
akao-amd/aiter 481127f5 (2026-08-19)	❌ crash
ROCm/aiter 73f5e114 (#4955 "Force fp4gemm preshuffle to triton on gfx1250")	❌ crash (this PR does not cover the MoE stage-1 FlyDSL path)
ROCm/aiter d9e5ef7ce08ee7045d583aed768cff41aa9210fe (#4406, 2026-07-29)	✅ works (predates the gfx1250 FlyDSL MoE path; routes MoE elsewhere; pins flydsl 0.2.4)

So the regression window is roughly d9e5ef7 (good, 2026-07-29) → 481127f5/fe4b72cd4 (bad).

Repro

Inside the container, on a crashing aiter (e.g. fe4b72cd4), launch dsv4:

SGLANG_DEFAULT_THINKING=1 AITER_GROUPED_CONTIGUOUS_TOKEN_THRESHOLD=16 \
SGLANG_DSV4_REASONING_EFFORT=max SGLANG_USE_ROCM700A=0 \
SGLANG_HACK_FLASHMLA_BACKEND=unified_kv_triton AITER_BF16_FP8_MOE_BOUND=0 \
AITER_FORCE_A8W4=1 SGLANG_USE_AITER_MOE_GU_ITLV=0 SGLANG_OPT_FUSE_MHC_POST_PRE=0 \
ENABLE_CK=0 SGLANG_USE_AITER=1 AITER_GROUPED_FORCE_SPLIT_K1=1 \
python3 -m sglang.launch_server --model-path /path/to/DeepSeek-V4-Flash \
  --trust-remote-code --tp 1 --attention-backend dsv4 --page-size 256 \
  --mem-fraction-static 0.60 --swa-full-tokens-ratio 0.15 --disable-shared-experts-fusion \
  --tool-call-parser deepseekv4 --reasoning-parser deepseek-v4 \
  --chunked-prefill-size 8192 --cuda-graph-max-bs 256 --max-running-requests 256 \
  --disable-radix-cache --kv-cache-dtype fp8_e4m3

It aborts during decode CUDA-graph capture.

What happens (log + traceback)

[aiter][fused_moe] no tuned FlyDSL config for
  ('gfx1250', 256, 256, 4096, 2048, 256, 6, <ActivationType.Silu>, 'torch.bfloat16',
   'torch.float8_e4m3fn', 'torch.float4_e2m1fn_x2', 'QuantType.per_1x32', True, False),
  using heuristic FlyDSL fallback
  (kn1='flydsl_moe1_afp8_wfp4_bf16_t32x128x256_w2_gui',
   kn2='flydsl_moe2_afp8_wfp4_bf16_t32x128x256_atomic_bnt2')

ExpandIntegerOperand Op #2: t197: ch = llvm.amdgcn.raw.ptr.buffer.load.lds<
  (dereferenceable load (s128) from ..., addrspace 8),
  (dereferenceable store (s4096) into ..., addrspace 3)> ...
LLVM ERROR: Do not know how to expand this operator's operand!
Fatal Python error: Aborted

  flydsl/compiler/jit_function.py:772  _run_pipeline
  flydsl/compiler/jit_function.py:924  compile
  aiter/ops/flydsl/moe_kernels.py:996   _run_compiled
  aiter/ops/flydsl/moe_kernels.py:1741  _flydsl_moe_stage1_impl
  aiter/ops/flydsl/moe_kernels.py:1942  flydsl_moe_stage1
  aiter/fused_moe.py:1451               _flydsl_stage1_wrapper
  ... (SGLang) ... decode_cuda_graph_runner capture ...
RuntimeError: Rank 0 scheduler died during initialization (exit code: -6)

Root-cause pointers (aiter source, fe4b72cd4)

• aiter/fused_moe.py:2774-2775 — the "no tuned FlyDSL config → heuristic FlyDSL fallback"
  branch in get_2stage_cfgs builds the fallback kernel names
  (flydsl_moe1_afp8_wfp4_bf16_t32x128x256_w2_gui / flydsl_moe2_...).
• aiter/fused_moe.py:1451 _flydsl_stage1_wrapper → aiter/ops/flydsl/moe_kernels.py:1884 flydsl_moe_stage1 → :607 compile_flydsl_moe_stage1 → flydsl JIT compile → the emitted
  IR contains llvm.amdgcn.raw.ptr.buffer.load.lds, which the gfx1250 backend can't expand.
• On gfx1250 the MoE path is FlyDSL-only (fused_moe.py:799 gfx1250 branch), so there is no
  non-FlyDSL fallback to route around it — the heuristic FlyDSL kernel is what runs.

Ask

Since FlyDSL is the intended high-performance MoE path on gfx1250, we'd like a real fix, in
rough order of preference:

1. Fix the FlyDSL codegen/lowering on gfx1250 so the heuristic-fallback MoE kernel does
   not emit an un-lowerable raw.ptr.buffer.load.lds (or so the backend can lower it).
2. Ship a tuned FlyDSL config for gfx1250 for this dsv4 MoE shape
   (256, 256, 4096, 2048, 256, 6, afp8/wfp4/bf16, per_1x32) so the heuristic fallback isn't
   hit in the first place — if the tuned kernels use a codegen path that already lowers.
3. If neither is near-term, at least make the fallback fail gracefully / pick a lowerable
   kernel on gfx1250 instead of aborting the whole scheduler.

Happy to test candidate fixes on our MI455 and report back.

Notes

• Grep tip: the rocdl dialect prints the underscore form raw_ptr_buffer_load_lds, but
  the LLVM lowering failure prints the dotted form raw.ptr.buffer.load.lds. Match
  raw[._]ptr[._]buffer[._]load[._]lds|Do not know how to expand this operator to catch both.
• Workaround (not a fix): pinning aiter to d9e5ef7 (fix(gemm_a8w8_blockscale): prevent scale OOB and support Triton 3.6 #4406) runs dsv4 (gsm8k 0.925/0.930,
  reproduced 2×) because that state doesn't route MoE through FlyDSL at all — but it forgoes
  later enhancements and the FlyDSL perf path, which is why we're filing this rather than
  staying on d9e5ef7.

[zufayu]

#4986 same bugs