tuned moe config

[amd-hhashemi]

Speedups from tuning (last column):

E	topk	n_active	block_m	best_bn	best_bk	best_nb	per_iter_us	TFLOPS	BW_GBs	base_bn	base_bk	base_nb	base_us	speedup
384	6	6	16	64	512	3	8.84	7.47	1990.4	128	512	2	10.78	1.22
384	6	12	16	64	512	3	9.15	28.88	3856.3	128	512	2	11.06	1.21
384	6	24	16	64	512	3	10.1	104.65	7024.8	128	512	2	11.7	1.16
384	6	24	16	64	512	3	10.39	203.46	6902.1	128	512	2	11.84	1.14
384	6	64	16	64	512	3	14.76	286.44	12887.4	128	512	2	17.18	1.16
384	6	128	16	256	512	2	31.73	266.49	11989.8	128	512	2	42.6	1.34
385	7	7	16	64	512	3	9.03	11.38	3029.1	128	512	2	10.82	1.2
385	7	14	16	64	512	3	9.57	42.95	5728.4	128	512	2	11.44	1.2
385	7	28	16	64	512	3	11.16	147.33	9865.6	128	512	2	12.51	1.12
385	7	28	16	64	512	3	11.23	292.82	9885.8	128	512	2	12.48	1.11
385	7	64	16	128	512	2	18.85	348.89	13434	128	512	2	18.85	1
385	7	128	16	512	512	1	42.51	309.42	11914	128	512	2	49.58	1.17
385	7	7	16	64	512	3	9.5	16.23	4316.3	128	512	2	11.35	1.19
385	7	14	16	64	512	3	10.36	59.51	7927.7	128	512	2	11.81	1.14
385	7	28	16	64	512	3	13.43	183.64	12267.3	128	512	2	16.43	1.22
385	7	28	16	64	512	3	14.2	347.36	11670.7	128	512	2	16.81	1.18
385	7	64	16	256	512	3	31.34	314.77	12069	128	512	2	43.41	1.39
385	7	128	16	256	256	2	75.57	261.08	10010.3	128	512	2	82.87	1.1
385	7	7	16	64	512	3	10.47	29.44	7828	128	512	2	11.77	1.12
385	7	14	16	64	512	3	13.33	92.51	12307.7	128	512	2	16.34	1.23
385	7	28	16	512	512	2	26.56	185.71	12375.6	128	512	2	42.1	1.59
385	7	28	16	512	512	2	29.76	331.49	11083.4	128	512	2	42.31	1.42
385	7	64	16	256	256	2	75.2	262.37	10016.9	128	512	2	83.63	1.11
385	7	128	16	512	256	3	143.88	274.26	10470.8	128	512	2	161.14	1.12
256	6	6	16	64	512	3	7.75	25.98	6906.6	256	512	1	13.53	1.75
256	6	12	16	64	512	3	9.99	80.61	10725.9	256	512	1	14.89	1.49
256	6	24	16	128	512	2	16.48	195.46	13027.7	256	512	1	19.85	1.2
256	6	24	16	128	512	2	17.93	359.31	12018	256	512	1	20.21	1.13
256	6	64	16	512	512	1	50.79	253.69	11293	256	512	1	55.33	1.09
256	6	128	16	512	256	3	99.08	260.09	11577.9	256	512	1	123.48	1.25
256	6	6	16	64	512	3	5.58	18.04	4802.9	256	512	1	8.51	1.53
256	6	12	16	64	512	3	6.88	58.53	7808.6	256	512	1	9.4	1.37
256	6	24	16	128	512	2	9.96	161.71	10837.1	256	512	1	11.57	1.16
256	6	24	16	512	512	2	10.41	309.44	10463.1	256	512	1	11.76	1.13
256	6	64	16	512	512	1	20.2	318.93	14314.1	256	512	1	22.51	1.11
256	6	128	16	512	256	3	51.35	250.92	11261.7	256	512	1	59.26	1.15
384	6	6	16	64	128	3	4.68	7.06	1893.6	256	512	1	INVALID
384	6	12	16	64	128	3	5.77	22.9	3102.3	256	512	1	INVALID
384	6	24	16	64	128	3	8	66.06	4563.5	256	512	1	INVALID
384	6	24	16	128	128	3	8.11	130.33	4675.8	256	512	1	INVALID
384	6	64	16	256	128	1	15.39	137.36	6448.2	256	512	1	INVALID
384	6	128	16	256	128	2	24.38	173.42	8140.9	256	512	1	INVALID
385	7	7	16	64	256	3	4.64	11.07	2963.8	256	512	1	5.53	1.19
385	7	14	16	512	512	2	5.55	37.03	4993.1	256	512	1	6.49	1.17
385	7	28	16	512	256	2	8.07	101.87	6970.8	256	512	1	9.02	1.12
385	7	28	16	512	512	1	8.31	197.85	6969.6	256	512	1	9.71	1.17
385	7	64	16	512	256	1	14.44	227.72	9102	256	512	1	16.27	1.13
385	7	128	16	512	512	1	22.86	287.69	11498.9	256	512	1	23.98	1.05
385	7	7	16	64	256	3	5.33	14.46	3860.7	256	512	1	INVALID
385	7	14	16	64	256	3	6.82	45.2	6065.5	256	512	1	INVALID
385	7	28	16	512	256	2	10.54	116.99	7929.7	256	512	1	INVALID
385	7	28	16	512	256	2	10.82	227.93	7880.8	256	512	1	INVALID
385	7	64	16	512	256	1	20.16	244.67	9619.9	256	512	1	INVALID
385	7	128	16	512	256	2	35.12	280.89	11044.3	256	512	1	INVALID
385	7	7	16	256	512	2	6.35	24.27	6465.3	256	512	1	8.5	1.34
385	7	14	16	512	512	2	8.93	69.04	9219.7	256	512	1	9.93	1.11
385	7	28	16	128	512	2	14.7	167.77	11262	256	512	1	16.21	1.1
385	7	28	16	512	512	1	14.95	329.93	11192.6	256	512	1	16.62	1.11
385	7	64	16	256	256	2	36.16	272.82	10549	256	512	1	44.36	1.23
385	7	128	16	512	256	2	65.79	299.89	11596.1	256	512	1	82.93	1.26

[github-actions]

🏷️ CI Guide

Runs automatically on every PR:

• ✅ Pre-checks (submodule verification, code formatting)
• ✅ Aiter op tests (gfx942 + gfx950)
• ✅ Triton tests on MI35X (only when aiter/ops/triton/** or related paths are changed)

Extended tests (opt-in via labels):

Label	Tests
ci:triton-300x	Run an additional Triton test job on MI300X in PRs; main branch always runs both MI35X and MI300X
ci:sglang	SGLang integration tests: DeepSeek-R1-MXFP4 accuracy, Qwen 3.5 accuracy
ci:atom	ATOM benchmark: DeepSeek-R1-0528, GPT-OSS-120B
ci:atom_full	ATOM accuracy suite for PR and main models from ATOM models_accuracy.json
ci:vllm	vLLM benchmark: GPT-OSS-120B, DeepSeek-R1-0528, Kimi-K2.5
ci:all	All standard extended tests (excludes ci:atom_full)

Only add ci:atom_full for FlyDSL or Triton upgrades.
Add labels via the sidebar or gh pr edit 4287 --add-label <label>

[Copilot]

Pull request overview

This PR updates gfx1250 (Gluon/TDM) kernel tuning and memory-load behavior to improve MoE GEMM configuration selection and adjust TDM async-load caching behavior.

Changes:

• Added a shape/bucket-based tuned config table for get_kernel_config_gluon() in the MoE A8W4 path.
• Updated the Gluon MoE config heuristic logic (block sizes / buffer counts) used when a tuned entry is not found.
• Adjusted gfx1250 batched BF16 GEMM TDM pipeline loads to apply a cache modifier to B-tensor loads.

Reviewed changes

Copilot reviewed 2 out of 2 changed files in this pull request and generated 1 comment.

File	Description
aiter/ops/triton/moe/moe_op_gemm_a8w4.py	Introduces a tuned-config lookup + revised heuristic for Gluon MoE GEMM kernel configuration selection.
aiter/ops/triton/_gluon_kernels/gfx1250/gemm/batched/batched_gemm_bf16.py	Changes TDM async-load ordering and applies a cache modifier for B loads in the pipeline.

---

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

[Copilot]

Pull request overview

Copilot reviewed 1 out of 1 changed files in this pull request and generated 1 comment.

Comments suppressed due to low confidence (1)

aiter/ops/triton/moe/moe_op_gemm_a8w4.py:357

• In the heuristic fallback for gfx1250/gluon (block_m==16), block_k is reduced until block_k reaches 32, but the loop can still exit with k % block_k != 0. The gfx1250 gluon kernel uses TDM async_load with block_shape=(BLOCK_M, BLOCK_K) and no tail masking, so a non-divisible K risks out-of-bounds reads or incorrect accumulation. Add an explicit guard after the loop (or an early k % 32 == 0 check) to fail fast on unsupported K shapes.

            if k <= 512:
                block_k = 256
            elif k <= 1024:
                block_k = 512 if k % 512 == 0 else 256
            else:
                block_k = 512
            while block_k > 32 and k % block_k != 0:
                block_k //= 2