Skip to content

test: add numerical reference coverage for the Qwen3.5 text decoder c…#4512

Open
chloechiaw wants to merge 1 commit into
AI-Hypercomputer:mainfrom
chloechiaw:test/qwen3_5-text-reference
Open

test: add numerical reference coverage for the Qwen3.5 text decoder c…#4512
chloechiaw wants to merge 1 commit into
AI-Hypercomputer:mainfrom
chloechiaw:test/qwen3_5-text-reference

Conversation

@chloechiaw

Copy link
Copy Markdown

Description

I noticed that the recent models here (DeepSeek-V4, GPT-OSS, Qwen3-Next) all have numerical
reference tests, but the Qwen3.5 text decoder added back in April 2026
never got one. The only Qwen3.5 test in MaxText repo covers the vision tower, so this is a consistency test

Since Qwen3.5 specifically stitches layers together, cause every 4th layer is a
full-attention layer and the rest are GatedDeltaNet, and Qwen3_5ScannableBlock builds one
full cycle of them. That wiring had no coverage, so that's what these tests target.

The tests reuse the existing Qwen3-Next PyTorch reference layers and compare a full Qwen3.5
decoder layer against them. Test-only change — nothing in src/ changes.

Tests

Adds tests/unit/qwen3_5_vs_reference_test.py with four tests (float32, same setup as the
existing Qwen3-Next reference test):

  • a decoder layer at a full-attention position (layer 3) vs. the PyTorch reference
  • a decoder layer at a GatedDeltaNet position (layer 0)
  • a structural check that the layer cycle picks the right attention type at each position
  • an end-to-end numerical check of a full 4-layer cycle

Run with:

python -m pytest tests/unit/qwen3_5_vs_reference_test.py -v

All four pass on a Cloud TPU from Google TRC (v4-8) in ~16s.

Checklist

Before submitting this PR, please make sure (put X in square brackets):

  • I have performed a self-review of my code. For an optional AI review, add the gemini-review label.
  • I have necessary comments in my code, particularly in hard-to-understand areas.
  • I have run end-to-end tests tests and provided workload links above if applicable.
  • I have made or will make corresponding changes to the doc if needed, including adding new documentation pages to the relevant Table of Contents (toctree directive) as explained in our documentation.
  • [ ]
    Please let me know if any fixes need to be made!!

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant