perf(render_images): stream multi-channel composite (avoid O(n_channels) RGBA cube)#736
Merged
Merged
Conversation
Codecov Report❌ Patch coverage is
Additional details and impacted files@@ Coverage Diff @@
## main #736 +/- ##
==========================================
+ Coverage 79.26% 79.27% +0.01%
==========================================
Files 17 17
Lines 4596 4594 -2
Branches 1028 1029 +1
==========================================
- Hits 3643 3642 -1
+ Misses 603 602 -1
Partials 350 350
🚀 New features to boost your workflow:
|
Five multi-channel composite branches built the full (n_channels, H, W, 4) float64 cube via np.stack([...], 0).sum(0) before reducing (peak ~64*n*H*W bytes; GB-scale on large multiplex images). Replace with a streaming helper _composite_channels that accumulates per-channel RGB into one (H, W, 3) buffer (O(1) full buffers), mirroring the existing n>3 branch. Byte-identical: numpy sum over the stacked outer axis reduces sequentially, matching acc += rgb. Verified end-to-end main-vs-branch (8 cases, max|delta|=0) and via the regression test across channel counts incl. >128. Peak memory 3357MB -> 26MB (129x) at 200 channels @512x512. The n>3 premultiplied-alpha branch and the RGB-direct branch keep their own math and are left untouched.
cd6db7b to
942be7b
Compare
`table` was assigned (None or sdata_filt[table_name]) but never read — color resolution goes through sdata_filt + table_name, not this local. Drop both assignments, keep the _check_instance_ids_overlap validation, and invert the guard. Clears a pre-existing ruff F841.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Closes #734.
Problem
Five multi-channel composite branches in
_render_imagesbuild the full(n_channels, H, W, 4)float64 cube vianp.stack([cmap(layers[ch]) for ch in channels], 0).sum(0)before reducing — peak ≈64·n·H·Wbytes (GB-scale on large multiplex images).Fix
A streaming helper
_composite_channelsaccumulates per-channel RGB into a single(H, W, 3)buffer (O(1) full buffers), mirroring the existing n>3 branch.Byte-identical
numpy.sum(axis=0)over the stacked outer axis reduces sequentially, matchingacc += rgb— so the result is bit-for-bit identical to the old stack-and-sum (incl. >128 channels, past numpy's pairwise threshold). Verified end-to-end main-vs-branch across 8 render cases (max|Δ| = 0) and locked by a regression test.Impact
Tests
TestCompositeChannels: asserts the helper equalsnp.stack(...).sum(0)[:, :, :3]across channel counts (2, 3, 7, 129, 200) and that it returns an owned C-contiguous float64(H,W,3)buffer.