feat(media): transcode HEIC/HEIF to JPEG on desktop upload#1257
Open
tellaho wants to merge 1 commit into
Open
Conversation
iPhone HEIC/HEIF photos rendered blank in the desktop composer and were unviewable for everyone because Chromium/the Tauri webview cannot decode HEIC. Mobile already transcodes to JPEG before upload; desktop did not. Add HEIC detection mirroring mobile (full ftyp brand set, broader than the infer crate) and an ffmpeg HEIC->JPEG transcode, wired into both upload paths: process_picked_path (extension OR magic-bytes) and upload_media_bytes (magic-bytes only — no filename on the paste/drag path). Because the composer preview renders the server-returned blob URL, this single Rust fix covers both upload and preview. Missing ffmpeg errors like the video path (no silent blank-preview fallback). Co-authored-by: Taylor Ho <taylorkmho@gmail.com> Signed-off-by: Taylor Ho <taylorkmho@gmail.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Category: fix
User Impact: iPhone HEIC/HEIF photos now preview and upload correctly in the desktop chat composer instead of showing a blank preview and shipping an unviewable file.
Problem: HEIC/HEIF (Apple's default iPhone photo format) didn't render in the desktop chat composer — the preview went blank and the uploaded file was unviewable for everyone. Shend and tho both hit it; tho had to manually convert to JPEG. Slack handles HEIC fine, so this was a visible gap. Mobile already transcodes HEIC→JPEG before upload; desktop did not.
Solution: Transcode HEIC/HEIF to JPEG in Rust before upload, mirroring the existing video-transcode branch. Because the composer previews the server-returned blob URL of the uploaded bytes (not a local File), a single upload-side fix covers both upload and preview — no JS decode lib or canvas pass needed. Detection mirrors mobile's full ftyp brand set so the two platforms stay consistent.
File changes
desktop/src-tauri/src/commands/media_transcode.rs
Adds `is_heic_file(buf)` (ftyp box at offset 4 + mobile's full brand set `{heic,heix,hevc,hevx,heim,heis,mif1,msf1}`), `has_heic_extension(path)` for the picker path, and `transcode_heic_to_jpeg` (`ffmpeg -y -loglevel error -i -frames:v 1 -q:v 2 <out.jpg>`, 60s timeout, same anti-deadlock pattern as the video branch). 11 unit tests covering the brand table (including variants the `infer` crate misses), JPEG-not-HEIC, empty input, plus an ffmpeg-gated round-trip.
desktop/src-tauri/src/commands/media.rs
Wires HEIC detection + transcode into both upload paths (`process_picked_path` and `upload_media_bytes`) ahead of the existing video/passthrough branch. Resulting JPEG is re-detected by `detect_and_validate_mime`. Missing ffmpeg is a hard error, consistent with the video path — no silent fallback.
Reproduction Steps
Notes