anodyine/vllm-omni
1
0 Fork 0
Code Issues 4 Pull Requests 2 Actions Packages Projects Releases Wiki Activity
3,424 Commits 235 Branches 31 Tags
pr-6169
Commit Graph
51 Commits
This Branch
This Branch
All Branches
Author SHA1 Message Date
Nick CaoandClaude 51b0184969 [Tests] Add tiny model test for StableDiffusionXLPipeline 
Covers base text-to-image plus CPU offload; UNet stays full-size
since its architecture is hardcoded rather than config-driven.

Co-Authored-By: Claude <noreply@anthropic.com>

Signed-off-by: Nick Cao <ncao@redhat.com>
 2026-10-05 11:44:08 -04:00
Nick CaoandCodex 0e6791e9c4 [Bugfix] Initialize eager state in Higgs test fixture (#8426) 
Signed-off-by: Nick Cao <ncao@redhat.com>
Co-authored-by: Codex <noreply@openai.com>
 2026-10-02 10:22:52 -04:00
Nick CaoandCodex c85f1a44fb [Bugfix] Break duplex OpenAI import cycle (#8287) 
Signed-off-by: Nick Cao <ncao@redhat.com>
Co-authored-by: Codex <noreply@openai.com>
 2026-09-29 11:07:03 -04:00
Nick CaoandCodex b656fcf45b [Tests] Use get_file_store_init_method helper from vllm (#7528) 
Signed-off-by: Nick Cao <ncao@redhat.com>
Co-authored-by: Codex <noreply@openai.com>
 2026-09-28 14:22:25 -04:00
Nick CaoandCodex d521b3f917 [Misc] remove legacy full_duplex alias for auto_response (#8024) 
Signed-off-by: Nick Cao <ncao@redhat.com>
Co-authored-by: Codex <noreply@openai.com>
 2026-09-23 06:09:19 -07:00
Nick CaoandCodex 1d3dc8bf83 [Misc] Refactor static stage metadata lookups (#7602) 
Signed-off-by: Nick Cao <ncao@redhat.com>
Co-authored-by: Codex <noreply@openai.com>
 2026-09-21 18:34:50 -07:00
Nick CaoandCodex b81f2c5d4b [Misc] treewide: remove shm_threshold_bytes from deploy config (#7522) 
Signed-off-by: Nick Cao <ncao@redhat.com>
Co-authored-by: Codex <noreply@openai.com>
 2026-09-17 19:22:14 +00:00
Nick CaoandCodex 78934753ff [Tests] Test image edit input limits in tiny model framework (#7434) 
Signed-off-by: Nick Cao <ncao@redhat.com>
Co-authored-by: Codex <noreply@openai.com>
 2026-09-15 12:36:34 -04:00
Nick CaoandCodex ed3776645d [Tests] Skip global GPU cleanup for parallel online diffusion tests (#7534) 
Signed-off-by: Nick Cao <ncao@redhat.com>
Co-authored-by: Codex <noreply@openai.com>
 2026-09-15 04:50:07 +08:00
Nick Cao fb23e8807a [BugFix] Fix importing renamed module vllm.entrypoints.openai.engine.… (#7531) 
Signed-off-by: Nick Cao <ncao@redhat.com>
 2026-09-14 13:15:27 -06:00
Nick CaoandCodex d3493384cf [BugFix] Fix leftovers of the legacy OpenAI realtime API event names (#7426) 
Signed-off-by: Nick Cao <ncao@redhat.com>
Co-authored-by: Codex <noreply@openai.com>
 2026-09-11 22:28:55 +08:00
Nick CaoandClaude Sonnet 5 498c41b8d6 [Realtime] Emit current (non-beta) OpenAI audio/transcript event names (#7339) 
Signed-off-by: Nick Cao <ncao@redhat.com>
Co-authored-by: Claude Sonnet 5 <noreply@anthropic.com>
 2026-09-11 08:50:12 +08:00
Nick Cao b2c9376539 [BugFix] Eliminate distributed test port race by switching to file:// rendezvous (#6468) 
Signed-off-by: Nick Cao <ncao@redhat.com>
 2026-08-23 07:08:44 +08:00
Nick Cao faa12af454 [Misc] Bump diffusers pin to 0.40.0 (#6459) 
Signed-off-by: Nick Cao <ncao@redhat.com>
 2026-08-22 12:20:34 +08:00
Nick Cao cfbf1392c5 [Bugfix] Drop librosa reintroduced by Step-Audio2 (#6467) 
Signed-off-by: Nick Cao <ncao@redhat.com>
 2026-08-21 21:33:07 +00:00
Nick Cao ebb4abde0a [Tests] Run diffusion tiny model tests in parallel (#6339) 
Signed-off-by: Nick Cao <ncao@redhat.com>
 2026-08-20 12:46:32 -04:00
Nick Cao 12f8168179 [Tests] Enable tiny model testing for Qwen-Image Edit and EditPlus (#5656) 
Signed-off-by: Nick Cao <ncao@redhat.com>
 2026-08-12 16:03:49 -04:00
Nick Cao 584d78c673 [Test] Add tiny model builder for FluxKontextPipeline (#5823) 
Signed-off-by: Nick Cao <ncao@redhat.com>
 2026-08-11 10:59:09 -04:00
Nick Cao 589afcef6c Add @NickCao to CODEOWNERS (#5807) 
Signed-off-by: Nick Cao <ncao@redhat.com>
 2026-08-05 21:50:48 +08:00
Nick Cao fbea788668 [Refactor] Use vLLM video loader for video reference decoding (#5085) 
Signed-off-by: Nick Cao <ncao@redhat.com>
 2026-08-05 01:49:41 +08:00
Nick CaoClaudeHongsheng Liu
 		273fc8eb1d [Tests] Further extend the tiny model testing framework and enable multiple models (#5353) 
Signed-off-by: Nick Cao <ncao@redhat.com>
Co-authored-by: Claude <noreply@anthropic.com>
Co-authored-by: Hongsheng Liu <liuhongsheng4@huawei.com>
 2026-07-31 07:47:26 +08:00
Nick Cao f10320f53f [Tests] Config driven tiny model builder (#5090) 
Signed-off-by: Nick Cao <ncao@redhat.com>
 2026-07-22 17:11:53 +00:00
Nick Cao 1f0ed96fcd [Bugfix] Fix broken FrameSimilarityFilter threshold test (#4786) 
Signed-off-by: Nick Cao <ncao@redhat.com>
 2026-07-18 13:52:07 +08:00
Nick CaoandHongsheng Liu d8edf0c9d5 [Bugfix] Use MediaConnector for image/video URL fetching to prevent SSRF (#2565) 
Signed-off-by: Nick Cao <ncao@redhat.com>
Co-authored-by: Hongsheng Liu <liuhongsheng4@huawei.com>
 2026-06-25 06:28:26 +08:00
Nick Cao 67d14b5ac4 [Refactor] Replace hardcoded CUDA device selection with platform-agnostic API (#4496) 
Signed-off-by: Nick Cao <ncao@redhat.com>
 2026-06-23 14:38:32 +08:00
Nick CaoandClaude 6646a66d41 [BugFix] Use ModuleDiscovery for HSDP DiT sharding to support nested pipelines (#3456) 
Signed-off-by: Nick Cao <ncao@redhat.com>
Co-authored-by: Claude <noreply@anthropic.com>
 2026-06-23 02:54:14 +00:00
Nick CaoandClaude e957d16f8d [Refactor] Migrate existing pipelines to use SupportsComponentDiscovery for offload discovery (#3076) 
Signed-off-by: Nick Cao <ncao@redhat.com>
Co-authored-by: Claude <noreply@anthropic.com>
 2026-06-17 21:33:06 +08:00
Nick Cao 864d22164c [Bugfix] Fix DFlash prefix cache corruption due to missing lookahead block (#4449) 
Signed-off-by: Nick Cao <ncao@redhat.com>
 2026-06-16 22:48:01 +08:00
Nick CaoandClaude 23484d8a11 [Bugfix] Pass media paths to use_mixed_modalities in example script (#3355) 
Signed-off-by: Nick Cao <ncao@redhat.com>
Co-authored-by: Claude <noreply@anthropic.com>
 2026-06-02 21:04:16 +00:00
Nick CaoandClaude c20afebce4 [Refactor][Qwen3-TTS] Construct speech tokenizer encoder natively (#3360) 
Signed-off-by: Nick Cao <ncao@redhat.com>
Co-authored-by: Claude <noreply@anthropic.com>
 2026-05-28 16:20:49 +08:00
Nick Cao 06ab723afc [Perf][VoxCPM2][Ming-Flash-Omni] Use global CUDA graph pool (#3361) 
Signed-off-by: Nick Cao <ncao@redhat.com>
 2026-05-20 05:01:42 +00:00
Nick CaoandClaude d4e90bbba8 [Bugfix][TTS] Only populate voice_name for uploaded voices without inline ref_audio (#3523) 
Signed-off-by: Nick Cao <ncao@redhat.com>
Co-authored-by: Claude <noreply@anthropic.com>
 2026-05-20 04:36:59 +00:00
Nick CaoClaudeHongsheng Liu
 		33220b1e39 [BugFix] Finish async_chunk requests without pad-token injection (#3613) 
Signed-off-by: Nick Cao <ncao@redhat.com>
Co-authored-by: Claude <noreply@anthropic.com>
Co-authored-by: Hongsheng Liu <liuhongsheng4@huawei.com>
 2026-05-16 23:34:04 +12:00
Nick CaoandClaude Opus 4.6 5313cf6d48 [Bugfix] Fix omni processing test for non-multimodal talker stage (#3559) 
Signed-off-by: Nick Cao <ncao@redhat.com>
Co-authored-by: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
 2026-05-13 10:08:23 +08:00
Nick CaoandClaude dca369d448 [Perf] Remove dead audio_tower and visual from Qwen2.5-Omni talker stage (#3425) 
Signed-off-by: Nick Cao <ncao@redhat.com>
Co-authored-by: Claude <noreply@anthropic.com>
 2026-05-12 23:24:13 +08:00
Nick CaoandClaude bd4ede391b [Perf] Remove dead audio_tower and visual from Qwen3-Omni talker stage (#3296) 
Signed-off-by: Nick Cao <ncao@redhat.com>
Co-authored-by: Claude <noreply@anthropic.com>
 2026-05-11 16:04:56 +00:00
Nick CaoandClaude 65bc968465 [Refactor] Rename SupportsModuleOffload to SupportsComponentDiscovery (#3354) 
Signed-off-by: Nick Cao <ncao@redhat.com>
Co-authored-by: Claude <noreply@anthropic.com>
 2026-05-08 14:16:49 +00:00
Nick Cao e6466cf0e6 [Refactor] Replace and ban a few torch.cuda functions in favor of torch.accelerator replacements. (#3365) 
Signed-off-by: Nick Cao <ncao@redhat.com>
 2026-05-07 23:23:59 +08:00
Nick CaoandClaude fde38ca54f [Refactor][Qwen3-TTS] Construct Code2Wav decoder natively (#2341) 
Signed-off-by: Nick Cao <ncao@redhat.com>
Co-authored-by: Claude <noreply@anthropic.com>
 2026-05-05 15:26:30 +00:00
Nick CaoandClaude 5fc0bfe01e [Cleanup] Use tokens_input() for TTS prompt construction (#3227) 
Signed-off-by: Nick Cao <ncao@redhat.com>
Co-authored-by: Claude <noreply@anthropic.com>
 2026-05-04 10:17:25 -04:00
Nick CaoandClaude 9de3806624 [Cleanup] Extract shared SinusPositionEmbedding/DiTTimestepEmbedding/timestep_embedding layer (#3285) 
Signed-off-by: Nick Cao <ncao@redhat.com>
Co-authored-by: Claude <noreply@anthropic.com>
 2026-05-01 20:19:42 +08:00
Nick CaoandClaude b9789091e9 [Bugfix][Qwen3TTS] Use float32 for code predictor on fp16-only GPUs (#3253) 
Signed-off-by: Nick Cao <ncao@redhat.com>
Co-authored-by: Claude <noreply@anthropic.com>
 2026-04-29 23:25:38 +00:00
Nick CaoandClaude e3b0afbb97 [Refactor] Let diffusion pipelines declare offloadable modules via SupportsModuleOffload (#2427) 
Signed-off-by: Nick Cao <ncao@redhat.com>
Co-authored-by: Claude <noreply@anthropic.com>
 2026-04-23 04:23:00 +08:00
Nick Cao 339a6da28c [Bugfix] treewide: drop references to librosa (#2996) 
Signed-off-by: Nick Cao <ncao@redhat.com>
 2026-04-21 16:04:15 +00:00
Nick Cao d2c23d7bd0 [Refactor] Remove resampy dependency (#2891) 
Signed-off-by: Nick Cao <ncao@redhat.com>
 2026-04-18 03:20:36 +00:00
Nick CaoandClaude f2edb81a59 [Cleanup] Remove dead runtime.defaults config parameters (#2343) 
Signed-off-by: Nick Cao <ncao@redhat.com>
Co-authored-by: Claude <noreply@anthropic.com>
 2026-04-17 23:36:41 +08:00
Nick CaoandClaude 1e3bb36521 [Refactor] Remove sox from dependencies (#2745) 
Signed-off-by: Nick Cao <ncao@redhat.com>
Co-authored-by: Claude <noreply@anthropic.com>
 2026-04-17 01:24:47 +00:00
Nick Cao eccee21c6c [Perf] Use global CUDA graph pool for MiMo Audio (#2657) 
Signed-off-by: Nick Cao <ncao@redhat.com>
 2026-04-11 13:45:26 +00:00
Nick CaoandClaude 2bc183f6f0 [Refactor] Remove dependency on librosa (#2273) 
Signed-off-by: Nick Cao <ncao@redhat.com>
Co-authored-by: Claude <noreply@anthropic.com>
 2026-04-11 02:11:31 +08:00
Nick CaoandClaude 3bd8a5239e [Bugfix] Enforce --max-generated-image-size on /v1/images/generations (#2599) 
Signed-off-by: Nick Cao <ncao@redhat.com>
Co-authored-by: Claude <noreply@anthropic.com>
 2026-04-09 03:54:12 +00:00
Nick CaoandClaude 149b9f179a [Perf][Qwen3-TTS][Voxtral-TTS] Share CUDA graph memory pool across decoder capture sizes (#2386) 
Signed-off-by: Nick Cao <ncao@redhat.com>
Co-authored-by: Claude <noreply@anthropic.com>
 2026-04-08 20:40:17 -04:00