December 2024 Summaries

ComfyUI has released version 0.3.10, which significantly reduces the VRAM requirements for running the HunyuanVideo model, lowering it from 32GB to just 8GB, through the introduction of temporal tiling in video VAEs. Temporal tiling differs from spatial tiling by expanding each temporal latent dimension by 4x, except for the last one, allowing for more efficient video processing. Additionally, ComfyUI now supports PixArt models, including Alpha and Sigma versions, for high-quality diffusion image generation, thanks to community contributions. Users can update to the latest version of ComfyUI to access these features and are invited to join bi-weekly office hours on Discord for discussions, feedback, and community interaction.

HunyuanVideo, a state-of-the-art 13-billion-parameter open-source video foundation model, is now integrated into ComfyUI, offering advanced features for image and video generation. This model employs a "Dual-stream to Single-stream" Transformer to seamlessly blend text and visuals, enhancing motion consistency and image quality. It also boasts superior text-video alignment through the MLLM text encoder and efficient video compression using a custom 3D VAE, which maintains resolution and frame rate while reducing data size. Enhanced prompt control is provided via the Prompt Rewrite model, which includes Normal and Master Modes to optimize user intent interpretation and improve visual output. Users can easily generate videos and images using HunyuanVideo by updating to the latest version of ComfyUI, downloading specific model files, and loading workflows into the interface.

ComfyUI has introduced native localization support to replace third-party translation extensions, enhancing performance, reliability, and maintainability by allowing users to select languages such as English, Chinese (Simplified), Russian, Japanese, and Korean directly from the interface. Previously, the AIGODLIKE-ComfyUI-Translation extension, which had performance and compatibility issues, was required for non-English translations. The new system integrates with vue-i18n for efficient locale message display and offers direct UI integration without the need for DOM matching, addressing previous limitations like translation persistence and compatibility challenges. The current support covers various UI elements, with plans to expand language options and implement translation support for custom nodes. Community contributions are encouraged for further improvements and expanding language support, with future additions to include Spanish, French, and German translations.

ComfyUI has introduced native support for masking and scheduling Low-Rank Adaptation (LoRA) and model weights within its conditioning system, enhancing customization and flexibility for users. This update includes the ability to hook specific LoRA and model weights to CLIP and conditioning, allowing for detailed control over how weights are applied during the text encoding and sampling processes. The system employs nodes such as 'Create Hook LoRA' and 'Create Hook Model as LoRA' to facilitate the application of weights, and new nodes like 'Cond Pair Combine' streamline the process of merging conditioning inputs. Moreover, the update introduces Hook Keyframes to schedule weight strengths dynamically across sampling steps, enabling nuanced control over model and LoRA application throughout the diffusion process. While masking allows for distinct weight assignments to different image areas, scheduling offers temporal control over weight application, enhancing the ability to achieve desired model outputs with precision.

In December 2024, ComfyUI issued a statement regarding the discovery of a crypto miner in versions v8.3.41 and v8.3.42 of the ultralytics pip package, which only affects Mac and Linux users, likely targeting servers rather than regular users. Windows users are not impacted. The compromised package, which is a dependency for popular custom nodes like the ComfyUI-Impact-Pack, involves the download and execution of a binary miner on affected systems. Users can determine their exposure by updating the ComfyUI manager, which now checks for and warns about these vulnerabilities. To mitigate the impact, users should kill the relevant process, delete the file, and remove the compromised package versions. In response, ComfyUI has updated its manager to flag these issues and pin the ultralytics version to 8.3.40, considered safe, while planning future sandboxing protections for their desktop app. The team expressed gratitude for the community's prompt reaction to this security issue and offered various channels for further inquiries.