适配器即模型 —— ShadowPEFT 现已集成到 🤗 PEFT 库

ShadowPEFT is a parameter-efficient fine-tuning method newly integrated into Hugging Face’s PEFT library in version 0.21.0, designed as an alternative to LoRA and DoRA that treats adaptation as a compact, stateful “shadow” model rather than isolated low-rank weight updates. While the pretrained backbone remains frozen, the shadow network maintains and updates a task-specific hidden state across Transformer layers, injecting information into the backbone and using updated backbone representations to refine its own state. This bidirectional design is intended to improve cross-layer coordination, enable scaling through the size of the task-specific shadow model, and allow the trained adapter to be detached with `unload_shadow()` and run independently, including potential edge-cloud deployment scenarios. ShadowPEFT uses familiar PEFT interfaces such as `get_peft_model`, `save_pretrained`, and `from_pretrained`, and can initialize its shadow component from either a reduced mirror of the base model or a separate pretrained smaller model. Reported experiments on mathematical reasoning and DreamBooth image generation show higher accuracy or image fidelity than LoRA and DoRA at comparable or lower trainable parameter counts and checkpoint sizes, although its persistent shadow computation can require more memory in language-model training.

Use this post, company, and trend context to find content marketing opportunities, perform competitive analysis, or address product feature gaps via the Plushcap MCP server or the Plushcap API.