Beyond LoRA: Can you beat the most popular fine-tuning technique?

A recent Hugging Face Blog article explores whether alternative fine-tuning methods can surpass LoRA, the widely adopted parameter-efficient technique. It compares LoRA with newer approaches like DoRA, AdaLoRA, and full fine-tuning across metrics such as performance, memory usage, and training speed. The findings indicate that while LoRA remains a strong baseline, certain methods like DoRA can achieve slightly better accuracy on some tasks, though often at higher computational cost. The article emphasizes that the choice depends on specific use cases, with LoRA still recommended for resource-constrained settings. For AI workflow builders, this underscores the importance of benchmarking against baselines rather than blindly adopting new techniques.