LLM Guide
  • An introduction to Large Language Models
  • Understanding Pre-trained Language Models
  • Solutions to LLM Challenges
    • Prompt Engineering
    • Neuro-Symbolic Methods
    • Retrieval-Augmented Generation (RAG)
    • Honorable Mentions
  • Fine-Tuning
  • Supervised Fine-Tuning Strategies
    • Full Parameter Fine-Tuning
    • Half Fine-Tuning (HFT)
    • Parameter-Efficient Fine-Tuning (PEFT)
      • LoRA (Low-Rank Adaptation)
      • QLoRA (Quantized LoRA)
      • DoRA (Decomposed Low-Rank Adaptation)
      • NEFTune (Noise-Enhanced Fine-Tuning)
  • Fine-tuning Best Practices
  • Fine-tuning Using Ubiai (No-Codesolution)
  • Evaluation of Fine-Tuned Models
    • Evaluation Techniques
    • Task specific Evaluation Metrics
    • Popular Benchmarks
    • Best Practices for Model Evaluation
  • Directory of Links By Section
Powered by GitBook
On this page
  1. Supervised Fine-Tuning Strategies

Parameter-Efficient Fine-Tuning (PEFT)

PreviousHalf Fine-Tuning (HFT)NextLoRA (Low-Rank Adaptation)

Last updated 4 months ago

Parameter-efficient fine-tuning (PEFT) is a method designed to adapt large pre-trained models for specific tasks while minimizing the number of parameters that need to be updated. Unlike traditional fine-tuning approaches, such as full fine-tuning or "half fine-tuning," where you freeze some layers and update the rest of the model, PEFT focuses on freezing most of the model's parameters while only modifying a small subset of them. This could include the addition of task-specific adapters or updates to certain layers, significantly reducing the number of parameters that need to be trained.

The concept of Parameter-Efficient Fine-Tuning (PEFT) has significantly lowered the barriers to applying large language models (LLMs) in development. This has sparked a wide range of research into various methods for achieving PEFT. These methods can be classified into three main categories:

  1. Selective Fine-Tuning: This approach focuses on updating a carefully chosen subset of a pre-trained model's parameters, rather than fine-tuning the entire model. This method enables more efficient adaptation to specific tasks.

  2. Additive Fine-Tuning: New modules are added to the pre-trained model for fine-tuning. These modules are then trained to incorporate domain-specific knowledge, allowing the model to adapt to new tasks while preserving the original model's capabilities.

  3. Reparameterization: Where a low-dimensional representation is created for specific model components. This reduces the complexity of the fine-tuning process by working with a smaller set of parameters.

In this section, we will be exploring a few of the most effective techniques for applying parameter-efficient fine-tuning (PEFT).

LoRA (Low-Rank Adaptation)
QLoRA (Quantized LoRA)
DoRA (Decomposed Low-Rank Adaptation)
NEFTune (Noise-Enhanced Fine-Tuning)