Fine Tunning

Fine Tunning methods Self-supervised training: Foundational models are itself trained on self-supervision. Feature based transfer learning: Embeddings from foundational models are used for other taks or train a separate models. Infilling finetuning: Making a pre-trained autoregressive model to predict next token or middle tokens based on before or after tokens. Supervised finetuning (SFT): Pretrained model is further trained on the labeled dataset of input-output pairs. Reinforcement Learning: Finetune model to generate responses that maximize human preference. This requires the data format (instruction, winning response, losing response). ...