Rotary Position Embeddings (RoPE)

State Space Model

Mixutre of experts

Supervised Finetuning (SFT)

Preference finetuning

Finetuning model to output responses that align with human preference.

  • Reinforcement learning from human feedback (RLHF)
  • Direct Preference Optimization (DPO)
  • Reinforcement Learning from AI feedback (RLAIF)
  • Proximal policy optimization (PPO)