Functional Correctness This metric measures whether the performance of any application performs what it is intended to do. This is easy to measure for deterministic systems. Similarity Measurement aginst reference data This checks how similar the LLM generated data is to the gold standard. The gold standard could be generated by AI or human. Exact match This can be one to one mapping of the answer or one to many mapping. ...
Rotary Position Embeddings (RoPE) State Space Model Mixutre of experts Supervised Finetuning (SFT) Preference finetuning Finetuning model to output responses that align with human preference. Reinforcement learning from human feedback (RLHF) Direct Preference Optimization (DPO) Reinforcement Learning from AI feedback (RLAIF) Proximal policy optimization (PPO)
Stopping condition For LLM applications, we can ask model to stop generating when it encounters the end-of-sequence token. Test time compute Instead of generating one answer to query, generate multiple using beam search and choose the best candidate. Best candidate can be chosen by letting humans choose or pick the output with highest probability. The probability of output is the product of the probabilities of all tokens in the output. ...
Pretraining evaluation metrics Cross entropy How different the predicted distribution is from the original distribution weighted by true probability. $$H(p, q) = -\sum_{i} p(i) \log q(i)$$ Where: $p$ = the true distribution (what actually happened) $q$ = the model’s predicted distribution Sum is over all possible classes/tokens $i$ Perplexity This measures how certain the model is in predicting next token. Lower value means it is certain while higher value means there are more equally likely options. ...
Decoding stragties are the methods that is used to choose the next token prediction in LLMs. Deterministic strategies Greedy decoding In this method, at every step token with the highest probability is chosen. The problem with this method is that this method choose locally optimal token but not globally optimal. This is used when the latency matters most and tasks has fairly unambiguous correct output like code completion for syntax, simple classification style generation. ...