Evaluation

Domain-specific capability Functionality of the AI system determines the domain specific capability. Like for coding system, functional correctness, readbility of code etc are important. Usually for open ended systems, MCQ or classification kind of evluation methods are used. Generation capability Fluency: Coherence: Faithfulness: Relevance: Factual consistency: The model’s output is verified against the know facts (local factual consistency) or against open knowledge (Global factual consistency). Self-verification: If a model generates multiple outputs that disagree with one another, the original output islikely hallucinated. ...