Domain 2 · lesson
Output Evaluation and Validation
Learning goal: decide whether an output deserves trust for its intended use. Attractive formatting and fluent prose are not substitutes for evaluation. S09 S10
Objective 2.1
Evaluate accuracy and completeness
Core content
Accuracy asks whether claims are correct. Completeness asks whether all relevant requirements, conditions, and exceptions are covered. A response can be accurate yet materially incomplete. Evaluate relevance, format, clarity, safety, and usefulness separately when they matter. A rubric should define observable criteria and critical failures. S09
Applied example
An incident summary correctly states the date and suspected cause but omits that the cause remains provisional. The visible facts are accurate, but the omission changes the meaning. A test-case table may have perfect columns yet omit all negative scenarios: format passes, coverage fails.
Exam decision rule
Map each requirement to the output and each important claim to evidence. A small representative set with reviewable criteria is often more informative than hundreds of ungraded outputs.
Objective 2.2
Detect hallucinations, contradictions, and bias
Core content
A hallucination introduces unsupported or false information. A contradiction presents incompatible claims. Bias may create unjustified differences in treatment, selection, or representation. Not every omission is a hallucination, and disagreement between sources does not by itself prove fabrication. Return to the evidence. S09 S10 S26
Applied example
A report says a ticket was resolved "on Tuesday" when no source contains a date: invented detail. Another says both "no users affected" and "twelve accounts blocked": contradiction. A process routes requests from one group to extra scrutiny without a relevant reason: possible bias requiring evaluation.
Exam decision rule
Apparent precision in names, numbers, or quotations must be checked. Multiple identical outputs from the same model are not independent confirmation.
Objective 2.3
Apply fact-checking and source verification
Core content
Break an answer into verifiable claims and use appropriate sources. A citation must exist, refer to the relevant version, and actually support the claim. For internal facts, use the authorized source of truth. For current product behavior, use current vendor documentation. For calculations, use a reproducible check. Confidence language is not verification. S10
Applied example
A draft claims "satisfaction increased 10%" based on a movement from 70 to 80 points. Determine whether the intended statement is ten percentage points or roughly 14.3% relative growth. The source values may be real while the interpretation is wrong.
Exam decision rule
Verify the claim that drives the decision first. If only part of a document is available, do not assume the missing section contains no exception. Record the limitation, retrieve the relevant section, or avoid the conclusion.
Objective 2.4
Determine when human review is required
Core content
Set review requirements according to consequence and required expertise: reversible internal work, external publication, sensitive data, decisions affecting people, regulated advice, or high-impact use. The reviewer needs evidence, responsibility, and the authority to stop or correct the action. "Human reviewed" without a process proves very little. S26
Applied example
Draft support replies may be checked by trained support staff before sending. An interpretation of a contractual obligation requires a competent owner, not approval from any random employee who happens to be available.
Exam decision rule
Choose the reviewer who has the expertise and authority relevant to the risk. A rubber-stamp approval is not meaningful oversight.
Objective 2.5
Edit, adapt, and compare outputs for the audience
Core content
Adaptation can change length, structure, vocabulary, and emphasis, but it must not silently change evidence, scope, uncertainty, or commitments. Compare candidates under common criteria using the same source material. S09
Applied example
Turning a technical incident analysis into an executive summary may remove implementation detail, but it must preserve the material exception that the root cause is still under investigation. Editing for confidence must not turn a hypothesis into a fact.
Exam decision rule
Start from the version that preserves evidence and required conditions, then edit for audience. Do not trade factual integrity for tone.
Objective 2.6
Curate information and select output format
Core content
Curation means selecting useful material while preserving provenance, uncertainty, and the distinction between known, missing, inferred, and recommended information. Choose a format that supports the downstream decision: table, checklist, short memo, structured fields, or artifact. S10 S15
Applied example
A reusable report should keep references, mark missing numeric values as unknown rather than zero, and separate verified findings from recommendations. A polished chart that converts unknown values to zero is visually tidy and semantically wrong.
Exam decision rule
Optimize for correct reuse, not decorative compression. Missing data must remain visibly missing.