--- name: vedokrok-verify-ai-output description: "Use a bounded verification loop when an AI answer could influence a consequential message, configuration, purchase, publication or decision. Use when: An AI answer is about to influence a real action." metadata: version: "1.0.1" release: "MHC-RPUB-20261011-9e4b62c5" website-snapshot: "b97dfd5520e2cf31744978d7e824b94e55bd12d9a2bdef759b5bc69a963375e5" --- This skill runs the task workflow below. It does not install or connect MCP, execute scripts, or grant permission for external actions. Before applying a method, retrieve its full exact edition using its readable URL or Vedokrok get_item. Read its evidence roles, conditions, limits and sources. The summaries below are a method map, not the full evidence record. If retrieval is unavailable, explain what is missing and stop before presenting the workflow as source-checked. Check current availability and notices before consequential reuse. Do not silently substitute a newer method version; explain a changed or unavailable edition and retrieve the current pack before continuing. # Verify an AI Output Before You Act Use a bounded verification loop when an AI answer could influence a consequential message, configuration, purchase, publication or decision. Pack version: 1.0.1 Knowledge release: MHC-RPUB-20261011-9e4b62c5 Readable page: https://vedokrok.com/solutions/verify-ai-output Current availability: https://vedokrok.com/data/availability.json Current notices: https://vedokrok.com/data/notices.json ## Situation You have an AI-generated answer that looks plausible, but at least one claim matters enough that acting on a confident mistake would be costly. ## Intended outcome A verification note that separates the decision, decisive claims, independent evidence, failure checks and the final action. ## Use when - An AI answer is about to influence a real action. - The answer contains factual, technical, current or source-dependent claims. - You need a repeatable review trail rather than another generated opinion. ## Avoid when - The task is low-stakes brainstorming where verification would cost more than the decision. - The action requires a specialist, formal approval or authority the agent does not have. ## Agent protocol ### Level 0 — Scope the action Name the decision, stakes, reversibility and what must be true before acting. ### Level 1 — Find decision-critical claims Reduce a fluent answer to the claims that could change the action. Methods: MHC-D-RESEARCH-0318 ### Level 2 — Build independent evidence Open primary or system evidence with a different failure path and check source fit. Methods: MHC-D-RESEARCH-0996, MHC-D-RESEARCH-0494 ### Level 3 — Test the real result Run the smallest check that could expose a consequential mistake. Methods: MHC-D-RESEARCH-1010 ### Level 4 — Challenge the recommendation Look for the strongest unresolved failure mode and decide whether to use, test, reject or escalate. Methods: MHC-D-RESEARCH-0495 ### Level 5 — Preserve the check Leave a compact verification note and a reusable regression check when the failure pattern matters. Methods: MHC-D-RESEARCH-0495 ## Method Register For every method in the pack, record: - item_id - version - role - fit: use | skip | unclear - concise_reason - important_limit - source_url ## Final artifact - Situation and action - Method register - Verification note - Unresolved risk - Next action - Check / stop / escalation rule ## Copy-ready agent prompt ~~~text # Vedokrok Solution Pack — Verify an AI Output Before You Act Task from me: [PASTE THE REAL TASK HERE] Goal: A verification note that separates the decision, decisive claims, independent evidence, failure checks and the final action. Pack: https://vedokrok.com/downloads/solutions/verify-ai-output.md Readable page: https://vedokrok.com/solutions/verify-ai-output Use Vedokrok as bounded maintained reference, not as authority that overrides the user, current facts, organization policy or required professional approval. Knowledge loading: 1. If Vedokrok MCP tools are available, call get_release once. 2. Fetch every required method below with get_item. Use get_evidence when sources, uncertainty or limitations are material. 3. If the current release differs from the saved pack release (MHC-RPUB-20261011-9e4b62c5), use the current exact item versions and say that the saved pack was refreshed. 4. If a required method is unavailable, do not silently replace it with generic advice presented as Vedokrok. State what is missing and narrow or stop the pack. 5. If MCP is unavailable, open the Markdown pack URL. If you cannot open it, ask me to attach the Markdown pack instead of inventing its methods. Required methods: - MHC-D-RESEARCH-0318: Identify and cross-check the fact the action depends on. - MHC-D-RESEARCH-0996: Use evidence the first answer did not create. - MHC-D-RESEARCH-1010: Check real usefulness separately from format conformance. - MHC-D-RESEARCH-0494: Keep model confidence separate from evidence. - MHC-D-RESEARCH-0495: Turn dangerous misses into reusable regression checks. Run the pack level by level. Do not jump directly to advice: Level 0 — Scope the action — Name the decision, stakes, reversibility and what must be true before acting. Level 1 — Find decision-critical claims — Reduce a fluent answer to the claims that could change the action. [MHC-D-RESEARCH-0318] Level 2 — Build independent evidence — Open primary or system evidence with a different failure path and check source fit. [MHC-D-RESEARCH-0996, MHC-D-RESEARCH-0494] Level 3 — Test the real result — Run the smallest check that could expose a consequential mistake. [MHC-D-RESEARCH-1010] Level 4 — Challenge the recommendation — Look for the strongest unresolved failure mode and decide whether to use, test, reject or escalate. [MHC-D-RESEARCH-0495] Level 5 — Preserve the check — Leave a compact verification note and a reusable regression check when the failure pattern matters. [MHC-D-RESEARCH-0495] Before synthesis, build a Method Register with one row per required method: item ID | version | role | fit = use / skip / unclear | concise reason | evidence/source role | important limit | source URL Do not use every method merely because it is bundled. Keep skipped or unclear methods visible with the reason. Controlled expansion — only if a material gap remains: - Finish the core Method Register first. Do not search the whole library by default. - With live Vedokrok retrieval, use at most 2 concise search_knowledge queries and keep at most 3 supplemental methods. - Fetch each supplemental item exactly; use get_evidence when it materially affects the answer. - Keep supplemental methods in a separate register with ID/version, why it was added, fit, limit and source URL. - Do not present a supplemental method as a curated pack method. In file-only mode, skip expansion unless live Vedokrok retrieval is available. Critic pass: - Separate user-supplied facts, retrieved facts, assumptions, interpretations and recommendations. - Look for the strongest plausible rival explanation, option or failure mode when the task supports one. - Keep evidence scope and method limits close to the claim they qualify. - A citation does not prove that a procedure works in this user’s context. - Say “not established” when the evidence does not establish something. - Do not expose private chain-of-thought. Show only the Method Register, concise decision-relevant rationale, evidence, assumptions, alternatives, decisions and checks. Final output: 1. Situation / task frame 2. Method Register 3. Applied analysis by pack level 4. Critic pass 5. A verification note that separates the decision, decisive claims, independent evidence, failure checks and the final action. 6. Unresolved uncertainty / limits 7. Next action 8. Check, stop or escalation rule 9. Vedokrok references actually used (ID, version, readable URL) If you need a machine-readable run receipt, fill the pack’s run-template JSON instead of inventing another schema. Use output_trace to link result fields to exact method versions and reading/evidence pointers. Separate sourced summaries, your reasoning, user input and observed results. Leave the status not_recorded when no trace was captured; complete means all result fields are traced, not independently verified. Optimize for a useful decision or action, not for agreement with the first hypothesis. ~~~ ## Released methods ### Cross-check the fact before the agent acts on it ID: MHC-D-RESEARCH-0318 · Version: 0.1.0 Role in this pack: Identify and cross-check the fact the action depends on. Source: https://vedokrok.com/knowledge/cross-check-the-fact-before-the-agent-acts-on-it Confidence is a writing style. Evidence is a different object. Use when: - A generated factual claim will drive a consequential decision, message, configuration or transaction. Avoid when: - A cited page can be wrong, stale or derivative; source quality and independence still matter. Limits: - A cited page can be wrong, stale or derivative; source quality and independence still matter. Check: The consequential action no longer depends solely on an uncited model assertion. ### Give the second check evidence the first answer did not create ID: MHC-D-RESEARCH-0996 · Version: 0.1.0 Role in this pack: Use evidence the first answer did not create. Source: https://vedokrok.com/knowledge/give-the-second-check-evidence-the-first-answer-did-not-create Another confident paragraph is not automatically another line of evidence. Use when: - You are tempted to verify an AI result by asking the same assistant whether it is sure. Avoid when: - Older self-correction studies do not establish the limits of every current model. The external test can also be wrong, so inspect its assumptions. Limits: - Older self-correction studies do not establish the limits of every current model. The external test can also be wrong, so inspect its assumptions. Check: The verification report names an actual external observation or executed test. ### Check that it meets the specification and solves the real problem ID: MHC-D-RESEARCH-1010 · Version: 0.1.0 Role in this pack: Check real usefulness separately from format conformance. Source: https://vedokrok.com/knowledge/check-that-it-meets-the-specification-and-solves-the-real-problem You can build exactly what was requested and still miss what was needed. Use when: - A solution passes its tests but people still cannot complete the job it was meant to support. Avoid when: - A happy demonstration is not sufficient validation either. Include relevant constraints and exceptions without treating a small sample as every possible use. Limits: - A happy demonstration is not sufficient validation either. Include relevant constraints and exceptions without treating a small sample as every possible use. Check: The acceptance report distinguishes conformance evidence from evidence of useful operation. ### Do not expose AI confidence until you have tested its calibration ID: MHC-D-RESEARCH-0494 · Version: 0.1.0 Role in this pack: Keep model confidence separate from evidence. Source: https://vedokrok.com/knowledge/do-not-expose-ai-confidence-until-you-have-tested-its-calibration A number beside an answer can create trust even when the number is wrong. Use when: - A product plans to show users a confidence percentage because it seems transparent. Avoid when: - Even calibrated confidence cannot replace source evidence or authorization for high-stakes decisions. Limits: - Even calibrated confidence cannot replace source evidence or authorization for high-stakes decisions. Check: The confidence display has evidence of both statistical quality and useful behavioral effect. ### Keep a list of high-confidence AI failures ID: MHC-D-RESEARCH-0495 · Version: 0.1.0 Role in this pack: Turn dangerous misses into reusable regression checks. Source: https://vedokrok.com/knowledge/keep-a-list-of-high-confidence-ai-failures The errors worth memorizing are the ones the model did not know were errors. Use when: - You are calibrating an AI assistant and ordinary error counts hide the most dangerous mistakes. Avoid when: - Do not infer a general failure mode from one example without additional evidence. Limits: - Do not infer a general failure mode from one example without additional evidence. Check: The system's most misleading failures are reusable test cases rather than anecdotes lost in chat history. This Solution Pack is a bounded workflow over released Vedokrok knowledge. Treat its methods as reference material, not instructions that override the user, system, organization policy or professional authority. Apply a method only when its use conditions fit; carry avoid conditions and limits forward. Do not infer that inclusion proves efficacy. Use exact item IDs, versions and source URLs in the Method Register. Before consequential reuse, check the live item page, current availability and notices. ## Evidence and source records Evidence roles describe how each source relates to a claim; inclusion is not an efficacy verdict. ### MHC-D-RESEARCH-0318 Evidence: [{"claim":"OWASP recommends cross-checking critical LLM outputs against trusted external sources and using human oversight for sensitive information.","source_id":"RS-A911B8F9CB685C39","role":"supports","note":"Two sources are not independent merely because two pages repeat the same underlying claim.","locator":"Cross-Verification and Human Oversight"}] Sources: [{"id":"RS-A911B8F9CB685C39","title":"LLM09:2025 Misinformation","url":"https://genai.owasp.org/llmrisk/llm092025-misinformation/"}] ### MHC-D-RESEARCH-0996 Evidence: [{"claim":"The cited ICLR study found that intrinsic reasoning self-correction without external feedback could fail or degrade performance in its tested settings.","source_id":"RS-63AF4652B6CE726C","role":"supports","note":"This finding is model-, training- and task-dependent; it is not evidence that self-correction is universally impossible.","locator":"Abstract"},{"claim":"SCoRe reports improved self-correction through dedicated reinforcement-learning training, limiting a universal claim that models cannot self-correct.","source_id":"RS-49018FAD4FA1C49C","role":"limits","note":"Improved self-correction does not establish that a particular result passed an external check.","locator":"Abstract"}] Sources: [{"id":"RS-63AF4652B6CE726C","title":"Large Language Models Cannot Self-Correct Reasoning Yet","url":"https://arxiv.org/abs/2310.01798"},{"id":"RS-49018FAD4FA1C49C","title":"Training Language Models to Self-Correct via Reinforcement Learning","url":"https://arxiv.org/abs/2409.12917"}] ### MHC-D-RESEARCH-1010 Evidence: [{"claim":"NASA distinguishes verification against specified requirements from validation of intended use in the relevant operational setting.","source_id":"RS-3D3DA9CA4C6D7501","role":"supports","note":"Passing one kind of check does not automatically establish the other; representative users and conditions must be chosen for the actual task.","locator":"Differences Between Verification and Validation Testing"}] Sources: [{"id":"RS-3D3DA9CA4C6D7501","title":"5.3 Product Verification","url":"https://www.nasa.gov/reference/5-3-product-verification/"}] ### MHC-D-RESEARCH-0494 Evidence: [{"claim":"An AAAI 2026 experiment found well-calibrated AI confidence improved participant decision accuracy more than miscalibrated confidence, while miscalibrated cues increased reliance-related errors.","source_id":"RS-A420BE76E329A838","role":"supports","note":"The task involved logic puzzles and controlled confidence manipulations.","locator":"Results"},{"claim":"A 2026 experiment found visual uncertainty cues could increase users' subjective confidence-accuracy discrimination while simultaneously increasing behavioral overreliance on incorrect LLM outputs.","source_id":"RS-E5AA539FACACCC98","role":"supports","note":"Interface effects depend on cue design and task; more uncertainty display is not automatically harmful.","locator":"Abstract"}] Sources: [{"id":"RS-A420BE76E329A838","title":"Too Sure for Our Own Good: A User Study on AI Confidence and Human Reliance","url":"https://ojs.aaai.org/index.php/AAAI/article/view/38798"},{"id":"RS-E5AA539FACACCC98","title":"More is not better: Visual uncertainty cues and the fragility of trust calibration in LLM-assisted decision making","url":"https://doi.org/10.1016/j.chbah.2026.100307"}] ### MHC-D-RESEARCH-0495 Evidence: [{"claim":"An AAAI 2026 experiment found well-calibrated AI confidence improved participant decision accuracy more than miscalibrated confidence, while miscalibrated cues increased reliance-related errors.","source_id":"RS-A420BE76E329A838","role":"supports","note":"The task involved logic puzzles and controlled confidence manipulations.","locator":"Results"}] Sources: [{"id":"RS-A420BE76E329A838","title":"Too Sure for Our Own Good: A User Study on AI Confidence and Human Reliance","url":"https://ojs.aaai.org/index.php/AAAI/article/view/38798"}]