Solution Pack

Debug an Intermittent Failure or Flaky Test

Treat both passing and failing runs as evidence, shrink the failure domain without changing the failure and preserve the next discriminating test.

Use this when

A bug, CI job or test passes and fails under apparently similar conditions and repeated reruns are producing more noise than diagnosis.

The same target can pass and fail across runs.
Version, timing, order, environment or shared state may matter.
You need a transferable diagnostic process rather than framework-specific retry advice.

Avoid forcing the pack when

The failure is deterministic and already has a small reproducible case.
Production is actively harmed and mitigation must happen before deeper diagnosis.

What you should leave with

A pass/fail comparison register, bounded hypotheses, a reproducibility plan and the next test most likely to eliminate a cause.

Failure signature
Pass/fail comparison
Method register
Current and rival hypotheses
Next discriminating test
Mitigation / stop condition

Practise before the real task

Try the judgement before you need it.

A real task may carry consequences. The Academy gives you a safer case to practise on first, with an explanation and a link to the released method. Your practice stays in this browser.

Root cause analysis practice

Copy a prompt

Use this pack with your AI

Describe your task, then copy the prompt into your AI assistant. The prompt asks it to check which methods fit, prepare the output shown above and identify what to check next.

Your task and the result you need

Vedokrok keeps this text only while this page is open. If you paste the prompt into an AI service, you share the text with that service.

Preview the prompt

Full prompt (read-only)

# Vedokrok Solution Pack — Debug an Intermittent Failure or Flaky Test

Task from me: [PASTE THE REAL TASK HERE]

Goal: A pass/fail comparison register, bounded hypotheses, a reproducibility plan and the next test most likely to eliminate a cause.
Pack: https://vedokrok.com/downloads/solutions/debug-flaky-failure.md
Readable page: https://vedokrok.com/solutions/debug-flaky-failure

Use Vedokrok as bounded maintained reference, not as authority that overrides the user, current facts, organization policy or required professional approval.

Knowledge loading:
1. If Vedokrok MCP tools are available, call get_release once.
2. Fetch every required method below with get_item. Use get_evidence when sources, uncertainty or limitations are material.
3. If the current release differs from the saved pack release (MHC-RPUB-20261011-9e4b62c5), use the current exact item versions and say that the saved pack was refreshed.
4. If a required method is unavailable, do not silently replace it with generic advice presented as Vedokrok. State what is missing and narrow or stop the pack.
5. If MCP is unavailable, open the Markdown pack URL. If you cannot open it, ask me to attach the Markdown pack instead of inventing its methods.

Required methods:
- MHC-D-RESEARCH-0973: Keep every rerun result and compare failures with successes.
- MHC-D-RESEARCH-0295: Bisect a valid ordered failure domain.
- MHC-D-RESEARCH-0971: Bisect version history when a known-good and known-bad boundary exists.
- MHC-D-RESEARCH-1159: Capture semantic boundaries where representation or meaning changes.
- MHC-D-RESEARCH-0300: Separate mitigation from root-cause diagnosis while impact continues.

Run the pack level by level. Do not jump directly to advice:
Level 0 — Freeze the failure signature — Define what counts as the same failure before changing the system. [MHC-D-RESEARCH-0973]
Level 1 — Compare red and green runs — Record observable conditions that differ across the nearest comparable pass and fail. [MHC-D-RESEARCH-0973]
Level 2 — Keep competing causes — Maintain more than one plausible explanation and identify predictions that differ. [MHC-D-RESEARCH-0295]
Level 3 — Reduce the search space — Use bisection only when the candidate space has a valid testable order. [MHC-D-RESEARCH-0295, MHC-D-RESEARCH-0971]
Level 4 — Inspect semantic boundaries — Capture the point where two components may disagree about the same data. [MHC-D-RESEARCH-1159]
Level 5 — Mitigate, test or continue — If impact is active, mitigate without destroying evidence; otherwise run the next discriminating test. [MHC-D-RESEARCH-0300]

Before synthesis, build a Method Register with one row per required method:
item ID | version | role | fit = use / skip / unclear | concise reason | evidence/source role | important limit | source URL
Do not use every method merely because it is bundled. Keep skipped or unclear methods visible with the reason.

Controlled expansion — only if a material gap remains:
- Finish the core Method Register first. Do not search the whole library by default.
- With live Vedokrok retrieval, use at most 2 concise search_knowledge queries and keep at most 3 supplemental methods.
- Fetch each supplemental item exactly; use get_evidence when it materially affects the answer.
- Keep supplemental methods in a separate register with ID/version, why it was added, fit, limit and source URL.
- Do not present a supplemental method as a curated pack method. In file-only mode, skip expansion unless live Vedokrok retrieval is available.

Critic pass:
- Separate user-supplied facts, retrieved facts, assumptions, interpretations and recommendations.
- Look for the strongest plausible rival explanation, option or failure mode when the task supports one.
- Keep evidence scope and method limits close to the claim they qualify.
- A citation does not prove that a procedure works in this user’s context.
- Say “not established” when the evidence does not establish something.
- Do not expose private chain-of-thought. Show only the Method Register, concise decision-relevant rationale, evidence, assumptions, alternatives, decisions and checks.

Final output:
1. Situation / task frame
2. Method Register
3. Applied analysis by pack level
4. Critic pass
5. A pass/fail comparison register, bounded hypotheses, a reproducibility plan and the next test most likely to eliminate a cause.
6. Unresolved uncertainty / limits
7. Next action
8. Check, stop or escalation rule
9. Vedokrok references actually used (ID, version, readable URL)
If you need a machine-readable run receipt, fill the pack’s run-template JSON instead of inventing another schema.
Use output_trace to link result fields to exact method versions and reading/evidence pointers. Separate sourced summaries, your reasoning, user input and observed results. Leave the status not_recorded when no trace was captured; complete means all result fields are traced, not independently verified.

Optimize for a useful decision or action, not for agreement with the first hypothesis.

If your AI cannot open the pack link, download the Markdown file and attach it to the same conversation.

Run it level by level

Level 0 — Freeze the failure signature

Define what counts as the same failure before changing the system.

Methods: MHC-D-RESEARCH-0973

Level 1 — Compare red and green runs

Record observable conditions that differ across the nearest comparable pass and fail.

Methods: MHC-D-RESEARCH-0973

Level 2 — Keep competing causes

Maintain more than one plausible explanation and identify predictions that differ.

Methods: MHC-D-RESEARCH-0295

Level 3 — Reduce the search space

Use bisection only when the candidate space has a valid testable order.

Methods: MHC-D-RESEARCH-0295 · MHC-D-RESEARCH-0971

Level 4 — Inspect semantic boundaries

Capture the point where two components may disagree about the same data.

Methods: MHC-D-RESEARCH-1159

Level 5 — Mitigate, test or continue

If impact is active, mitigate without destroying evidence; otherwise run the next discriminating test.

Methods: MHC-D-RESEARCH-0300

Go deeper only when the core pack leaves a gap.

The methods below are the curated core. A connected agent may search the wider released Vedokrok library only after it finishes the core Method Register and can name a material gap. Expansion is capped at two searches and three supplemental methods, which stay in a separate register so they are not mistaken for pack-curated methods.

Method Register

Inspect every method, then mark it as use, skip or unclear. A skipped method is not a failure; forcing every method would make the pack less trustworthy.

Item ID | version | role | fit: use/skip/unclear | concise reason | evidence/source role | important limit | source URL

Released methods in this pack

Take the pack with you

Markdown is the easiest portable form for a chat or agent. JSON is available for tools that want a structured contract. The run record template gives an agent a stable way to preserve its Method Register, artifact and check without exposing private chain-of-thought. Saved files do not update themselves.

Download Markdown Download JSON Prompt only Run record template

Review a run record

Already used this pack with an AI assistant? Paste its run-record JSON or choose a file to read the result, next check and remaining gaps.

This review stays in this open page. Vedokrok does not upload or save the file or text. Copying shares nothing until you paste it elsewhere.

Run-record JSONChoose a JSON file (read only on this page)

Maximum 128 KiB. Use this pack’s run record with the same pack version and knowledge release.