Protocol
Reconcile the result of a bulk write
A finished job is not an account of what changed. Compare the intended update with authoritative results and leave a repairable list of exceptions.
Before you use it
- Use when
- A large update reports technical completion but business correctness still needs proof.
Keep this limit in mind
- Reconciliation is only as strong as the intended-state specification; an incorrect source file can reconcile perfectly to the wrong goal.
Current editorial reading
Account for each intended change
Keep the approved keys and expected values. After the write, compare them with the target state and classify matches, missing changes, incorrect values and unexpected changes. Check downstream results when they form part of the intended outcome. For example, a customer update may finish in the main table while replication is still pending; record that state rather than treating it as confirmed failure.
Check the comparison itself
This is an editorial workflow for bulk updates. Google SRE discusses expected-versus-actual data checks and comparisons during pipeline canaries; it does not specify this complete post-write repair procedure. Define when the comparison is valid and account for concurrent edits. A perfect match to an incorrect input still produces the wrong business result.
Take it with you
Produce a key-level exception list with the expected value, observed state and next check. Resolve those exceptions before choosing a selective repair or another run.
Sources and evidence (1)
What we checked
Google SRE describes checking expected and actual pipeline output and monitoring discrepancies.
Selected official passages read on 2026-10-04. This supports the comparison principle, not every detail of the card's post-write repair workflow.
Improve and Optimize Data Processing Pipelines · Define and Measure Service Level Objectives > Data correctness
The chapter compares canary or dry-run output with the live pipeline before rollout expansion.
Selected official passages read on 2026-10-04. A pre-expansion comparison is context for, rather than direct validation of, a post-write reconciliation procedure.
Improve and Optimize Data Processing Pipelines · Map Your Development Lifecycle > Canarying
Data correctness depends on the product and may require a separately defined expected result.
Selected official passages read on 2026-10-04. The exception-list design is an editorial recommendation; no production success or comprehensive correctness is established.
Improve and Optimize Data Processing Pipelines · Define and Measure Service Level Objectives > Data correctness; End-to-end measurement
Editorial sources (1)
Save and practise this idea
My learning
Read. Recall. Repeat.
Save this idea and return to it on any device.
Keep exploring
Connected ideas
You are readingReconcile the result of a bulk write
Ideas that link here
These ideas link to this one. Check their conditions and limits before use.
Use before this ideaStage the mutation before committing itUseful with this idea
Carry data lineage through derived outputs
Original edition evidence and sources
These records belong to the original released edition. The current editorial reading has its own evidence above; neither set silently replaces the other.
Google SRE recommends comparing canary or dry-run pipeline results with the live pipeline to check for data differences before expansion.
Reconciliation only catches differences represented by the comparison logic; missing invariants remain missing.
Improve and Optimize Data Processing Pipelines · Verification of canary or preproduction environment
All sources (1)
Original edition, downloads and provenance
The reading above is editorial version 1.0.0, bound to this exact source edition. The original data and saved downloads retain that edition and the separate editorial layer. They may contain earlier wording; use the current reading above when interpreting it.
Version 0.1.0
No review details are supplied for this edition.
Read the original edition and overlay as JSON · Release details