Protocol

Reconcile the result of a bulk write

A finished job is not an account of what changed. Compare the intended update with authoritative results and leave a repairable list of exceptions.

About 2 min read

Before you use it

Use when
  • A large update reports technical completion but business correctness still needs proof.

Keep this limit in mind

  • Reconciliation is only as strong as the intended-state specification; an incorrect source file can reconcile perfectly to the wrong goal.

Current editorial reading

Account for each intended change

Keep the approved keys and expected values. After the write, compare them with the target state and classify matches, missing changes, incorrect values and unexpected changes. Check downstream results when they form part of the intended outcome. For example, a customer update may finish in the main table while replication is still pending; record that state rather than treating it as confirmed failure.

Check the comparison itself

This is an editorial workflow for bulk updates. Google SRE discusses expected-versus-actual data checks and comparisons during pipeline canaries; it does not specify this complete post-write repair procedure. Define when the comparison is valid and account for concurrent edits. A perfect match to an incorrect input still produces the wrong business result.

Take it with you

Produce a key-level exception list with the expected value, observed state and next check. Resolve those exceptions before choosing a selective repair or another run.

Sources and evidence (1)

What we checked

Supports

Google SRE describes checking expected and actual pipeline output and monitoring discrepancies.

Selected official passages read on 2026-10-04. This supports the comparison principle, not every detail of the card's post-write repair workflow.

Improve and Optimize Data Processing Pipelines · Define and Measure Service Level Objectives > Data correctness

Provides context

The chapter compares canary or dry-run output with the live pipeline before rollout expansion.

Selected official passages read on 2026-10-04. A pre-expansion comparison is context for, rather than direct validation of, a post-write reconciliation procedure.

Improve and Optimize Data Processing Pipelines · Map Your Development Lifecycle > Canarying

Limits

Data correctness depends on the product and may require a separately defined expected result.

Selected official passages read on 2026-10-04. The exception-list design is an editorial recommendation; no production success or comprehensive correctness is established.

Improve and Optimize Data Processing Pipelines · Define and Measure Service Level Objectives > Data correctness; End-to-end measurement

Editorial sources (1)

Editorial version 1.0.0 · Evidence checked 2026-10-04

Save and practise this idea

My learning

Read. Recall. Repeat.

Save this idea and return to it on any device.

Use email
Keep exploring

Connected ideas

You are readingReconcile the result of a bulk write

Original edition evidence and sources

These records belong to the original released edition. The current editorial reading has its own evidence above; neither set silently replaces the other.

Supports

Google SRE recommends comparing canary or dry-run pipeline results with the live pipeline to check for data differences before expansion.

Reconciliation only catches differences represented by the comparison logic; missing invariants remain missing.

Improve and Optimize Data Processing Pipelines · Verification of canary or preproduction environment

All sources (1)
Original edition, downloads and provenance

The reading above is editorial version 1.0.0, bound to this exact source edition. The original data and saved downloads retain that edition and the separate editorial layer. They may contain earlier wording; use the current reading above when interpreting it.

Version 0.1.0

No review details are supplied for this edition.

Read the original edition and overlay as JSON · Release details