Question
Choose the clock before assigning an event to a period
Arrived today and happened today are different statements.
Before you use it
- Use when
- Late messages or delayed processing move events into the wrong reporting day.
Keep this limit in mind
- Source clocks and timestamps can be wrong. A streaming watermark estimates progress; it does not make late arrival impossible.
An idea to explore.
Try this idea.
Inspect the example, try the idea in your situation, then check your next move.
A question to ask
Does this report concern when work happened or when the system received it? · Which time zone and cutoff define the period? · How will late events correct or annotate an already issued result?
Why it matters
Keep occurrence time and processing time separate. Decide which clock answers the business question, and define what happens when an event arrives after a reporting cutoff. Ordering records by arrival cannot reconstruct the event sequence by itself.
An example
A service event occurs before midnight but reaches reporting after midnight. Its business date follows the agreed occurrence-time rule, not an accidental queue delay.
Check your result
A deliberately late test event lands in the expected period or follows the documented correction route.
Save and practise this idea
My learning
Read. Recall. Repeat.
Save this idea and return to it on any device.
Keep exploring
Connected ideas
You are readingChoose the clock before assigning an event to a period
Evidence and sources
Event time records when an event occurred, while processing time records when a system processes it; arrival order need not follow event order.
Choosing the reporting clock and late-data correction policy remains a domain decision.
Apache Beam Programming Guide · Section 8.4
All sources (1)
Edition and review
Version 0.1.0
No review details are supplied for this edition.