Windowing

Windowing is the concept that is used to answer questions such as, "What happened in the last x minutes?" In case of an unbounded stream, we can't really get the global view on the entire stream of data. We can only get the view for a specified time window. In stream processing systems, this time window is defined as the current time. As discussed, the time of an event can actually be divided into three aspects:

  • Event time: The time an event actually occurred
  • Ingestion time: The time at which the event was ingested into the processing system
  • Processing time: The set of times at which each of the processing jobs processed the event

Event time, while most useful, is also the most difficult to handle as it gives the least guarantees; events may arrive out of order or late, so we can never be sure whether we saw all the events in a given time window. Processing time is easiest, as it is monotonic: you know precisely when a five-minute window ended (by looking at the clock).

..................Content has been hidden....................

You can't read the all page of ebook, please click here login for view all page.
Reset
3.15.2.78