Savepoints in Flink

Savepoint is another important feature in Flink, which takes it ahead of many of its competitors. Savepoint is a point in time snapshot that keeps track of where exactly we are in the processing of input streams and also holds its associated metadata. It also keeps track of all the pending states or in flight sessions in Flink execution engine. Conceptually its like taking a picture by literally stopping the stream data. By doing so, however, it doesn't actually stop the operation but does this silently in the background.

Internally Flink handles savepoints very much similar to checkpointing, but it does have some notable differences, as follows:

  • Triggering can be done manually by relevant configurations
  • Is never terminated by Flink until it is explicitly done by the user

The following figure shows the savepoint working in a pictorial fashion for easy understanding:

Figure 14: Savepoints (SP) in Flink

Savepoint features allows us to do many operations quite easily and give a versioning mechanism of the snapshots being taken at an internal. Some of those are:

  • Can help in applying Flink upgrade/bug fixes quite easily. If due to any reason, issue is seen, it can always use a previous savepoint to restore.
  • Can help in doing A/B testing. A savepoint can be used to spin off another Flink execution with variation and tested.
  • If you would like to reprocess data streams due to some reason or replay certain scenarios.

For more details, we would suggest going through the following blog http://data-artisans.com/how-apache-flink-enables-new-streaming-applications/.

..................Content has been hidden....................

You can't read the all page of ebook, please click here login for view all page.
Reset
18.188.85.135