Sink Processor

Sink Processor dictates how the Sink Group will function and achieve the load balancing or failover scenarios required by the reliability guarantee agreed for your Flume setup. Sink Processor is also a top level component in the Flume configuration. Broadly Sink Processor is classified into two:

  1. Built-in Sink Processor: These are processors present by default with Apache Flume.
    • Default Sink Processor:
      • Accepts only one sink.
      • Doesn't have to be explicitly put as a single sink has this processor by default.
    • Failover Sink Processor:
      • Keeps a prioritized list of sinks
      • Uses that priority to select the sink and makes sure that there is always a sink to process an event.
      • If an event fails while sending to a sink, the next event automatically selects the next sink in the priority list.
    • Load Balancing Sink Processor:
      • Keeps an indexed list of sink
      • When events come along it uses conventional load balancing approach of looking at the load and then distributing the load evenly as possible.
      • Round robin load balancing is defaulted
      • Apart from round robin Flume has another algorithm namely random.
  2. Custom Sink Processor: Does have support for this but not yet in there with the latest Flume release.
..................Content has been hidden....................

You can't read the all page of ebook, please click here login for view all page.
Reset
3.129.22.164