Flink has many connectors for easy integration that can be broadly categorized as follows:
- Built-in Connectors: Connectors that are already there along with the Flink installation and developed by various providers and are already supported and maintained come under this category:
- File: Reads file from a specified file path. It can monitor a directory and as and when a file is added or changed can read and do the needful
- Collection: Reads from elements and also is capable of reading from Java collections
- Socket: Reads text socket from exposed port
- Basic:
- File
- Socket
- Standard output
- Advanced:
- Elasticsearch: Used to store and index JSON documents. We do have a dedicated chapter on this in which we will discuss this technology in a bit more detail. Flink supports both Elasticsearch 1.x and Elasticsearch 2.x
- Cassandra
- HDFS (Hadoop FileSystem)
- RabbitMQ
- Apache Kafka: Gives provision to convert a Kafka topic to DataStream and also can write to a DataStream. Flink has good integration support with Kafka
- Apache NiFi
- Amazon Kinesis Streams
- Sources: Connectors that are configured as source in Flink comes in this.
- Sinks: Connectors which are configured as sink in Flink are categorized in this
- Source and Sink: Connectors that behave as both source and sink in the Flink ecosystem come into this category
- Custom Connectors: If these built-in connectors suit your chosen use case, Flink does give provision to implement your own connector. Flink has a toolbox using which you could do this with ease. Using the toolbox (specification interfaces), both custom source and sink can be implemented.