For processing streaming data in real time, we may have to use a service designed specifically for such use cases.
Approach: We can use Kinesis firehose as follows:
- Producers produce data in Kinesis firehose (fan-in)
- Kinesis firehose stores the data in a supported data store such as S3, Redshift, ES, or Splunk