For processing streaming data in real time, we may have to use a service designed specifically for such use cases.
Approach: We can use Kinesis streams for the stream, with Kinesis Analytics for routing, as follows:
- Producers produce data in Kinesis Stream, configured for high throughput (fan-in)
- Kinesis Analytics routes it into an appropriate stream with a lower throughput (fan-out)