In the previous chapter, we learned how we can use SQS for messaging. SQS is good for standard data transfer (messaging) within serverless microservice applications; however, applications that work on big data and data analytics demand more. KDS is a highly scalable data streaming service that is used for such use cases.

KDS consists of an ordered sequence of data records. A stream is composed of multiple shards with different unique sequences of data records. A partition key is used to group data into shards. In the following recipe, we will create a simple KDS, put data into the stream, and retrieve data from the stream, all using AWS CLI.