Streaming data processing is continuous and is done in real-time. You are continuously writing data to a streaming service such as Kinesis. Typically, operating on small-sized events (say a 1 KB event), writing to a stream, aggregating that data, and then persisting to Amazon S3.

Amazon Kinesis is made up of three services:

• Amazon Kinesis Data Streams: This helps you to build custom applications that process and analyze streaming data. You can build real-time applications with framework of choice – Kinesis Analytics, Spark on EMR, custom code on EC2, or custom code on Lambda. It is easy to administer, secure, and uses durable storage.
• Amazon Kinesis Data Analytics: This helps you to easily process and analyze streaming data with standard SQL. You can build powerful real-time applications such as continuous anomaly detection, continuous time-series analysis, continuous filtering, aggregation, and enrichment. It is easy to use, fully managed, and provides automatic elasticity.
• Amazon Kinesis Data Firehose: This helps you to easily load streaming data into AWS. It requires zero-administration and provides seamless elasticity. It supports direct-to-data store integration to Amazon S3, Amazon Redshift, and Amazon ElasticSearch Service. Typically, if you have many small files, then you would aggregate to larger files on S3 to optimize batch processing. It is serverless and supports continuous data transformations, compression, encryption, and you can use AWS Lambda to create a serverless ETL pipeline to get the data into AWS.