In this section, we will implement a delivery stream using Kinesis Firehose. The application reads data from a Kinesis Stream and stores it in a S3 bucket.
- Log in to the Management Console and go to the Kinesis Firehose console. Click on the Create delivery stream button. Specify a name (kinesisfirehosedeliverystream) for the Delivery stream name:
- Choose the source as Kinesis stream and select the Kinesis stream we created in a previous section (KinesisTestStream) from the dropdown list:
- To keep our example simple, we will not do any Lambda-based processing of the incoming data. So select the Disabled option for the Record transformation parameter. Click on the Next button:
- Create an Amazon S3 bucket called kinesistestdata from the S3 console. Select Amazon S3 as the Destination for streaming data to be stored by Kinesis Firehose:
- Select kinesistestdata from the S3 bucket dropdown list. Click on the Next button:
- Click on the Create new or Choose button:
- Select firehose_delivery-role. Click on the Allow button:
- Click on the Next button.
- Review all the details of the delivery stream and then click on the Create delivery stream button:
- You should see the success message as shown in the following screenshot:
>
The newly created Kinesis Firehose delivery stream should be listed:
- Insert a test record into the Kinesis stream KinesisTestStream with the put-record command:
Aurobindos-MacBook-Pro-2:Downloads aurobindosarkar$ aws kinesis put-record --stream-name KinesisTestStream --partition-key 123 --data "Test Event Record"
- Go to the S3 console and you should see your test record data in the destination bucket (kinesistestdata):
