S3 as an event source

In a previous chapter, we talked about the types of invocation for Lambda functions and how other services in the AWS ecosystem can trigger a function to be executed. Now, it's time to learn a little more about how this applies to S3 and see how we can create one ourselves. 

The example in this section and the next is a trivial exercise to show that we can enable some processing to happen when an object is uploaded to a bucket in S3. Setting the scene, let's say we have users who are uploading photos into an S3 bucket. We want to classify these photos and move them into another location within the bucket. We also want to extract and store the metadata of the photo, such as the EXIF and a timestamp. Storing this data in a database is a good idea because we are planning to be able to run searches against the metadata later down the line.

Connecting an action on an S3 bucket to a Lambda function is the first task for enabling the event-driven approach that's used by serverless applications. Let's take a look:

  1. To do this, in the S3 console, we need to navigate to the Properties tab of a bucket.
  2. Under the Advanced settings, you will find where to create an event and set up a notification to occur when the event happens.

Let's see what this looks like for our example. In the following screenshot, we have set up a new event that triggers when new objects are created under the specified prefix and suffix:

Adding an event to an S3 bucket

Under the Send to dropdown, we actually have three options to choose from:

  • SNS topic
  • SQS queue
  • Lambda function

These options give us the flexibility to change our architecture, depending on what we need. For example, we may use an SNS topic so that multiple message consumers can subscribe to the new object event.

If you create the event in the console, a new permission will be added for you to allow S3 to invoke Lambda. You'll also be able to see the new connection in the Lambda console. This is easy to try yourself, so create a new bucket and set up a new Lambda function to try it out. There's a good Lambda blueprint to get started with called s3-get-object. The code in this function will actually download or get the object from the location it was uploaded to. This is useful when we want to perform further processing on that object.

When everything has been created, we get the following sequence of events. An object is created in the bucket using the putObject API action, which causes an event that triggers our Lambda function:

Event trigging sequence

That's the basics of how to integrate S3 with Lambda and enabling our function to respond to events that happen in S3. When the function is executing, we may need to interact with an S3 bucket even further. We might have to download the new object, run some processing, and then upload the resulting object to the same bucket. In the next section, we will learn how to use the AWS SDK to interface with S3 from code.

..................Content has been hidden....................

You can't read the all page of ebook, please click here login for view all page.
Reset
3.145.75.217