Chapter 12
AWS Lambda

WHAT'S IN THIS CHAPTER

  • Introduction to AWS Lambda
  • Creating a Python Lambda function
  • Testing an AWS Lambda function
  • Viewing execution logs
  • Deleting AWS Lambda functions and Amazon CloudWatch log groups

AWS Lambda is a service that lets you run code on the Amazon cloud without provisioning servers. Amazon manages the infrastructure needed to run your code, and you are billed for the time when your code is running.

AWS Lambda code is triggered in response to events. AWS can trigger events for a variety of scenarios, such as a file being uploaded to an Amazon S3 bucket, a change in an Amazon DynamoDB table, arrival of data on an Amazon Kinesis stream, and so on. With AWS Lambda, you can provide some code that can be triggered when one of these events occurs with very low (millisecond) latency.

AWS Lambda is highly scalable and is capable of running parallel instances of your code in response to concurrent events with AWS managing the provisioning of resources in the background. You can also use Amazon API Gateway to build RESTful APIs that run AWS Lambda code in response to HTTP events. Entire application back-end systems can be built in this way, without provisioning a single server.

Common Use Cases for Lambda

AWS Lambda is extremely powerful, but it only works for you if your code is in one of the supported languages and you do not need access to the underlying hardware that is executing the code. Some of the common use cases for AWS Lambda follow:

  • Serverless back end: Using a combination of API Gateway and AWS Lambda, you can build a complete, highly scalable, OAuth2.0-compliant RESTful API to support an external application without provisioning a single server.
  • Triggers: You can use AWS Lambda to run code in response to changes in Amazon S3 buckets and Amazon DynamoDB tables. The code could perform a variety of tasks such as enforcing data integrity checks, firing emails, writing to queues, and interacting with other AWS services such as Amazon Machine Learning and AWS Rekognition, to name a few.
  • Maintenance: You can use AWS Lambda to run code in response to scheduled events. Such code could perform essential maintenance and cleanup of content in databases.
  • Streams: You can configure AWS Lambda code to run in response to new data arriving on Kinesis streams. Amazon Kinesis streams allow you to build applications that process streaming data from several sources such as social media streams, financial transactions, and IOT hardware.

Key Concepts

In this section you will learn some of the key concepts you will encounter while working with AWS Lambda.

Supported Languages

AWS Lambda supports the following languages:

  • Node.js
  • Ruby
  • Java
  • C#
  • Python
  • Go

You can author your code using a variety of IDEs, such as Eclipse and Visual Studio. Amazon provides a web-based IDE as part of the AWS Lambda console. The web-based IDE provided by Amazon has limited features when compared against Eclipse or Visual Studio, but is very useful if you want to quickly put together a function in Node.js or Python. For a complete list of IDEs and tools available to create your Lambda code, visit http://docs.aws.amazon.com/lambda/latest/dg/lambda-app.html.

The examples in this chapter are presented in Python. Visit http://docs.aws.amazon.com/lambda/latest/dg/lambda-introduction-function.html for information on the other supported languages.

Lambda Functions

To create an AWS Lambda function, you first create a deployment package that contains the code you want to execute along with any dependencies. When you use the AWS Lambda console to create your function, the deployment package is created for you. If you are using your own IDE, you will need to create the deployment package yourself and then upload this deployment package and configuration information to AWS Lambda to create the Lambda function.

The deployment package in most cases is a .zip file that is uploaded to AWS Lambda using either the command-line tools or the AWS Lambda management console. You do not need to create the deployment package manually; you can use tools such as Jenkins and Maven. For more information on creating a deployment package for one of the supported languages, visit http://docs.aws.amazon.com/lambda/latest/dg/deployment-package-v2.html.

The configuration information that accompanies the deployment package provides the following key information:

  • Compute requirements: You specify the amount of memory you want to allocate to your Lambda function. AWS Lambda allocates CPU resources in proportion to the amount of memory you have requested. The ratio of the CPU to memory allocation is the same as that of an M3 EC2 instance. If you would like to know the precise hardware that you will get for a particular instance type, visit https://aws.amazon.com/ec2/instance-types/.
  • Execution timeout: This is a number in seconds that determines the maximum amount of time the function is allowed to execute. Once this time limit is reached, the Lambda function is terminated.
  • Execution role: This is an IAM role that AWS Lambda assumes when it executes your function.
  • Handler name: This is the name of the method in your code where AWS Lambda begins execution.

Programming Model

Regardless of the language you choose to write your Lambda function, a few core concepts are common to all Lambda functions: handlers, events, context, logging, and exceptions.

HANDLERS

The handler is the entry point in your Lambda function. It is a method that AWS Lambda calls to start execution of your function. The handler method can subsequently invoke other methods in the code that make up the Lambda function.

When your handler method is invoked, AWS Lambda injects data about the event that triggered your Lambda function, as well as a context object. You can access this event data through the first parameter of the handler method.

The name of the handler method is identified in the configuration information you supply when creating the Lambda function. The syntax of the handler method for a Lambda function written in Python is as follows:

def lambda_handler(event, context):
    return a_value

A handler can return a value. If the lambda function was invoked synchronously, the caller will receive the return value serialized as a JSON object. An example of how to use the return statement is provided in the following snippet:

def lambda_handler(event, context):
 
    # return the account details that were detected.
    return {
      'bankAccountNumber': '57478289274' ,
      'accountName': 'Mr. Chris Woods'
    }

EVENTS

The event is the first parameter of the handler method. It is a standard JSON dictionary with the following general syntax:

{
    "key3": "value3",
    "key2": "value2",
    "key1": "value1"
}

Events are generated by event sources. An event source is an AWS service or custom application that publishes an event. Table 12.1 lists some of the commonly used event sources and the type of events they generate.

TABLE 12.1: Common Event Sources for AWS Lambda

SERVICE EVENTS DESCRIPTION
Amazon S3 S3 Put, S3 Delete The S3 Put and S3 Delete events are fired when a new object is created or deleted in an S3 bucket.
Amazon DynamoDB DynamoDB Update The DynamoDB Update event is fired when any kind of update is made to a DynamoDB table. To use Lambda with DynamoDB, you need to enable a DynamoDB stream for the table. DynamoDB writes an entry for each update to the stream. Lambda polls this stream and invokes your function for each entry in the stream.

You can find a complete list of AWS services that can act as event sources at http://docs.aws.amazon.com/lambda/latest/dg/invoking-lambda-function.html.

Although the body of each event is a JSON dictionary, the entries within the dictionary are specific to the event. For instance, the payload for an S3 PUT event is as follows:

{
    "Records": [
    {
    "eventVersion": "2.0",
    "eventTime": "1970-01-01T00:00:00.000Z",
    "requestParameters": {
        "sourceIPAddress": "127.0.0.1"
    },
    "s3": {
        "configurationId": "testConfigRule",
        "object": {
            "eTag": "0123456789abcdef0123456789abcdef",
            "sequencer": "0A1B2C3D4E5F678901",
            "key": "HappyFace.jpg",
            "size": 1024
        },
        "bucket": {
            "arn": bucketarn,
            "name": "sourcebucket",
            "ownerIdentity": {
            "principalId": "EXAMPLE"
            }
        },
        "s3SchemaVersion": "1.0"
    },
    "responseElements": {
        "x-amz-id-2": "EXAMPLE123/5678abcdee/mnopFGH",
        "x-amz-request-id": "EXAMPLE123456789"
    },
    "awsRegion": "us-east-1",
    "eventName": "ObjectCreated:Put",
    "userIdentity": {
        "principalId": "EXAMPLE"
    },
    "eventSource": "aws:s3"
    }]
}

The payload of a DynamoDB Update event is as follows:

{
    "Records": [
    {
    "eventID": "1",
    "eventVersion": "1.0",
    "dynamodb": {
        "Keys": {
            "Id": {
                "N": "101"
            }
        },
        "NewImage": {
            "Message": {
                "S": "New item!"
            },
            "Id": {
                "N": "101"
            }
        },
        "StreamViewType": "NEW_AND_OLD_IMAGES",
        "SequenceNumber": "111",
        "SizeBytes": 26
    },
    "awsRegion": "us-west-2",
    "eventName": "INSERT",
    "eventSourceARN": eventsourcearn,
    "eventSource": "aws:dynamodb"
    }]
}

You can find sample event data for different types of AWS events at https://docs.aws.amazon.com/lambda/latest/dg/lambda-services.html.

CONTEXT

This is the second parameter of a handler method. Using this object, your code can interact with AWS Lambda to get useful information about the execution environment. Here is some of the information you can obtain from the context object:

  • The number of seconds remaining before Lambda terminates the function.
  • The CloudWatch log group stream associated with the function.
  • The AWS request ID that was returned to the client when the function was invoked. This ID can be used for follow-up inquiries with AWS support.
  • The name of the mobile app, and the device invoking the function, if the function is invoked using the AWS Mobile SDK.

LOGGING

Any logging statements in your Lambda function are written out to CloudWatch logs. The precise statements you use in your code to generate these logs depend on the programming language you are using. If you are creating your function using Python, you can use the following statements to create log entries:

  • print()
  • logging.Logger.info ()
  • logging.Logger.error()

If you use the Logger module, your log entries will contain additional information such as time stamp and log level.

EXCEPTIONS

Your function can create an exception to notify AWS Lambda that an error had occurred while executing the function code. The manner in which exceptions are created depends on the programming language you are using.

In case of Lambda functions written in Python, you can use the raise statement to raise an exception, as illustrated in the following handler:

def lambda_handler(event, context):
       raise AnException('Something went wrong!')

When this handler is invoked, AWS Lambda will return the following error message:

{
    "errorMessage": "Something went wrong!",
    "stackTrace": [
     [
        "/var/task/some_function.py",
        3,
        "lambda_handler",
        "raise AnException('Something went wrong!')"
    ]
    ],
    "errorType": "AnException"
}

If the client that invoked the AWS Lambda function has invoked it synchronously, the client will receive the error message and AWS Lambda will write a copy of the error message to the Amazon CloudWatch log. If the client invoked the AWS Lambda function asynchronously, the error message will only be written to the Amazon CloudWatch log.

Execution Environment

When an AWS Lambda function is invoked in response to an event, AWS Lambda launches an execution environment (container) for the function based on the configuration settings provided when the function was created. The underlying operating system for a container is Amazon Linux. Each container comes preinstalled with a number of libraries and provides some disk space in the /tmp directory.

For more information on the standard libraries that come with a container, refer to https://docs.aws.amazon.com/lambda/latest/dg/lambda-runtimes.html.

Lambda needs a little time to create a new container and bootstrap it before passing control to your handler method. To increase efficiency, Lambda keeps the container around for a short while after your function has finished executing. If another copy of your function is executed within this short time, Lambda reuses the container.

You cannot assume that a container will be reused. It is entirely up to AWS Lambda to make that decision. However, if an execution container is reused, it has the following implications for your code:

  • Variable declarations in your Lambda function code, outside the handler method, remain initialized. This could be useful if your code was establishing a database connection and storing the connection in a variable. You could add logic in your code that checks whether the connection variable is already initialized and only create a new connection if it is not.
  • The contents of the /tmp directory of the container are not deleted.
  • Any background processes (or callbacks in the case of Node.js) initiated by the previous instance of your function, that did not complete when the previous instance of your function ended, are resumed. You must ensure any such processes are complete before your Lambda function exits.

Service Limitations

AWS Lambda places limits on the size of your Lambda function as well as the runtime compute resources they can utilize. The maximum number of concurrent executions is capped at 1000 and the total storage that can be used by the Lambda function and its components is capped at 75 GB. You can request an increase to these limits.

However, there are other limits for which you cannot request an increase. Some of these limits are:

  • The memory allocated to your function at runtime must lie between 128 MB and 3008 MB, allocated in 64 MB increments.
  • The maximum timeout interval for the function is 900 seconds.
  • For synchronous invocations, the size of the request and response payloads must not exceed 6 MB.
  • For asynchronous invocations, the size of the request and response payloads must not exceed 256 KB.

You can find out more about these limits, including how to request an increase to the number of concurrent executions and file size, at https://docs.aws.amazon.com/lambda/latest/dg/limits.html.

Pricing and Availability

AWS Lambda is available on a pay-per-use model. You will be charged based on the number of requests for your function and the duration for which the function code executes. Duration is rounded up to the nearest 100ms boundary. This service is included in the AWS free-tier account. You can get more details on the pricing model at https://aws.amazon.com/lambda/pricing/.

Common Tasks

In this section, you learn to create and test Python AWS Lambda functions using the AWS Lambda console and the AWS CLI tools.

To create and configure AWS Lambda functions, you should use an IAM user with suitable privileges. Log in to the AWS management console using the dedicated sign-in link for your development IAM user account. The screenshots in this section assume that the console is connected to the EU (Ireland) region. Click the Services menu and navigate to the AWS Lambda service home page (Figure 12.1).

Screenshot of AWS Lambda service home page.

FIGURE 12.1 AWS Lambda service home page

Creating a Simple Python Lambda Function Using the AWS Management Console

If you are using Lambda for the first time, you are presented with the AWS Lambda splash screen (Figure 12.2).

Screenshot of AWS Lambda splash screen.

FIGURE 12.2 AWS Lambda splash screen

Follow the instructions listed below to create a Lambda function with Python 3.6 code.

  1. Expand the menu on the left side of the page and click the Dashboard link to access the AWS Lambda dashboard. If you have used Lambda in the past, you arrive at the AWS Lambda dashboard (Figure 12.3).
    Screenshot of AWS Lambda dashboard.
    FIGURE 12.3 AWS Lambda dashboard
  2. Click the Create Function button on the AWS Lambda dashboard screen to start the process of creating a new Lambda function.
  3. If you want to view a list of existing Lambda functions, click the Functions menu item in the menu on the left side of the page (Figure 12.4). You can also create a Lambda function from the Functions page by clicking the Create Function button on the page.
    Screenshot of list of existing AWS Lambda functions.
    FIGURE 12.4 List of existing AWS Lambda functions
  • NOTE   You can use the management console to create Lambda functions using Ruby, Node.js, or Python. For Java and C#, you need to use the AWS command-line tools and dedicated IDE plug-ins to package the Lambda function.

After clicking the Create Function button, you are taken to the Create Function screen where you have three choices (Figure 12.5):

Screenshot of AWS Lambda Create Function screen.

FIGURE 12.5 AWS Lambda Create Function screen

  • Create a function from scratch
  • Select a blueprint as a starting point
  • Look up the AWS Serverless Application repository for a function created by someone else that does what you need

A blueprint is a template for building a Lambda function. AWS Lambda offers several blueprints for Lambda functions written in Node.js and Python. Blueprints make it easy to configure event sources for your Lambda function code.

  1. Since we are going to create a function from scratch. Ensure the Author From Scratch option is selected.
  2. Name the function TestFunction, use the Runtime drop-down to select the Python 3.6 runtime, and select the Create A New Role With Basic Permissions option in the Execution Role drop-down (Figure 12.6).
    Screenshot of Lambda function Name and Runtime settings.
    FIGURE 12.6 Lambda function Name and Runtime settings
  3. Click on the Create Function button to finish creating the function. While creating the new function, AWS Lambda will also create a new service role for your function that will allow access to Amazon Cloudwatch logs. The name of the role will begin with the name of the function, which in this case is TestFunction.

    The function that we are creating in this section does not need access to any additional AWS resources and therefore, we will not update the policy document associated with the role. If, however, you want your function to access other AWS resources in addition to Amazon CloudWatch, you can use the IAM management console (Figure 12.7) to modify the permissions policy document associated with the role.

    Screenshot of inspecting the permissions policy document associated with the IAM role created by AWS Lambda.

    FIGURE 12.7 Inspecting the permissions policy document associated with the IAM role created by AWS Lambda

  4. When the Lambda function has been created, you will be taken to the function configuration page (Figure 12.8) where you can configure triggers and the code for the function.
    Screenshot of lambda function configuration page.
    FIGURE 12.8 Lambda function configuration page

The Lambda function configuration page can also be accessed by selecting the Functions menu item from the menu on the left-hand side of the AWS Lambda management console page and selecting the TestFunction from the list of available Lambda functions (Figure 12.9).

Screenshot of list of AWS Lambda functions.

FIGURE 12.9 List of AWS Lambda functions

In Chapters 13 and 18 you create Lambda functions that are triggered when items are uploaded to an S3 bucket, and use AWS Comprehend and Rekognition to detect and process the content of the uploaded item. The Lambda function being built in this section is not going to be associated with AWS event sources and therefore has no triggers. This function is tested by manually triggering it with a JSON document in place of an event.

  1. Scroll down to the Function Code section of the page and replace the existing sample Lambda function code with the following (Figure 12.10):
    Screenshot of updating the code for the AWS Lambda function.
    FIGURE 12.10 Updating the code for the AWS Lambda function
    import json
    import logging
     
    def lambda_handler(event, context):
     
        logger = logging.getLogger()
        logger.setLevel(logging.INFO)
     
        logger.info('Found event{}'.format(event))
     
        accountName = event['accountName']
        accountNumber = event['accountNumber']
        sortCode = event['sortCode']
     
        logger.info('AccountName:' + accountName)
        logger.info('AccountNumber:' + accountNumber)
        logger.info('Sort Code:' + sortCode)
     
        if accountNumber == '1234' and sortCode == '5678':
            return {
            'statusCode': 200,
            'body': json.dumps('This is the correct account!')
            }
        else:
            return {
            'statusCode': 200,
            'body': json.dumps('This is not the correct account!')
        }

  2. Click the Save button to save these changes.

Testing a Lambda Function Using the AWS Management Console

You can use the management console to test the Lambda function. In the previous section, you created a Python Lambda function. In this section, you test it using a dummy event.

  1. Click the Functions menu item in the Lambda dashboard to access a list of Lambda functions (Figure 12.11).
    Screenshot of list of AWS Lambda functions.
    FIGURE 12.11 List of AWS Lambda functions
  2. Click the function called TestLambdaFunction to access the function's code and settings.
  3. Click the Test button located at the top-right corner of the page. You are presented with a dialog box that lets you provide a test event in JSON format. Name the event TestJSONEvent, replace the default JSON content of the event with the following JSON code, and click the Save and Test button (Figure 12.12):
    Screenshot of configuring a test event.
    FIGURE 12.12 Configuring a test event
    {
        "accountName": "Abhishek Mishra",
        "accountNumber": "1234",
        "sortCode": "5678"
    }

You will see your test event listed in a drop-down menu beside the Test button. If you have defined multiple test events, the drop-down menu allows you to select a test event. The Configure Test Events menu item allows you to edit existing test events as well as create additional test events (Figure 12.13).

Screenshot of configuring a test event.

FIGURE 12.13 Configuring a test event

  1. Ensure the TestJSONEvent event is selected from the available test events and click the Test button to execute the AWS Lambda function. Your function code will be executed and the results will be presented to you (Figure 12.14).
    Screenshot of AWS Lambda function execution results.

    FIGURE 12.14 AWS Lambda function execution results

    You should see that the function succeeded with the following output message:

    {
        "statusCode": 200,
        "body": ""This is the correct account!""
    }
  2. Expand the chevron beside the function execution result to get more detailed information, including the time taken to execute the function, the cost of execution, and the logs (Figure 12.15).
    Screenshot of accessing AWS Lambda function execution statistics and logs.
    FIGURE 12.15 Accessing AWS Lambda function execution statistics and logs

The console log resembles the following:

START RequestId: 92019371-ceca-429b-ad82-979254f5ff1b Version: $LATEST
[INFO]. 2019-02-09T16:55:33.241Z. 92019371-ceca-429b-ad82-
979254f5ff1b Found event{'accountName': 'Abhishek Mishra',
'accountNumber': '1234', 'sortCode': '5678'}
[INFO] 2019-02-09T16:55:33.241Z 92019371-ceca-429b-ad82-
979254f5ff1b AccountName:Abhishek Mishra
[INFO] 2019-02-09T16:55:33.241Z 92019371-ceca-429b-ad82-
979254f5ff1b AccountNumber:1234
[INFO] 2019-02-09T16:55:33.241Z 92019371-ceca-429b-ad82-
979254f5ff1b Sort Code:5678
END RequestId: 92019371-ceca-429b-ad82-979254f5ff1b
REPORT RequestId: 92019371-ceca-429b-ad82-979254f5ff1b Duration:
0.46 ms Billed Duration: 100 ms Memory Size: 128 MB Max
Memory Used: 54 MB

The logs generated by your function are enclosed between START and END elements. The console log also contains a REPORT object that has information on the execution time, memory footprint, and billed duration.

Deleting an AWS Lambda Function Using the AWS Management Console

You can use the AWS management console to delete a Lambda function. Deleting an AWS Lambda function does not automatically delete the execution role or the Amazon CloudWatch log for the function.

  1. To delete an AWS Lambda function, click the Functions menu item in the AWS Lambda dashboard to access a list of functions. Select the function you want to delete and select the Actions ➢ Delete function menu item (Figure 12.16).
    Screenshot of accessing the Delete function menu item.

    FIGURE 12.16 Accessing the Delete function menu item

    You do not need to delete the execution role if you plan to use it for another function. You can delete the Amazon CloudWatch log for the function by accessing the Amazon CloudWatch service from the Services drop-down (Figure 12.17).

    Screenshot of accessing the Amazon CloudWatch dashboard.

    FIGURE 12.17 Accessing the Amazon CloudWatch dashboard

  2. Once you are in the CloudWatch dashboard, click the Logs menu item to access a list of logs (Figure 12.18).
    Screenshot of list of Amazon CloudWatch log groups.
    FIGURE 12.18 List of Amazon CloudWatch log groups
  3. Select the log corresponding to your Lambda function, and select the Actions ➢ Delete Log Group menu item (Figure 12.19).
    Screenshot of accessing the Delete Log Group menu item.
    FIGURE 12.19 Accessing the Delete Log Group menu item

Summary

  • AWS Lambda is a service that lets you run code on the Amazon cloud without provisioning servers.
  • AWS Lambda code is triggered in response to events.
  • AWS Lambda is highly scalable and is capable of running millions of parallel instances of your code in response to concurrent events.
  • You can also use Amazon API Gateway to build RESTful APIs that run AWS Lambda code in response to HTTP events.
  • You can use AWS Lambda to run code in response to changes in Amazon S3 buckets, Amazon DynamoDB tables, and other AWS resources.
  • AWS Lambda supports Node.js, Java, C#, Ruby, Go, and Python.
  • A deployment package is usually a .zip file that contains your function code along with any dependencies.
  • A deployment package can be uploaded using either the command-line tools or the AWS Lambda management console.
  • You specify the amount of memory you want to allocate to your Lambda function. AWS Lambda allocates CPU resources in proportion to the amount of memory you have requested.
  • AWS Lambda functions must execute within a fixed amount of time, capped at 900 seconds. Once this time limit is reached, the Lambda function is terminated.
  • You need to set up an IAM role that will be assumed by AWS Lambda when it executes your function. This IAM role should contain relevant policies that will allow your code to access other AWS resources such as Amazon S3 buckets.
..................Content has been hidden....................

You can't read the all page of ebook, please click here login for view all page.
Reset
3.15.220.219