Chapter 25. AWS Lambda
AWS Lambda is a new software execution environment created by AWS designed to provide event-driven compute capabilities without the need to purchase, set up, configure, or maintain servers. Lambda gives you virtually unlimited scalability with the ability to pay at a subsecond-metered level.
Lambda is a great technology for background processing of event-triggered actions. Here are some typical use cases:
-
Image transformation for newly uploaded images
-
Real-time metric data processing
-
Streaming data validation, filtering, and transformation
It is best suited for any sort of processing where:
-
Operations need to be performed as the result of an event occurring in your application or environment
-
A data stream needs filtering or transformation
-
Edge validation or regulation of inbound data is necessary
But the real power of Lambda is that it completely removes the concerns around scalability. Lambda can scale to almost any rational scaling size necessary, without any actions required to make that happen.
Using Lambda
At this point, you might be thinking “Sounds great, but what are some architectures that can utilize Lambda?” The following sections describe some of these.
Event Processing
Consider a picture management application. Users can upload pictures to the cloud, which are then stored in S3. The application displays thumbnail versions of those pictures and lets users update attributes associated with those pictures, such as name, location, names of people in the picture, and so on.
This simple application can utilize AWS Lambda to process images after they are uploaded to S3. When a new picture is uploaded, a Lambda function can be automatically triggered that takes the picture and creates a thumbnail version of that picture and stores the thumbnail version in S3. Additionally, a different Lambda function can take various characteristics about the picture (such as size, resolution, etc.) and store that metadata in a database. The picture management application can then provide capabilities for manipulating the metadata in the database.
This architecture is shown in Figure 25-1.
Figure 25-1. File upload lambda usage
The picture management application does not need to be involved in the file upload process at all. It can rely on standard S3 upload capabilities and the two Lambda functions to do all processing necessary to complete the file upload process. So, the picture management application only has to deal with what it is good at: manipulating metadata in the database for existing pictures.
Mobile Backend
Consider a mobile game that stores user progress, trophies, and high scores in the cloud, making that data available for a shared community as well as device portability for individual users.
This application involves a series of APIs on the backend that are created so that the mobile application can store data in the cloud, retrieve user information from the cloud, and then perform community interactions.
The necessary APIs are created by using an API Gateway1 that connects with a series of Lambda functions. The scripts perform the operations necessary, in conjunction with some form of database, to handle the cloud backend for the mobile game.
This architecture is shown in Figure 25-2.
Figure 25-2. Mobile backend lambda usage
In this model, no servers are needed on the backend, and all scaling is handled automatically.
Internet of Things Data Intake
Consider an application that takes data from a huge quantity of data sensors deployed around the world. Data from these sensors arrives regularly. On the server side, this results in an enormous quantity of data being regularly presented to the application for storage in some form of data store. The data is used by some backend application, which will be ignored for this example. The data intake needs to validate the data, perhaps perform some limited processing, and store the resulting data into the data store.
This is a simple application that only performs basic data validation and verification and stores the data in a backend data store for future processing. However, although the application is simple, it must run at a massive scale, in the order of millions or billions of data intake events per minute. The exact scale is dependent on the number of sensors.
This architecture makes use of a data intake pipeline2 that sends data to an AWS Lambda function that performs any necessary filtering or processing of the data before it’s stored in the data store.
This architecture is shown in Figure 25-3.
Figure 25-3. Internet of Things sensor intake example
Lambda is well suited to handling the huge volume of data that must be ingested at a high speed regularly.
Advantages and Disadvantages of Lambda
AWS Lambda has one primary advantage: scale. Lambda is very good at handling massive scale loads without the need to increase the amount of infrastructure allocated to your application.
It accomplishes this by requiring the code that is run to be extremely simple in nature, allowing it to be easily spun up on multiple servers in multiple stacks quickly and effectively, on an as-needed basis.
This is Lambda’s sweet spot: small code footprint executed at mammoth scale.
So, where should you not use Lambda? To answer that, let’s look at the disadvantages of Lambda:
- Implicit coding requirements (simple, event driven, fast processing)
- Complex configuration and setup
- No native built-in staging or testing environments
- No native deploy/rollback capabilities
- No native A/B capability testing
- No development environment for building and testing Lambda functions
In short, Lambda is great for scaling small scripts, but it is poor at all of the other things necessary for a large-scale application deployment.
Used effectively, AWS Lambda is a technology that will significantly help in your extreme scaling needs. However, be careful to limit its use only to those tasks for which it is well suited. For compute execution needs outside of the AWS Lambda sweet spot, use other deployment/execution options.