Chapter 1. Cloud Functions

Google Cloud Functions is a serverless compute solution that allows you to run event-based applications. There are two distinct types of Cloud Functions: HTTP functions and background functions. You invoke HTTP functions from standard HTTP requests. You use background functions when you want to have your Cloud Function invoked indirectly in response to an event, such as a message on a Pub/Sub topic, a change in a Cloud Storage bucket, or a Firebase event.

Google Cloud Functions allow you to write single-purpose functions which can be attached to events from your services. They also abstract the compute infrastructure and allow you to focus on your code. You don’t have to worry about patching operating systems or provisioning resources. Cloud Functions scale automatically; they can scale from a single invocation to millions without intervention from the developer. In this chapter, we will present a range of recipes, from introductory recipes you can use to send emails or respond to SMS messages, to advanced recipes that show you how to integrate CI/CD into your development workflow, and integrate with Cloud Endpoints for API management.

All code samples for this chapter are located at https://github.com/ruiscosta/google-cloud-cookbook/chapter-1. You can follow along and copy the code for each individual recipe by going to the folder with that recipe’s number

1.1 Creating a public HTTP Google Cloud Function

Problem

You want to create your first HTTP cloud function to perform a simple Hello World.

Solution

Leverage Node.js to create an HTTP cloud function to use the response object to send a short Hello World response.

  1. On your local workstation create a temporary folder to hold the files you will need to create the hello world http function.

  2. In your terminal run the following command: npm init

  3. Accept the defaults when prompted to set up a new npm package

  4. In your favorite IDE create an index.js file in the root of the directory you created on step 1 and copy the code below to the file.

    const escapeHtml = require('escape-html');
    exports.helloHttp = (req, res) => {
      res.send(`Hello World!`);
    };
  5. Run the following command to install the escape-html package which will also add the package to your package.json dependencies.

    npm install escape-html --save 
  6. To deploy the Cloud Function run the following command:

    gcloud functions deploy hello-http-function --entry-point helloHttp --runtime nodejs10 --trigger-http
  7. When presented with the choice to allow unauthenticated [helloHttp]? (y/N) Select Y. Selecting Y allows users to access your function without authentication.

Discussion

You have successfully deployed your first HTTP Google Cloud Function that allows all users to access it. It is a simple hello world but provides you some basic concepts on deploying functions. Let’s further break down the deployment process and understand the arguments passed to the gcloud functions deploy command.

gcloud functions deploy NAME --entry-point NAME --runtime RUNTIME TRIGGER
NAME The registered name of the Cloud Function you are deploying. NAME can only contain letters, numbers, underscores, and hyphens. NAME can either be the name of a function in your source code, or it can be a custom string (for example, hello-http-function). If you use a custom string, you must also use the --entry-point flag to specify a function contained in your code, to tell the deploy command what function to execute.
--entry-point ENTRY-POINT If you simply want to make your registered NAME different from the name of the executed function. If you don’t use this flag, it means that NAME must specify the function in your source code to be executed during deployment. Your registered name will then have the same name as the function.
--runtime RUNTIME The name of the runtime you are using. Example:
  • nodejs10
  • python37
  • go111
  • java11
TRIGGER When deploying a new function, you must specify one of --trigger-http, --trigger-topic, or --trigger-bucket, or specify both --trigger-event AND --trigger-resource. You use the flag --trigger-http to deploy HTTP functions. You use the other flags to deploy background functions.

1.2 Creating a Secure HTTP Google Cloud Function

Problem

You want to create your first Secure HTTP cloud function to perform a simple Hello World.

Solution

Configure the cloud function to deny unauthenticated requests.

The steps for creating a secure HTTP cloud function are the same for Creating a public HTTP Google Cloud Function recipe with one difference: When you are prompted to allow unauthenticated [helloHttp]? (y/N) you would Select N. Selecting N restricts only the users you have allowed to access your function.

Discussion

At this point you have successfully created a hello world Cloud Function that is restricted to the users who you add and provided the Cloud Functions Invoker role. To test, navigate to the Cloud Function HTTP URL. To find your URL in your Cloud Console navigate to Cloud Functions, click on the function you deployed. Click on the TRIGGER tab. If you click on the link it should present the following message:

Since you did not provide credentials the function denied you access. To securely access the function, you can test with your email address that you use for Google Cloud. Run the following command

curl -X GET "[YOUR_CLOUD_FUNCTION_URL]" -H "Authorization: Bearer $(gcloud auth print-identity-token)" --header "Content-Type: application/json"

Hello World” should print on the command line. By default your user account to access Google Cloud has the Cloud Functions Invoker role.

1.3 Accessing Environment Variables at runtime

Problem

You need a way for your code at runtime to access key/value pairs you define.

Solution

Create a cloud functions with environment variables to specify arbitrary key/value pairs at the time of deployment. Which will be surfaced as key/value pairs acessible by your code at runtime.

  1. Create and deploy a new cloud function with the code below and allow unauthenticated access to your function.

    const escapeHtml = require('escape-html');
    exports.helloHttp = (req, res) => {
      res.send(process.env.MY_MESSAGE);
    };
  2. Run the following command to configure your environment variable listed as MY_MESSAGE in the command below which will hold a simple string value:

    gcloud functions deploy sendGrid --set-env-vars MY_MESSAGE=”Hello World”
  3. Test your cloud function with a simple curl request. You will see the message Hello World which from your code retrieves the environment variable MY_MESSAGE and returns the string value of the key.

Discussion

You have successfully created an environment variable that holds a value to your assigned key. Environment variables are key/value pairs deployed alongside a function. These variables are scoped to the function and are not visible to other functions in your project. You can add or remove environment variables using both the

gcloud

command-line tool and Cloud Console UI.

1.4 Sending Emails from Cloud Functions with SendGrid

Problem

You need the ability to send emails programmatically from your applications by calling a secure generic REST API.

Solution

Leverage the SendGrid SDK for Node.js to send emails from Google Cloud functions.

  1. Create an empty folder, and run npm init. Get your code ready for the SenGrid function by creating the index.js file and copy the code below. You will also need to install the @sendgrid/mail npm package in your package.json file.

    const sendgrid = require('@sendgrid/mail');
    exports.sendGrid = async (req, res) => {
        console.log('running sendGrid Function')
        try {
            if (req.method !== 'POST') {
                const error = new Error('Only POST requests are accepted');
                error.code = 405;
                throw error;
            }
            const msg = {
                to: req.body.to,
                from: req.body.from,
                subject: req.body.subject,
                text: req.body.text
            };
            sendgrid.setApiKey(process.env.SENDGRID_API_KEY);
            sendgrid.send(msg)
                .then((response) => {
                    console.log(response)
                    if (response.statusCode < 200 || response.statusCode >= 400) {
                        const error = Error(response.body);
                        error.code = response.statusCode;
                        throw error;
                    }
                    res.status(200).send(`
    
     Email Sent to ${req.body.to} 
    
    `);
                })
            return Promise.resolve();
        } catch (err) {
            console.error(err);
            const code =
                err.code || (err.response ? err.response.statusCode : 500) || 500;
            res.status(code).send(err);
            return Promise.reject(err);
        }
    };
  2. Deploy your Cloud Function and set unauthenticated to No.

  3. To configure SendGrid you will need to enable the SendGrid API in the Google Cloud Marketplace as well as retrieve your SendGrid API key

  4. Use the https://console.cloud.google.com/marketplace to sign up for the SendGrid email service.

  5. Once the SendGrid service has been enabled you can retrieve API Keys on the sendGrid website. Click on Manage API keys on SendGrid website

  6. Within the SendGrid website navigate to Settings > API Keys

  7. Click Create API Key

  8. Copy the API key created.

  9. You will then set a Cloud Function environment variable value with the created API key.

  10. Set a Cloud Function environment variable (see Recipe 1-3) which will hold your SendGrid API key, name the key SENDGRID_API_KEY and set the value to your SendGrid API key.

  11. Set the Google Cloud Application Default Credentials locally. The below command obtains the user access credentials via a web flow and puts them on your local workstation. This command is useful when you are developing code that would normally use a service account but need to run the code in a local development environment where it’s easier to provide user credentials.

    gcloud auth application-default login 
  12. To test your newly created function run the following CURL command, replace the values within brackets with your settings. Example: replace “[TO_EMAIL]” with "[email protected]“.

  13. To find your cloud function url go to Google Cloud Console > Compute > Cloud Functions, select the newly deployed function and select the trigger tab

  14. curl is a tool to transfer data from or to a server, using a supported protocol. You will be using HTTPS.

    curl -X POST "[YOUR_CLOUD_FUNCTION_URL]" -H "Authorization: Bearer $(gcloud auth print-identity-token)" --data '{"to":"[TO_EMAIL]", "from":"[FROM_EMAIL]", "subject":"[YOUR_SUBJECT]","text":"[YOUR_TEXT]"}' --header "Content-Type: application/json"
  15. You will see the following message on terminal on a successful execution

    	Email Sent to [TO_EMAIL]

Discussion

At this point you have successfully created a Cloud Function to send emails with SendGrid, and set the parameters in the curl request to whom you want to send the email to plus the email contents. To allow users you will need to give the respective user who needs to use your cloud function the Cloud Functions Invoker role, you can follow the recipe Authenticating End-users to add users.

1.5 Deploying Cloud Functions with a GitLab CI/CD Pipeline

Problem

You want an automated way of deploying cloud functions when commiting your code to a git repository.

Solution

Leverage GitLab’s CI/CD pipeline to automate your deployment.

You will need the following prior to following the instructions:

  1. You will need a GitLab Account

  2. You will need to have an empty GitLab repository created and cloned locally to your workstation

To authorize GitLab access to our Google Cloud project we will need to create a Google Cloud service account and assign the required roles to the service account. This service account will authorize GitLab to deploy the Cloud Function to the defined project.

  1. Open the IAM & Admin in the Google Cloud Console.

  2. On the menu click Service Accounts

  3. Click CREATE SERVICE ACCOUNT

  4. Enter your Service account details:

    • Service Account Name

    • Service Account ID

    • Service Account Description

    • When completed it should look like Figure 1-1

    Figure 1-1. Service Account Details Screenshot
    • Click Create.

    • Assign the following roles to the Service Account

    • Cloud Functions Developer

    • Service Account User

    • Click Continue

    • Click Done

    • Locate the newly created Service Account, click on the actions icon, and select create a key.

  5. Choose JSON and click CREATE. This will download the JSON file to your local workstation.

  6. Head over to your GitLab Project, and under settings go to CI/CD

  7. Click Expand Variables

  8. Create two new variables labeled as:

    PROJECT_ID
    SERVICE_ACCOUNT
  9. Enter your Google Cloud Project ID

  10. Open the JSON file for the service account you downloaded before, copy and paste it’s content to the SERVICE_ACCOUNT key. It should look something like Figure :

    At this point you have the authorization configured for GitLab to deploy Cloud Functions to your Google Cloud Project. The next step is to prepare your application code to be pushed to the master branch in your GitLab repository.

  11. In your empty local GitLab repository run npm init, get your code ready by creating the index.js file and copy the code below.

    const escapeHtml = require('escape-html');
    exports.helloHttp = (req, res) => {
      res.send(`Hello World!`);
    };
  12. Commit your changes and push the code to your repository

  13. If you head back to GitLab CI/CD you should see your job running

  14. You should see the following message once the Pipeline has successfully executed

  15. Open the Cloud Function in the Google Cloud Console.

  16. You should see the Cloud Function successfully deployed.

Discussion

Congratulations you have successfully configured GitLab to Continuous Deploy to Google Cloud Functions when commits are performed on the master branch. Continuous Delivery (CD) ensures the delivery of Continuous Integration (CI) validated code to your application by means of a structured deployment pipeline. In recipe number 9 you will learn how to perform Continuous Integration (CI) to validate your code prior to deployment of your code to a production environment.

1.6 Responding to SMS Messages with Twilio and Cloud Functions

Problem

You need a programmatic method to reply to SMS messages.

Solution

Create a Cloud Functions to reply to an SMS message using Twilio, a software to send and receive text messages globally.

  1. Create an empty folder, run npm init, and get your code ready for the SMS function by creating the index.js file and copy the code below. You will also need to install the “"twilio” npm package in your package.json file.

    const twilio = require('twilio');
    const MessagingResponse = twilio.twiml.MessagingResponse;
    const projectId = process.env.GCLOUD_PROJECT;
    const region = 'us-central1';
    const TWILIO_AUTH_TOKEN = process.env.TWILIO_AUTH_TOKEN
    exports.reply = (req, res) => {
      let isValid = true;
      if (process.env.NODE_ENV === 'production') {
        isValid = twilio.validateExpressRequest(req, TWILIO_AUTH_TOKEN, {
          url: `https://${region}-${projectId}.cloudfunctions.net/reply`
        });
      }
      if (!isValid) {
        res
          .type('text/plain')
          .status(403)
          .send('Twilio Request Validation Failed.')
          .end();
        return;
      }
      const response = new MessagingResponse();
      response.message('Hello from the Google Cloud Cookbook.');
      res
        .status(200)
        .type('text/xml')
        .end(response.toString());
    };
  2. Deploy your HTTP Cloud Function

  3. Create a Twilio account at the following location https://www.twilio.com/try-twilio

  4. In your Twilio console, create a phone number

  5. Once you have a phone number, click Manage Numbers and then click on your phone number

  6. Under Messaging:

  7. Set Configure with to Webhooks/TwiML.

    • Set A Message Comes In to Webhook and enter the following URL:

      https://us-central1-[YOUR_PROJECT_ID].cloudfunctions.net/reply
    • Click Save.

  8. Return to your Twilio Account Settings and take note of the Auth Token for your Live Credentials. You will need it later in this recipe

  9. Set a Cloud Function environment variable which will hold your Twilio Auth token, name the key TWILIO_AUTH_TOKEN and set the key to your Twilio Auth token.

  10. Send an SMS message to your Twilio number

  11. You should receive a response with a message that has been defined in the Cloud Function

Discussion

In this receipt you configured a Cloud Function that is triggered by a webhook request from Twilio when a SMS message is sent to your Twilio phone number. The webhook validates that the request came from Twilio and then sends a simple reply defined in the code. Twilio allows you to send and receive text messages globally and provide you with a pay-as-you-go pricing plan. Twilio also provides you with robust documentation to get you started using their service quickly.

1.7 Unit Testing with GitLab and Cloud Functions

Problem

You need a method to perform testing on your code before it’s deployed to production.

Solution

Use Mocha and the GitLab CI/CD to perform unit testing on your code prior to deployment to a production environment.

You will need the following prior to following the instructions:

  1. You will need a GitLab Account

  2. You will need to have an empty GitLab repository created and cloned locally to your workstation

  3. To authorize GitLab access to our Google Cloud project you will need to create a Google Cloud service account and assign the required roles to the service account. You can reference Receipe 1-5 on the steps to do this.

    At this point you have the authorization configured for GitLab to deploy Cloud Functions to your Google Cloud Project. The next step is to prepare your application code to be pushed to the master branch in your GitLab repository.

  4. In your empty GitLab repository local folder, create the following:

    • Create a Folder called test in the root folder of your GitLab repository local folder

    • Create index.test.js in the newly created test folder

    • In the root folder of your GitLab repository local folder create a file index.js and .gitlab-ci.yml

  5. Copy the code below to the respective files created in step 16

    test/index.test.js
    const assert = require('assert');
    const sinon = require('sinon');
    const uuid = require('uuid');
    const {helloHttp} = require('..');
    it('helloHttp: should print a name', () => {
      // Mock ExpressJS 'req' and 'res' parameters
      const name = uuid.v4();
      const req = {
        query: {},
        body: {
          name: name,
        },
      };
      const res = {send: sinon.stub()};
      // Call tested function
      helloHttp(req, res);
      // Verify behavior of tested function
      assert.ok(res.send.calledOnce);
      assert.deepStrictEqual(res.send.firstCall.args, [`Hello ${name}!`]);
    });
    .gitlab-ci.yml
    image: google/cloud-sdk:latest
    stages:
      - test
      - deploy_production
    test:
      stage: test
      script:
        - npm install
        - npm run test
    deploy_production:
      stage: deploy
      only:
        - master
      script:
        - echo $SERVICE_ACCOUNT > ${HOME}/gcloud-service-key.json
        - gcloud auth activate-service-account --key-file ${HOME}/gcloud-service-key.json
        - gcloud --quiet --project $PROJECT_ID functions deploy funcTest --runtime=nodejs10
    index.js
    const escapeHtml = require('escape-html');
    exports.helloHttp = (req, res) => {
      res.send(`Hello ${escapeHtml(req.query.name || req.body.name || 'World')}!`);
    };
  6. You will also need to install the “mocha”, “sinon” and “uuid” npm packages in your package.json file.

  7. Commit your changes and push the code to your repository

  8. If you head back to GitLab CI/CD you should see your job running

  9. You should see the following message once the Pipeline has successfully executed (Figure 1-3)

    Figure 1-3. GitLab Completed Pipeline
  10. Open the Cloud Function in the Google Cloud Console.

  11. You should see the Cloud Function successfully deployed.

You have now successfully implemented unit testing, incorporating into the GitLab pipeline and automated the deployment.

Discussion: Understanding the code

In your repository you created a file called .gitlab-ci.yml. The stages section define the stages for your GitLab pipeline, here you have two stages: a test and deploy_production. On the test: stage, the following actions will be performed at runtime. They must succeeded before the next stage runs, if one of the actions fails the pipeline will fail and your code will not be deployed:

  • apt update

  • apt install -y nodejs npm

  • npm install

  • npm run test

The npm run test is where you are telling GitLab at runtime to run the test script defined in the package.json. In the sample code the test script executes the index.test.js.

"test": "mocha test/index.test.js --exit"

If the test fails the GitLab pipeline will fail, however if all tests pass GitLab will deploy the Cloud Function.

1.8 Building an API Gateway to Gather Telemetry Data

Problem

You need a method to gather telemetry data for your API service running on Cloud Functions.

Solution

With this recipe, you will build an Extensible Service Proxy V2 Beta (>ESPv2 Beta) as an API gateway.

  1. Make a note of your project ID, the steps following ESP_PROJECT_ID as your project id

  2. Make a note of the project number, the steps following ESP_PROJECT_NUMBER as your project number

  3. The code is fairly long, please visit the following github repository to access the code samples.

  4. Deploy your Cloud Function and set function to unauthenticated

  5. Run the following command to deploy ESPv2 Beta to Cloud Run, change the CLOUD_RUN_SERVICE_NAME to name you want and se the ESP_PROJECT_ID to the one you noted above

    gcloud run deploy CLOUD_RUN_SERVICE_NAME 
    --image="gcr.io/endpoints-release/endpoints-runtime-serverless:2" 
    --allow-unauthenticated 
    --platform managed 
    --project=ESP_PROJECT_ID
  6. On successful completion, the command displays a message similar to the following:

    Service [esphello] revision [esphello-00001-zuf] has been deployed and is serving 100 percent of traffic at https://esphello-6bc24kwh7a-uc.a.run.app
  7. In this example, https://esphello-6bc24kwh7a-uc.a.run.app is the CLOUD_RUN_SERVICE_URL and esphello-6bc24kwh7a-uc.a.run.app is the CLOUD_RUN_HOSTNAME.

  8. Make a note of CLOUD_RUN_HOSTNAME. You will specify CLOUD_RUN_HOSTNAME in the host field of your OpenAPI yaml file.

  9. You can verify that the initial version of ESPv2 Beta is deployed on Cloud Run by visiting the CLOUD_RUN_SERVICE_URL in your web browser. You should see a warning message about a missing environment variable. This warning message is expected.

  10. Open openapi-functions.yaml file in your favorite IDE

  11. In the address field in the x-google-backend section, replace REGION with the Google Cloud region where your function is located, FUNCTIONS_PROJECT_ID with your Google Cloud project ID and FUNCTIONS_NAME with your function name. For example:

    Figure 1 12 Screenshot of openapi functions.yaml
    Figure 1-4. Screenshot of openapi-functions.yaml
  12. In the host field, specify CLOUD_RUN_HOSTNAME, the hostname portion of the URL that Cloud Run created when you deployed ESPv2 Beta. For example:

    Figure 1 13 Screenshot of openapi functions.yaml
    Figure 1-5. Screenshot of openapi-functions.yaml
  13. To deploy the Endpoints configuration, run the following command, replace the ESP_PROJECT_ID with yours.

  14. gcloud endpoints services deploy openapi-functions.yaml 
        --project ESP_PROJECT_ID
  15. Keep a note of the CONFIG_ID we will need this later, For example the CONFIG_ID for the example below is 2020-09-05-r3

    Screenshot of gcloud endpoints services deploy output
    Figure 1-6. Screenshot of gcloud endpoints services deploy output
  16. To enable your Endpoints service run the following command:

    gcloud services enable ENDPOINTS_SERVICE_NAME
  17. To determine the ENDPOINTS_SERVICE_NAME you can either go to the Endpoints page in the Cloud Console. The list of possible ENDPOINTS_SERVICE_NAME are shown under the Service name column.

  18. To build the service config into a new ESPv2 Beta docker image run the following command:

    chmod +x gcloud_build_image
    ./gcloud_build_image -s CLOUD_RUN_HOSTNAME 
        -c CONFIG_ID -p ESP_PROJECT_ID
  19. For CLOUD_RUN_HOSTNAME, specify the hostname of the URL that Cloud Run created when you deployed ESPv2 Beta above, enter the CONFIG_ID you noted in step 14. Example:

         chmod +x gcloud_build_image
         ./gcloud_build_image -s esphello-6bc24kwh7a-uc.a.run.app 
          -c 2020-09-05r3 -p ruicosta-blog
  20. The output should look like:

    Screenshot of gcloud_build_image bash script output
    Figure 1-7. Screenshot of gcloud_build_image bash script output
  21. Keep a note of the the full URI after “serverless:”, you will use that as your ESP_VERSION-CLOUD_RUN_HOSTNAME-CONFIG_ID in the next command

  22. Redeploy the ESPv2 Beta Cloud Run service with the new image. Replace CLOUD_RUN_SERVICE_NAME with the same Cloud Run service name you used when you originally deployed it above in Deploying ESPv2 Beta:

  23. gcloud run deploy esphello-6bc24kwh7a-uc.a.run.app 
    --image="gcr.io/ESP_PROJECT_ID/endpoints-runtime-serverless:ESP_VERSION-CLOUD_RUN_HOSTNAME-CONFIG_ID" 
    --allow-unauthenticated 
    --platform managed 
    --project=ESP_PROJECT_ID
    Example:
    gcloud run deploy esphello 
    --image="gcr.io/ruicosta-blog/endpoints-runtime-serverless:2.17.0-esphello-6bc24kwh7a-uc.a.run.app-2020-09-05r3" 
    --allow-unauthenticated 
    --platform managed 
    --project=ruicosta-blog
  24. To test your deployed Cloud Function with Cloud Endpoints run the following command:

    curl --request GET 
       --header "content-type:application/json" 
       "https://YOUR_ENDPOINT_HOST/hello"
  25. You can view telemetry data for your function, go to Endpoints > Services page in the Google Cloud Console. Below are some of the telemetry data you will collect for your API.

    Endpoint Requests
    Figure 1-8. Endpoint Requests
    Endpoint Errors
    Figure 1-9. Endpoint Errors
    Endpoint Latency
    Figure 1-10. Endpoint Latency
    Endpoint Summary
    Figure 1-11. Endpoint Summary

Discussion

With this recipe Cloud Endpoint intercepts all requests to your function and performs any necessary checks (such as authentication) before invoking the function. When the function responds, ESPv2 Beta gathers and reports telemetry.

..................Content has been hidden....................

You can't read the all page of ebook, please click here login for view all page.
Reset