Chapter 3. Google App Engine

Google App Engine is a serverless compute solution that allows you to run your applications without having to manage the underlying infrastructure. App Engine supports Node.js, Java, Ruby, C#, Go, Python, PHP, or you can bring your own language runtime. App Engine has two editions: Standard and Flexible. With Flexible you can run your custom runtimes that allow you to bring any library and framework to App Engine.

App Engine provides you with enterprise-ready deployment features such as application versioning, traffic splitting, security, monitoring and debugging. With App Engine, all you need to focus on is your code; Google Cloud manages the underlying infrastructure.

All code samples for this chapter are located at https://github.com/ruiscosta/google-cloud-cookbook/04-appengine. You can follow along and copy the code for each individual recipe by going to the folder with that recipe’s number.

3.1 Deploy a Hello World to App Engine Standard

Problem

You want to deploy your first App Engine application application to perform a simple Hello World.

Solution

Using Google Cloud command line and your favorite editor to build a simple Express.js application to run on App Engine.

  1. On your local workstation create a temporary folder to hold the files you will need to create the hello world application.

  2. In your favorite IDE create an app.js file in the root of the directory you created on step 1 and copy the code below to the file.

    'use strict';
    const express = require('express');
    const app = express();
    app.get('/', (req, res) => {
    	res.status(200).send('Hello, world!').end();
    });
    const PORT = process.env.PORT || 8080;
    app.listen(PORT, () => {
    	console.log(`App listening on port ${PORT}`);
    	console.log('Press Ctrl+C to quit.');
    });
    module.exports = app;
  3. Now create an app.yaml file in the root of the same directory and copy the code below to the file. The app.yaml file defines the settings for your application including the runtime of your code.

    runtime: nodejs14
  4. Now create a package.json file in the root of the same directory and copy the code below to the file.

    {
      "name": "appengine-hello-world",
      "engines": {
        "node": ">=14.0.0"
      },
      "scripts": {
        "start": "node app.js"
      },
      "dependencies": {
        "express": "^4.17.1"
      }
    }
  5. In your terminal run the following command in the root directory you created in step 1: npm install

  6. To deploy the application to App Engine Standard run the following command: gcloud app deploy

  7. To view your deployed application, run the following command: gcloud app browse

Discussion

You have successfully deployed your first App Engine application using Node.js. It is a simple hello world application, but demonstrates some of the basic concepts in deploying App Engine services.

3.2 Deploy a Hello World to App Engine Flexible

Problem

You want to deploy an App Engine application running as a container to perform a simple Hello World.

Solution

App Engine Flexible supports running a Docker container that can include custom runtimes or other source code written in a different programming language. Since App Engine Flexible supports running Docker containers, you will use the Flexible version of App Engine to deploy a simple hello world.

  1. On your local workstation create a temporary folder to hold the files you will need to create the hello world application.

  2. In your favorite IDE create a Dockerfile in the root of the directory you created in step 1 and copy the code below to the file.

    FROM nginx
    COPY nginx.conf /etc/nginx/nginx.conf
    RUN mkdir -p /var/log/app_engine
    RUN mkdir -p /usr/share/nginx/www/_ah && 
        echo "healthy" > /usr/share/nginx/www/_ah/health
    ADD www/ /usr/share/nginx/www/
    RUN chmod -R a+r /usr/share/nginx/www
  3. Note: The FROM command builds a base image using the official nginx docker image.

  4. Create a new file named app.yaml in the root of your temporary directory and type the following code into it:

    runtime: custom
    env: flex
  5. Now create a new file named nginx.conf also in the root of your temporary directory you create and type the following code into it:

    events {
        worker_connections 768;
    }
    http {
        sendfile on;
        tcp_nopush on;
        tcp_nodelay on;
        keepalive_timeout 65;
        types_hash_max_size 2048;
        include /etc/nginx/mime.types;
        default_type application/octet-stream;
        access_log /var/log/app_engine/app.log;
        error_log /var/log/app_engine/app.log;
        gzip on;
        gzip_disable "msie6";
        server {
            listen 8080;
            root /usr/share/nginx/www;
            index index.html index.htm;
        }
    }
  6. Create a new folder called www in the root of your temporary directory

  7. Within the www folder create a new file called index.html and copy the code below to it:

    <!doctype html>
    <html>
      <head>
        <title>Hello World!</title>
      </head>
      <body>
        <h1>Welcome to nginx!</h1>
        <p>Brought to you by Google App Engine.</p>
      </body>
    </html>
  8. Run the following command to install the gcloud App Engine extension for Go: gcloud components install app-engine-go

  9. Run the following command to deploy your application to App Engine: gcloud app deploy

  10. To see your app running run the following command to launch your browser: gcloud app browse

Discussion

You have successfully deployed a static web application running on nginx web server as a custom runtime running on App Engine Flexible. App Engine Flexible is a perfect choice for applications that:

  • Applications that run in a Docker container that includes a custom runtime

  • Depends on frameworks that include native code.

  • Accesses the resources or services of your Google Cloud project that reside in the Compute Engine network

The table below summarizes at a high level the differences between App Engine Standard and Flexible:

Feature Standard environment Flexible environment
Instance startup time Seconds Minutes
SSH debugging No Yes
Scaling Manual, Basic, Automatic Manual, Automatic
Scale to zero Yes No, minimum 1 instance
Modifying the runtime No Yes (through Dockerfile)
Deployment time Seconds Minutes
WebSockets No Yes
Supports installing third-party binaries Yes Yes

3.3 Securing your application with Identity-Aware Proxy (IAP)

Problem

You’ve deployed your Hello World application running on App Engine Flexible and want only certain users to be able to access it.

Solution

You will use Google Cloud Identity-aware proxy (iap) to restrict access to only a set of predefined users. You will use the Cloud Hello World application created in recipe 1 to secure with IAP.

  1. Go to the Identity-Aware Proxy page in the Google Cloud Console.

  2. Note: If you haven’t configured your OAuth consent screen, you’ll be prompted to do so.

  3. Select the resource you want to secure by checking the box to its left.

  4. On the right side panel, click Add Member.

    Figure caption TK
  5. Add the email addresses of groups or individuals to whom you want to grant access to your App Engine application.

  6. IAP IAM supports the following accounts:

  7. Click Add when you completed adding the accounts to access your application.

  8. On the IAP page, under HTTPS Resources, find the App Engine app you want to restrict access to and toggle the on/off switch in the IAP column.

    Fig caption TK
  9. Access the URL for your App Engine application. You should be prompted to sign in, if you’re not prompted try sign in with an Incognito window.

  10. If you have not granted access to an account and they try to access your application they will receive the following message:

    Fig caption TK
  11. If you have authorized a user account and they sign in with the associated account they will have full access to your application running on App Engine.

Discussion

With Google Cloud Identity-aware proxy you can restrict access to your application running on App Engine, preventing unauthorized access to your resources. Identity-aware proxy also supports external identities such as Google, Microsoft, Email/Password and others which provide a robust set of sign-in options for your users to access your application.

Fig caption TK

3.4 Custom Domains with App Engine

Problem

You want to use your own custom domain rather than the default address that App Engine provides for you.

Solution

Google Cloud provides you the ability to map custom domains as well it issues a managed certificate for SSL for HTTPS connections. You will use the Cloud Hello World application created in recipe 1 to enable your custom domain.

Note: You will require a custom domain for this recipe.

  1. In the Google Cloud Console, go to App Engine → Settings →Custom Domains.

  2. Click Add a custom domain.

  3. If your domain name has been verified, the domain name will appear in the dropdown. Select the domain from the drop-down menu and click Continue.

  4. If you haven’t verified your domain name follow these steps to verify:

    • Select Verify a new domain from the drop-down menu.

    • Enter your domain name and click Verify.

    • Enter the required information in the Webmaster page.

  5. After you complete the steps above in the Webmaster page, you will then return to the Add a new custom domain page in the Google Cloud Console.

  6. In the Point your domain to section, specify the domain and or subdomains that you want to map. Click Save mappings when you completed adding all your mappings.

    Fig caption TK
  7. Click Continue to see your domain’s DNS records.

  8. Sign in to your domain registrar web site and update your DNS records with the records displayed.

  9. Test by browsing to your new mapped domain, for example https://www.example.com.

  10. Note: That it can take several minutes for the automatic SSL certificate to be issued.

Discussion

By mapping a custom domain you can enable your App Engine application to align with your branding as well as keep a secure site, since Google Cloud will provide an SSL certificate for your mapped domain.

3.5 Using Machine Learning APIs with App Engine

Problem

You need to build a real-time translation application for voice.

Solution

Google Cloud offers a Media Translation API that adds real-time audio translation to your applications. You will build two applications, a broadcast application and a client application in this recipe using App Engine to host your application.

Diagram

Fig caption TK

Notes

This recipe requires you to git clone the following repository: https://github.com/ruiscosta/google-cloud-cookbook/

  1. In the cloned application go to 04-appengine/4-5-media

  2. In your IDE edit the following file client/client.js and replace [PROJECT_ID] with your Google Cloud project:

    const socket = io.connect('https://[YOUR_PROJECT_ID].uc.r.appspot.com');
  3. Repeat the process in step 2 but edit the broadcast/client.js file

  4. Enable the Media Translation API in the Google Cloud Console

  5. Deploy your App Engine application by running the following command: gcloud app deploy

  6. In the app.js file in the root directory, you will notice the following Expres.js routes declared. The client path is for the users reading the translations from the persona broadcasting. The person broadcasting would visit the root for the App Engine application.

    app.use('/', express.static('broadcast'))
    app.use('/client', express.static('client'))
  7. Once your application is deployed visit the root path and open a new tab with the /client path.

  8. In the broadcast application click start translating and start speaking in English and watch the translation on the second tab. The translation is from English to Portugusse.

  9. You can change the languages in app.js file, line 29 and 30

      const sourceLanguage = 'en-US';
      const targetLanguage = 'pt-BR';
  10. Here is a sample of the Broadcast Application

    Figure caption TK
  11. Here is a sample of the Client Application

    Fig caption TK

Discussion

In this recipe you used an existing repository to deploy a translation application to App Engine. This application uses Express.js, Websockets and the Media Translation API to allow realtime audio to be translated to the language you define in the code. Since we are using WebSockets we used App Engine Flexible since Standard does not support WebSockets. WebSockets allowed the real time communication of the broadcaster and users.

3.6 Cube.js and React Dashboards with App Engine

Problem

You want a secure method to display charts/graphs to users from aggregated BigQuery data.

Solution

App Engine provides for a great platform to run any of your applications. For this recipe you will use Cube.js, BigQuery and App Engine to build a user interface for viewing charts/graphs from data stored in your BigQuery dataset.

Note:

  • You will be using Cube.js which is a Open Source analytical API platform, you can learn more about Cube.js at https://cube.dev/

  • You will also learn how to deploy React.js to App Engine since the user dashboards will be running with the React.js framework.

    Fig caption TK
  1. On your local workstation create a temporary folder to hold the files you will need to create App Engine user dashboards.

  2. In your temporary folder using the Cube.js CLI run the following command to create a new Cube.js application for BigQuery: npx cubejs-cli create real-time-dashboard -d bigquery

  3. You will need credentials to access BigQuery, in the Google Cloud Console create a new service account. Add the BigQuery Data Viewer and BigQuery Job User roles to this service account and then generate a new JSON key file. Copy the JSON key to the root of the real-time-dashboard folder.

  4. In your IDE edit the real-time-dashboard/.env file to include your Google Cloud Project as well as the location of your key file:

    	
    CUBEJS_DB_BQ_PROJECT_ID=example-google-project
    CUBEJS_DB_BQ_KEY_FILE=./examples.json
    CUBEJS_DB_TYPE=bigquery
    CUBEJS_API_SECRET=SECRET
  5. Cube.js uses a data schema to generate SQL code, you will be working with a BigQuery public dataset, create a file called Stories.js in the real-time-dashboard/schema folder with the following code:

    cube(`Stories`, {
        sql: `
          SELECT *
          FROM bigquery-public-data.hacker_news.full
          WHERE type = "story" AND STARTS_WITH(UPPER(url), "HTTP")
        `,
      
        measures: {
          count: {
            type: `count`,
          },
        },
      
        dimensions: {
          protocol: {
            sql: `UPPER(REGEXP_EXTRACT(${CUBE}.url, r"^([a-zA-Z]+):"))`,
            type: `string`,
          },
      
          time: {
            sql: `timestamp`,
            type: `time`,
          },
        },
      });
  6. Now run real-time-dashboard locally to test and validate it’s working as expected. Run the following command in the real-time-dashboard folder: npm run dev

    In your browser, go to http://localhost:4000 which should launch Cube.js playground as seen below.

    Fig caption TK
  7. To test it’s connecting to BigQuery, click on Measure and choose Stories Count, it should look like:

    Fig caption TK
  8. This service will become an API running on App Engine. The user dashboard will connect to the Cube.js API to fetch the required data to visualize to the user.

  9. To build the dashboard, click on the Dashboard App in the Cube.js playground and choose ………..

  10. Once it’s installed, locate the folder in your IDE, it will be under real-time-dashboard/dashboard-app

  11. In your IDE edit the src/pages/DashboardPage.js to include the following:

    	-   const DashboardItems = []
    +   const DashboardItems = [
      {
        id: 0,
        name: "Orders Status by Customers City",
        vizState: {
          query: {
            "measures": [
              "Stories.count"
            ],
            "timeDimensions": [],
            "order": {
              "Stories.count": "desc"
            },
            "dimensions": [
              "Stories.protocol"
            ]
          },
          chartType: "pie",
        }
      },
      {
        id: 1,
        name: "Orders Status by Customers City",
        vizState: {
          query:   {
            "measures": [
              "Stories.count"
            ],
            "timeDimensions": [
              {
                "dimension": "Stories.time",
                "granularity": "year"
              }
            ],
            "order": {},
            "dimensions": []
          },
          chartType: "line",
        }
      },
    ];
  12. In your IDE edit the src/components/ChartRenderer.js to include the following:

     const ChartRenderer = ({
    -   vizState
    +   vizState, cubejsApi
    -   const renderProps = useCubeQuery(query);
    +   const renderProps = useCubeQuery(query, { subscribe: true, cubejsApi });
  13. Create an app.yaml file in the real-time-dashboard/dashboard-app with the following code:

    runtime: nodejs14
    handlers:
    - url: /(.*..+)$
      static_files: build/1
      upload: build/(.*..+)$
    - url: /.*
      static_files: build/index.html
      upload: build/index.html
  14. This configuration let’s App Engine serve the optimized React.js build. When making changes you will always need to run npm run build before deploying your new version to App Engine.

  15. Run the following commands to deploy the Dashboard to App Engine:

    	npm run build
    	gcloud app deploy
  16. After this has been successfully deployed run the following command to view the application in your browser: gcloud app browse

  17. Copy the URL of your deployed dashboard-app and edit the real-time-dashboard/dashboard-app/App.js file to replace the const API_URL variable with yours, it should look like this:

    const API_URL = "https://ruicosta-blog.uc.r.appspot.com";
  18. Go ahead and redeploy:

    npm run build
    	gcloud app deploy
  19. You have at this point the Dashboard ready to connect to the Cube.js API which you just updated in the App.js file. Now it’s time to deploy the API to App Engine.

  20. Create a Dockerfile in the real-time-dashboard folder with the following code:

    FROM cubejs/cube:latest
    COPY . .
  21. Create an app.yaml file in the real-time-dashboard folder with the following code:

    runtime: custom
    env: flex
    service: api
  22. Since Cube.js uses Websockets and App Engine standard does not support Websockets we need to use a custom runtime so you will use Flexible for the API.

  23. Update the content of the cube.js file the following located in the root of the real-time-dashboard folder:

    module.exports = {
        processSubscriptionsInterval: 1,
        orchestratorOptions: {
          queryCacheOptions: {
            refreshKeyRenewalThreshold: 1,
          }
        },
      };
  24. Update the content of the .env file the following located in the root of the real-time-dashboard folder to include: CUBEJS_WEB_SOCKETS=true

  25. Create a new file in the root of the real-time-dashboard folder called dispatch.yaml

      - url: "*/cubejs-api*"
        service: api
  26. The dispatch.yaml allows you to override routing rules, which allows your Dashboard application to access the API via the main URL of the Dashboard as not to cause issues with CORS.

  27. You are now ready to deploy the API and have users access data via the dashboard.

  28. In the root of the real-time-dashboard run the following command to deploy the API: gcloud app deploy

  29. Once this has completed, deploy the dispatch rules by running the following: gcloud app deploy dispatch.yaml

  30. If you now access the URL of the Dashboard you should be able to see the following:

    Fig caption TK
  31. Don’t forget to secure your application by enabling IAP.

Discussion

In this recipe you deployed the Cube.js API, a user Dashboard running on the React.js Framework, toApp Engine and created routes with dispatch rules to build an interactive real-time dashboard for users. There are many moving parts in this recipe, but the key takeaways are:

  • App Engine Standard does not support Websockets, so you used App Engine Flexible because the Cube.js API relies on Web Sockets.

  • You can enable routes in App Engine, allowing you to redirect traffic to a different service based on the URL path. In this recipe you used it to avoid CORS issues.

  • App Engine is very flexible; with custom runtimes, the possibilities of running your application on App Engine are endless.

3.7 Debugging an Instance

Problem

You notice an issue with your application, and you need a way to access the logs to debug.

Solution

With App Engine Flexible you can enable debug mode and while debug is enabled you can access the VM to view the log files of your custom runtime.

  1. To enable debug mode, run the following command: gcloud app --project [PROJECT-ID]

  2. It will prompt you with the instances available to enable debugging. Choose one.

  3. Go to the Google Cloud Console > App Engine > Instances

  4. You should notice the instance you choose that Debug mode is now enabled

    Fig caption TK
  5. Click on the SSH button to connect to the instance

  6. At this point you are connected to the instance host, which has several containers running in it.

  7. Run the following command to list the containers running: sudo docker ps

  8. The output of the sudo docker ps command lists each container by row; locate the row that contains your project ID and note the NAME of this container.

    To view the logs run the following command: sudo docker logs [CONTAINER-NAME]

    Fig caption TK
  9. This allows you to view the logs from your application for debugging purposes.

  10. You can also connect to the instance by running the following: sudo docker exec -it [CONTAINER-NAME] /bin/bash

  11. When completed don’t forget to disable debugging by running the following command: gcloud app --project [PROJECT-ID] instances disable-debug

Discussion

The ability to connect to an instance and its containers allows you debug your application running on App Engine Flexible.

3.8 GitLab CI/CD and App Engine

Problem

You need a method to automate the deployment of your application to App Engine every time a change is made to the source code.

Solution

With this recipe you will use GitLab CI/CD which is a tool that allows you to apply all the continuous methods (Continuous Integration, Delivery, and Deployment) to your application.

  1. Create a new GitLab project and clone the new repository to your local machine

  2. Create a the Hello World application from recipe 1 of this chapter but do not deploy it to App Engine

  3. In the root of the directory create a GitLab CI/CD file named .gitlab-ci.yml with the following contents:

    	image: google/cloud-sdk:slim 
    deploy: 
        stage: deploy 
        environment: Production
        only:
            -  master
        script: 
            - gcloud auth activate-service-account --key-file $GOOGLE_SERVICE_ACCOUNT_FILE 
            - gcloud app deploy app.yaml --quiet --project $GOOGLE_PROJECT_ID --version 1
  4. In the Google Cloud Console go to Identity > Service Accounts

  5. Click Create Service Account

  6. Enter name and description, then click on create

  7. Select the role “Editor” and click on continue

  8. Select the Service Account you just created and on the options click Create a key in JSON format, download the key to your local workstation

  9. In the GitLab console within your project, go to the Setting > CI / CD

  10. Expand the variables section

  11. Create a new variable

  12. Change the type of variable to “File”, the key will be named GOOGLE_SERVICE_ACCOUNT_FILE and in the value the content of the file that has been previously downloaded

  13. Create another variable named GOOGLE_PROJECT_ID and the value will be the ID of the Google Cloud project

  14. It’s time to commit your code and deploy your application to App Engine, commit your changes to your GitLab repository.

  15. Head over to GitLab console and go to the GitLab CI/CD page, you will notice your pipeline running as an example:

    Fig caption TK
  16. If you click on the pipeline you will see the deployment steps, as seen below this completed successfully.

    Fig caption TK

Discussion

The continuous methodologies of software development are based on automating the tests and deployments of your source code to minimize the chance of errors. GitLab CI/CD provides a set of tools, in this recipe you used the Continuous Deployment methodology: now, instead of deploying your application manually, it to be deployed automatically to Google Cloud App Engine.

..................Content has been hidden....................

You can't read the all page of ebook, please click here login for view all page.
Reset