How Does an Application Communicate?

A web application connects to a web server, which is nothing but a remote computer unit (like a CPU). Figure 10-1 explains how the web technology evolved from just static HTML to advance applications such as Gmail, Facebook, etc. One important thing that gets missed in discussion is the evolution of database technologies. Although traditional applications were built on a SQL DB, now more advanced DB technologies are available such as MongoDB, Cassandra, Neo4J, etc.

Generally these websites were supported by local servers maintained by a company’s IT but as the applications became complex and extremely connected (with data, people, and other applications), it was difficult to scale the servers proportionately. It didn’t make business sense and the resources weren’t available to maintain such a highly performant system.

Cloud Technology

And then came cloud technology. For the uninitiated, the cloud is an on-demand computer system available to many users over the Internet. This on-demand system helps us get desired storage and processing power through virtualization (i.e. dividing (through resource-locking via software) servers into smaller virtual machines).

With the cloud making enterprise-scale technology available at a really low cost, many services started to pop up. A view of such technologies is shown in Figure 10-2.

On-site is very rare these days. It might be used for some internal software/websites that can only be accessed via a company’s intranet.

In IaaS, only the infrastructure is rented (i.e. a machine with certain storage, RAM, and compute resources is commissioned to you). Imagine buying a CPU. Now you can do anything: install software, make applications, or even start a website. Yes, you can host a website with your computer but can you guarantee uptime and speed?

While using PaaS you are only concerned with developing your code and data scripts. You are not concerned with how many VMs you need to run your code efficiently and also provision the OS, library versions, and so on for each VM separately.

SaaS are generally web-based tools like Google Colab, Facebook, LinkedIn, etc. They are so called because you don’t need to set up anything in order to use them. All you need is an Internet connection that communicates to the cloud.

Why Docker?

Modern web apps contain a lot of dependencies. Some of them are OS dependent. Some of them are dependent on versions of different libraries that are used. This situation is only expected to grow as more and more libraries are developed independently. You could be using one library from one developer and another one from another, and thus is the case in ML.

This can be very troublesome if you have to integrate code tested on multiple machines (in development) and then integrate it finally to a staging server. To manage issues like this, a new development paradigm is emerging called containerized applications . It basically keeps the code, libraries used to run it, and OS-level information as a separate, isolated unit. This isolated unit can be run on another machine as is without worrying about configuring the machine for the application code to run. Docker is the most widely used container technology today and is very popular in the ML community.

OS Virtualization

Docker containers run on top of a host operating system and provide a standardized environment for code running within the container. Docker is suitable when the development environment’s operating system and testing operating system are the same. These containerized units basically solve the DevOps issue in ML because you now, along with code, get all of the dependent libraries with the right version and even the OS (i.e. an exact replica of the developer's environment).

This OS virtualization using Docker allows you to achieve efficient use of resources as compared to hardware virtualization done using VM creation with applications such as Hypervisor because you can now dynamically allocate resources between Docker containers, although they all use the same server compared to VMs, which block resources to their respective unit.

Kubernetes

Now, imagine a full-fledged app like Amazon that uses multiple such images of a container. One is allowing search results to come up, one is recommending new items, and one is capturing user behavior and interaction touchpoints with the web app. Can we scale all of them dependent on their usage? Yes. For orchestrating independent Docker containers, we use Kubernetes.

Covering Kubernetes or Docker in more detail than I have is out of the scope for the book but there are some excellent resources online such as articles on https://mlinproduction.com/.

Building a Flask Structure

Flask is a microservices web-based framework that allows you to expose any business logic/functions via an API. Although I am not going to cover a lot of Flask, for those of you using Flask for the first time, here are some basics.

Start by creating a folder named covidquest. You will use this as the folder for your application.

Install Flask so you can download the latest Flask via a pip channel.

After setting it up, let’s create the Flask app.

There are two essential things that are required to make your Flask app, one that handles the client side (front end) and another that handles the server side (back end).

You can clone the app files from https://github.com/NeverInAsh/covidquest. This will serve as your starting point, but let’s quickly see the basics of what’s in each of your files.

Deep Dive into app.py

from flask import Flask, render_template, request

import pandas as pd

import numpy as np

import sys

    app = Flask(__name__, template_folder='./templates/')

@app.before_first_request

    def at_startup():

    global answer_df, question_map, top_k_map

        answer_df = pd.read_csv("./all_question_comprehension.csv", index_col=None)

        question_map = {'1': 'Is the virus transmitted by aerosol, droplets,  food, close contact, fecal matter, or water?',

    ... skipped lines

                        '30': 'Can 2019-nCoV infect patients a second time?'}

        top_k_map = {'0': 5, '1': 10, '2': 20, '3': 30, '4': 50}

@app.route('/')

    def home():

        return render_template("index.html")

    def create_answer(text, start, end):

        output = [text[0:start],

              text[start:end],

              text[end:len(text)]]

    return output

@app.route('/top_k_results', methods=['GET', 'POST'])

    def top_k_results():

        question_select = "0"

        weight = "0.2"

        top_k = "0"

        if request.method == "POST":

            question_select = request.form.get('question_select', '')

            weight = request.form.get('weight', '')

            top_k = request.form.get('top_k', '')

    query = question_map[question_select]

    # Filtering answer dataframe for the query

        _df = answer_df[answer_df['query'].isin([query])]

        _df = _df.drop_duplicates(subset=['passage_id']).reset_index(drop=True)

        _df["final_score"] = np.float(

            weight)*_df["score"] + (1-np.float(weight))*_df["pass_rank_score"]

    _df = _df.sort_values(

            'final_score', ascending=False).reset_index(drop=True)

    # results-dictionary

        results = [{'passage': create_answer(row['passage'], row['start'], row['end']),

                    'title':row['title'],

                    'task':row['task']} for i, row in _df.head(top_k_map[top_k]).iterrows()]

        return render_template("index.html", question_select=question_select,

                           weight=weight, top_k=top_k, results=results)

    if __name__ == '__main__':

        port = int(os.environ.get("PORT", 5000))

        app.run('0.0.0.0', port)

Your app.py is organized in the following way:

1. 1)

   You start by importing all the relevant libraries used to write your backend logic.

    
2. 2)

   You then create an app object, which is an instance of the Flask object. Using it, you can configure your entire application. For example, you make sure that Flask knows which web page to render by explicitly giving the link to the templates folder. The templates folder is used to store all of the HTML files of the app, whereas all of the CSS and .js files (other technologies used for front-end/client side) are stored in static folders.

    
3. 3)

   The app object also helps set up routes for the endpoints/functions, which in turn invoke a URL. (A URL is the addresses of an endpoint.) This is done using the decorator @app.route(<url>, methods), which is an HTTP method for communication.

    
4. 4)

   The most common data communication/transfer methods are GET and POST. Whereas GET sends unencrypted information to the server, POST masks this information and passes the data in the request's body.

    
5. 5)

   You use the home endpoint as the landing page for your website. It simply renders the index file.

    
6. 6)

   You also use decorators like @app.before_first_request, which makes sure that all of the required files/variables needed to generate a response to a request are loaded before the server is ready for communication.

    
7. 7)

   app.route() is used to map the specific URL with the function. For example, you are mapping the landing page/home page of the website with the URL “/”. Similarly, you’re mapping “/top_k_results” with the function top_k_results.

    
8. 8)

   render_tempalte() is used to render HTML that is the skeleton of the UI for the client to interact with. Flask uses the Jinja template library to render templates. Read more about it at https://jinja.palletsprojects.com/en/2.11.x/.

    
9. 9)
   The main code logic is stored in the top_k_results() endpoint, which collects data from the website form (Figure 10-3). This data is

   1. a)

      Query

       
   2. b)

      Weight of comprehension score in the final score, which is a linearly weighted sum of the comprehension score and the med-marco rank score

       
   3. c)

      Top k results to show for the question asked

       
    
10. 10)

    The above data is returned via the POST method in the request body and is all string, so you convert it into the write datatype and also get an actual value and not an HTML element’s value.

     
11. 11)

    You return with a render_template() function to render the HTML or URL associated with the endpoint. Note that you pass a number of variables along with render_template(). This helps you embed logic into the markup using backend data. This is done using a Jinja template (discussed more below).

     
12. 12)

    Finally, you run the Flask app by calling it with the address and the port number for the server to listen to for the requests.

     

Understanding index.html

Your index file looks something like

    <form action="{{url_for('top_k_results',_anchor='resultsView')}}" method="post">

            <div class="container my-4">

              <p class="font-weight-bold">Questions</p>

              <select class="mdb-select md-form" id="question-select" name="question_select">

                <option value="" disabled selected>Choose your question</option>

                <option value='1' {% if question_select=='1' %} selected {% endif %}>Is the virus transmitted by aerosol,

                  droplets, food, close contact, fecal matter, or water?</option>

                <option value='2' {% if question_select=='2' %} selected {% endif %}>How long is the incubation period for

                  the virus?</option>

    ....

      <button type="submit" class="btn btn-primary btn-block btn-large">Get Top Results</button>

          </form>

        </div>

You use a form to get post requests from the front end. The weird template {{}} you see is called a Jinja template. It helps create HTML, XML, and other markup formats, which are returned to the user via an HTTP response.

You can use any variable passed as a response from the endpoint you interacted with. It is very helpful. In your use case, you don’t have any pre-hand knowledge of how many responses a user would like to see for the question asked, so this is something that can’t be static.

See how easy it is easy to replicate a template for the number of results you want?

    <ul class="timeline">

              {% for result in results %}

              <li class="timeline-item bg-white rounded ml-3 p-4 shadow">

                <div class="timeline-arrow"></div>

                <h2 class="h5 mb-0">{{result.title}}</h2><span class="small text-gray"><i class="fa fa-clock-o mr-1"></i>{{result.task}}</span>

                <p class="text-small mt-2 font-weight-light">{{result.passage[0]}}<strong><span

                      style="color:orange">{{result.passage[1]}}</span></strong>{{result.passage[2]}}</p>

              </li>

              {% endfor %}

            </ul>

By now you should have a good idea of your Flask app and how it is structured. Before I close this section, I would like you to see the directory tree of your Flask app.

|   all_question_comprehension.csv

|   app.py

|

+---static

|   |   favicon-32x32.png

|   |

|   +---css

|   |       bootstrap.min.css

|   |       choices.min.css

|   |       font-awesome.min.css

|   |       index.css

|   |       jquery.mCustomScrollbar.min.css

|   |

|   ---js

|           bootstrap.bundle.min.js

|           choices.min.js

|           index.js

|           jquery-3.3.1.slim.min.js

|           jquery.mCustomScrollbar.concat.min.js

|

+---templates

|       index.html

|

To run the Flask app, go to the project folder directory using the command line tool of your OS and type flask run, as shown in Figure 10-4.

Dockerizing Your Application

So far, you have built your application. Now it can be deployed on the server. Although for your use case it is not absolutely necessary to dockerize your application because you’re not using very many libraries and packages, this is something that can change with time and hence can cut short the lifetime for your application.

Also, you are coding on Windows but most deployment servers are Unix-based kernels. It is very likely that when you make this app live, there will be package issues and also hardware resource usage issues if the code leverages a GPU.

So to create an isolated and portable machine that can stay true to your present configuration, you will need Docker to sail smoothly through the journey of taking your app from your laptop to the production environment.

Creating a Docker Image

In order to create a Docker image, which is a single file containing all the config and dependency information required to run the app, you must create a Dockerfile. It contains all the startup commands that are executed once the container is spun off. Containers are running instances of an image. For example, a house’s blueprint is the image and the actual house is the container. In the same way that you can use a blueprint to create many houses, a Docker image can be used to create many instances that are run in separate containers.

The following commands are used to create a Dockerfile:

• FROM

• COPY

• WORKDIR

• EXPOSE

• RUN

• CMD or ENTRYPOINT

Base Image and FROM Command

Every Docker container is an image with a read/write layer on top of a bunch of read-only layers. What this means is you start with an OS distribution, say Linux Ubuntu, which is your read-only layer, and then keep on adding different layers like Anaconda to set up your Python environment and libraries like Flask, pandas, and NumPy to run your application. See Figure 10-5.

You use the FROM command to get the base image. This is a necessary command to build a Dockerfile. For your application, you are going to use the continuum Anaconda distribution. This image can be found on the Docker hub, which is a collection of container applications: https://hub.docker.com/r/continuumio/anaconda3.

COPY and EXPOSE

Using COPY command you essentially pass your files and folders to the Docker image. This, in your case, is the covidquest folder that contains your Flask app. Once copied, you will be able to fire the app from inside the Docker image.

The EXPOSE command tells the Docker OS’s network to open some ports for the server to listen for requests.

WORKDIR, RUN, and CMD

WORKDIR helps you set up the work directory, which in your case is where the app.py file resides. This is typically the directory you copied your files into using the COPY command.

The RUN command helps you to install a set of dependencies and libraries to run the app inside the container. Instead of installing each dependency separately, you make use of a requirement.txt file that contains all of the required files with particular versions.

This can also be used to run not just library installations but any other command line command. Obviously, it varies with the base image you choose.

The last command in the Dockerfile is CMD, which is the startup command for the container. It’s just like when you ran flask run on your local.

Dockerfile

Now that you are armed with this knowledge, you can finally use these commands to build your Docker image.

You start by copying your covidquest folder and renaming it to covidquest_docker. Inside this folder you create your Dockerfile. It will be an extension-less file. Your directory will now look something like this:

|   Dockerfile

|

---covidquest

    |   all_question_comprehension.csv

    |   app.py

    |   requirements.txt

    |

    +---static

    |   |   favicon-32x32.png

    |   |

    |   +---css

    |   |       bootstrap.min.css

    |   |       choices.min.css

    |   |       font-awesome.min.css

    |   |       index.css

    |   |       jquery.mCustomScrollbar.min.css

    |   |

    |   ---js

    |           bootstrap.bundle.min.js

    |           choices.min.js

    |           index.js

    |           jquery-3.3.1.slim.min.js

    |           jquery.mCustomScrollbar.concat.min.js

    |

    ---templates

            index.html

Add the following commands to your Dockerfile. You can use any text editor, but make sure that there is no extension to the Dockerfile.

FROM continuumio/anaconda3

    MAINTAINER Anshik, https://www.linkedin.com/in/anshik-8b159173/

RUN mkdir /app

COPY ./covidquest /app

WORKDIR /app

EXPOSE 5000

RUN pip install -r requirements.txt

CMD flask run --host 0.0.0.0

One more thing to note is that requirements.txt is kept inside the app folder because multiple containers that are spun off using this image will know exactly what libraries are used to build this app logic.

Building Docker Image

Finally, you use the following command to build your Docker image (see Figure 10-6):

docker build -t <docker_image_name> .

Note

The -t flag is used to give a name to the newly created image.

The process can take some time depending upon your network speed. Figure 10-7 shows whether the image has been created.

After the image gets created, you can run the container using the following command. The command below the -p flag is used to publish a container’s port to the host. Here, you’re mapping port 5000 inside your Docker container to port 5000 on your host machine so that you can access the app at localhost:5000. See Figure 10-8.

Even after you press Ctrl + C or CMD + C, the container will still run in the background.

Please note that each Docker container is associated with an ID. You can find out how many containers are running by using the command docker container ls, as shown in Figure 10-9.

Make sure to kill the container after use (Figure 10-10). If you don’t, it can throw an error like this:

    (tfdeploy) C:UsersansaDesktopBookChapter 10covidquest_docker>docker run -p 5000:5000 -d covidquest

    4778247c6c95a5a5093edd1279b03a1e41e243afb6ab84788752c9629fbaf69b

docker: Error response from daemon: driver failed programming external connectivity on endpoint funny_jemison (dc7d4acc7671b41c701558a8c4200406ec9f0474e360e8aea38b075cc1c2d5d0): Bind     for 0.0.0.0:5000 failed: port is already allocated.

When building Docker image covidquest and running the Docker container, you generate a lot of garbage, such as

• Stopped containers

• Networks not used by at least one container

• Images (see Figure 10-7)

• Build cache

You can delete all of these unwanted files and reclaim space by running the command

docker system prune

Making It Live Using Heroku

Now that you have dockerized your application, you can take it anywhere you want and deploy it to an actual address. But before you do that, let’s understand a bit about development servers.

What you have been using until now was Flask’s very own development server. This server is very limited in a sense that it can’t handle multiple users or multiple requests well.

When running a web app in production, you want it to be able to handle multiple users and many requests such that there are no noticeable amounts of time for the pages and static files to load.

To make the server more “production-ready,” you can use Gunicorn. Gunicorn is a pure-Python HTTP server for WSGI (Web Service Gateway Interface) applications. It allows you to run any Python application concurrently by running multiple Python processes over the machine commissioned by Heroku (also called dynos).

You also add a Procfile. Procfile is a format for declaring the process types that describe how your app will run. A process type declares its name and a command-line command. This is a prototype that can be instantiated into one or more running processes such as your Docker container.

You are finally ready to deep-dive into Heroku. Heroku is a PaaS system that helps build data-driven apps with fully managed data services. To learn more about Heroku, see the video “Heroku Explained: Icebergs, Lumberjacks, and Condos.”

You will do so by using the Heroku CLI. The Heroku command line interface (CLI) makes it easy to create and manage your Heroku apps directly from the terminal. It’s an essential part of using Heroku. You can follow the CLI installation from https://devcenter.heroku.com/articles/heroku-cli.

To check whether you have successfully set up Heroku or not, run the command shown in Figure 10-11.

Next, you must log in to Heroku. You can do it from the command line by typing the command heroku login, which redirects you to the browser for login. After successfully logging in (Figure 10-12), close the tab and return to the CLI.

You can now create your Heroku app by using the command heroku create <app-name>. This prepares Heroku to receive your source code. Heroku doesn’t allow you to take names that are already taken. But before that, make sure you move to the app directory (Figure 10-13).

But before you push the app to the Heroku container registry, you need to tell the Heroku CLI which app you want to run this command for. For this, you convert your folder to a Git repository using git init. If it is already a Git repo, then you don’t need to worry.

To build an image and push it to container registry, make sure that your directory contains a Dockerfile and run the command heroku container:push web. See Figure 10-14.

Finally, you can open your app using the following command. This will open the app in the browser (Figure 10-15).

    heroku open

Since you are using the free tier, the app will go down after 30 mins of idle time. To keep your app up forever, you can explore paid apps.