© Navin Sabharwal, Amit Agrawal 2021
N. Sabharwal, A. AgrawalHands-on Question Answering Systems with BERThttps://doi.org/10.1007/978-1-4842-6664-9_5

5. BERT Model Applications: Question Answering System

Navin Sabharwal1   and Amit Agrawal2
(1)
New Delhi, Delhi, India
(2)
Mathura, India
 

We are surrounded by massive amounts of information present in the form of documents, images, blogs, websites, and more. In most cases, we always look for a direct answer instead of reading the entirety of lengthy documents. Question answering systems are generally being used for this purpose. These systems scan through a corpus of documents and provide you with the relevant answer or paragraph. It is part of the computer science discipline in the field of information retrieval and NLP, which focuses on building systems that automatically extract an answer to questions posed by humans or machines in a natural language.

Two of the earliest question answering systems, BASEBALL and LUNAR, have been popular because of their core database or information system. BASEBALL was built for answers to American League baseball questions over a one-year cycle. LUNAR was built to answer questions related to geological analysis of lunar rocks based on data collected from the Apollo moon mission. Such earlier systems concentrated on closed domains where every query must be about the specific domain and the answer text must be from a restricted vocabulary only.

Some of the advanced question answering systems of the modern world are Apple Siri, Amazon Alexa, and Google Assistant. There are various popular datasets available for question answering systems that can be leveraged to check your model performance. These include the following.

  • SQuAD : The Stanford Question Answering Dataset (SQuAD) is a reading comprehension dataset that we covered in Chapter 4.

  • NewsQA : This dataset has been created to help the research community build algorithms that are capable of answering questions requiring human-level comprehension and reasoning skills. By using CNN articles from the DeepMind Q&A dataset, authors have prepared a crowd-sourced machine reading comprehension dataset of 120,000 Q&A pairs.

  • WikiQA : This publicly available dataset contains pairs of questions and answers. It has been collected and annotated for research on open-domain question answering systems. In addition, the WikiQA dataset also includes questions for which there are no correct answers, enabling researchers to work on negative cases as well to avoid selection of irrelevant answers.

Types of QA Systems

Question answering systems are broadly divided into two categories: open-domain QA (ODQA) system and closed-domain QA (CDQA) system.

  • Closed-domain : In closed-domain systems, questions belong to a particular domain. They can answer the questions from a single domain only. As an example, a question answering system for the health care domain cannot answer any IT-related questions. These systems exploit domain-specific knowledge by using a model that has been trained on a domain specific dataset. The CDQA suite can be used to build such a closed-domain QA system.

  • Open-domain : In open-domain systems, questions can be from any domain, such as health care, IT, sports, and more. These systems are designed to answer questions from any domain. These systems actually mimic human intelligence to answer questions. One example of such a system is the DeepPavlov ODQA system, an ODQA developed by MIPT that uses a large dataset of articles from Wikipedia as its source of knowledge.

These systems can be further divided into factoid and non-factoid as briefly covered in Chapter 4 and described here.

  • Factoid question : A factoid question is about providing concise facts. Answers to factoid questions are based on proven facts. As an example, a learner might be asked to look at a passage, then answer a series of factual questions based on what he or she just read. These types of questions usually start with who, what, when, or where.

    Here are some examples of factoid questions.

    • Who is the president of the United States?

    • Who is the prime minister of India?

    • Who is the CEO of Google?

    All of these questions can be answered from any document or blog if text contains relevant data which is sufficient to answer questions.

    • Non-factoid question : A non-factoid question expects detailed answers about any topic. As an example, a user can ask questions related to mathematical problems, how to run a vehicle, what does temperature mean, and so on. Non-factoid questions usually require multiple sentences as answers, and these answers come from a particular paragraph in a document. Thus, the context of a sentence plays an important role to retrieve the relevant answer.

    Here are some examples of non-factoid questions.

    • What is the process of installing Python on Windows?

    • How can I reset my Microsoft Outlook password?

    • What do you mean by temperature?

    Answers to these questions will be a document, a paragraph, or a definition from a paragraph.

Question Answering System Design Using BERT

This section details how BERT can be used for implementation of a factoid question answering system. For this book, we are using a pretrained model that has been trained on the SQuAD version 1 dataset.

As an example, consider this question, along with the passage from a Wikipedia article on the Football League.

  • Question: Where was the Football League founded?

  • Passage: In 1888, The Football League was founded in England, becoming the first of many professional football competitions. During the 20th century, several of the various kinds of football grew to become some of the most popular team sports in the world.

The answer to this question will therefore be England.

Now, we look closer at how this question and passage will be processed using BERT to find the relevant answer. This is all in the context of a question answering system, compared to the text classification approach in Chapter 4.

BERT extracts tokens from the question and passage and combines them together as an input. As mentioned earlier, it starts with a [CLS] token that indicates the start of a sentence and uses an [SEP] separator to separate the question and passage. Along with the [SEP] token, BERT also uses segment embeddings to differentiate between the question and the passage that contains an answer. BERT creates two segment embeddings, one for the question and other for the passage, to differentiate between question and passage. Then these embeddings are added to a one-hot representation of tokens to segregate between question and passage as shown in Figure 5-1.
../images/498208_1_En_5_Chapter/498208_1_En_5_Fig1_HTML.jpg
Figure 5-1

BERT input representation

Next, we pass our combined embedded representation of question and passage as input in the BERT model. The last hidden layer of BERT will then be changed and uses softmax to generate probability distributions for the start and end index over an input text sentence that defines a substring, which is an answer, as shown in Figure 5-2.
../images/498208_1_En_5_Chapter/498208_1_En_5_Fig2_HTML.jpg
Figure 5-2

BERT architecture for question answering system

To this point, we have discussed how BERT will process input questions and passages. Next, we will see an implementation of a question answering system using BERT in Python.

Follow the steps given here to install the required prerequisites for a BERT-based question answering system. Many of them are the same as those for the examples in Chapter 4, but are included for completeness to ensure you can run the examples in this chapter.
  1. 1.

    Make sure Python is installed on your system. Open a command prompt and run the following command to determine if Python is installed, as shown in Figure 5-3.

    python

    ../images/498208_1_En_5_Chapter/498208_1_En_5_Fig3_HTML.jpg
    Figure 5-3

    Python console

    This will open your Python console at the command prompt. If Python is not installed on your system, download and install it as per your operating system from https://www.python.org/downloads/.

     
  1. 2.

    Next, install Jupyter Notebook, which we will use to code. Open a command prompt and run the following command.

     
pip install notebook
  1. 3.

    Open a command prompt and run the following command to run Jupyter Notebook.

    jupyter notebook

    The notebook will open in your default browser with the host address as localhost and the port number as 8888, along with a unique token ID. Now, you can start writing code as mentioned in subsequent steps, as shown in Figure 5-4.

     
../images/498208_1_En_5_Chapter/498208_1_En_5_Fig4_HTML.jpg
Figure 5-4

Jupyter Notebook console

  1. 4.

    You can also use Google Colab Notebook for the same purpose. It provides a fast and free environment to run your Python code if your system doesn’t have sufficient resources available. You can also use the GPUs and TPUs for free but for a limited time (12 hours) in Google Colab. You just need a Google account to log in to Google Colab Notebook. For this book, we will be using Google Colab Notebook to demonstrate a question answering system using BERT. Log in to your Google account and click https://colab.research.google.com. You will see the screen shown in Figure 5-5.

     
../images/498208_1_En_5_Chapter/498208_1_En_5_Fig5_HTML.jpg
Figure 5-5

Google Colab interface to create or import a notebook

  1. 5.

    To create a new Colab notebook, click New Notebook in the bottom right corner as shown in Figure 5-5.

     
  2. 6.

    Install the transformers library from Huggingface. Run the following command in your Jupyter Notebook or Colab Notebook.

    pip install transformers

     
  3. 7.

    After successful installation of the transformers library, you should be able to see the output shown in Figure 5-6.

     
../images/498208_1_En_5_Chapter/498208_1_En_5_Fig6_HTML.jpg
Figure 5-6

Installing transformers library

Next, we will proceed to implementation of a question answering system using BERT. The included code snippets will provide a step-by-step explanation for the question answering system.
  1. 1.

    Import the BertQuestionAnswering and BertTokenizer classes of the transformers library as shown here.

     
from transformers import BertForQuestionAnswering
from transformers import BertTokenizer
import torch
  1. 2.

    Next, load the BERT question answering model fine-tuned on the SQuAD version 2 dataset. It will be a large version of BERT, with 24 layers, 340 million parameters, and an embedding size of 1,024. Along with the BERT model, we have also downloaded a trained model vocabulary set as shown here.

     
# Load pretrained model for Question Answering
bert_model = BertForQuestionAnswering.from_pretrained('bert-large-uncased-whole-word-masking-finetuned-squad')
#Load Vocabulary
bert_tokenizer = BertTokenizer.from_pretrained('bert-large-uncased-whole-word-masking-finetuned-squad')
Note

This will take a few minutes, depending on your Internet bandwidth, as the model size is approximately 1.34 GB.

  1. 3.

    Next, it requires a question and candidate paragraph context where an answer to the question would exist. You can find a candidate paragraph using any search engine or document indexer system such as Apache Solr or Watson Discovery Service (WDS). These systems will provide context paragraphs for the question asked by the user.

     
  2. 4.

    Then , the question, along with the context paragraph, will be passed to the question answering system, where first they will be tokenized based on downloaded vocabulary. As mentioned earlier, these will be concatenated together using a special character [SEP] token in between as shown here (reference text has been taken from a Wikipedia article).

    question = "Where was the Football League founded?”
    reference_text = " In 1888, The Football League was founded in England, becoming the first of many professional football competitions. During the 20th century, several of the various kinds of football grew to become some of the most popular team sports in the world."
    #Perform tokenization on input text
    input_ids = bert_tokenizer.encode(question, reference_text)
    input_tokens = bert_tokenizer.convert_ids_to_tokens(input_ids)
     
  3. 5.

    Next, we need to concatenate them using segment embedding to differentiate between the question and the context passage. Segment embedding for the question will be added to the token vector of the question and similarly for segment embedding for the context passage. This should be done before even using it as an input to the BERT model. These additions are managed internally by the transformer library, but we need to provide Boolean values (0 or 1) to differentiate for each token as shown here.

    #Find index of first occurrence of [SEP] token
    sep_location = input_ids.index(bert_tokenizer.sep_token_id)
    first_seg_len, second_seg_len = sep_location+1, len(input_ids)-(sep_location+1)
    seg_embedding = [0]*first_seg_len + [1]*second_seg_len
     
  4. 6.

    Now we can pass our example to the model.

    #Test model on our example
    model_scores=bert_model(torch.tensor([input_ids]),token_type_ids=torch.tensor([seg_embedding]))
    ans_start_loc, ans_end_loc = torch.argmax(model_scores[0]),
    torch.argmax(model_scores[1])
    result = ' '.join(input_tokens[ans_start_loc:ans_end_loc+1])
    result = result.replace(' ##','')
     
  5. 7.

    The model will provide start and end index from context passage as an answer such as start index value as 11 and end index value as 18. The final output will be extracted from context passage using these indexes.

     
Here is the complete Python code that takes the question and reference passage as an input and finds the answer to that question.
from transformers import BertForQuestionAnswering
from transformers import BertTokenizer
import torch
def get_answer_using_bert(question, reference_text):
    # Load pretrained model for Question Answering
    bert_model = BertForQuestionAnswering.from_pretrained('bert-large-uncased-whole-word-masking-finetuned-squad')
    #Load Vocabulary
    bert_tokenizer = BertTokenizer.from_pretrained('bert-large-uncased-whole-word-masking-finetuned-squad')
    #Perform tokenization on input text
    input_ids = bert_tokenizer.encode(question, reference_text)
    input_tokens = bert_tokenizer.convert_ids_to_tokens(input_ids)
#Find index of first occurrence of [SEP] token
    sep_location = input_ids.index(bert_tokenizer.sep_token_id)
    first_seg_len, second_seg_len = sep_location+1, len(input_ids)-(sep_location+1)
    seg_embedding = [0]*first_seg_len + [1]*second_seg_len
    #Test model on our example
    model_scores = bert_model(torch.tensor([input_ids]), token_type_ids=torch.tensor([seg_embedding]))
    ans_start_loc, ans_end_loc = torch.argmax(model_scores[0]), torch.argmax(model_scores[1])
    result = ' '.join(input_tokens[ans_start_loc:ans_end_loc+1])
    result = result.replace(' ##','')
    return result
if __name__ == "__main__" :
question = "Where was the Football League founded?"
reference_text = " In 1888, The Football League was founded in England, becoming the first of many professional football competitions. During the 20th century, several of the various kinds of football grew to become some of the most popular team sports in the world."
print(get_answer_using_bert(question, reference_text))
After running this code in Colab Notebook, we get following output:
england

Now, we have seen how a BERT-based question answering system can be used for research purposes. Next, consider a scenario where you need to deploy this feature to be consumed by some website or conversation system to serve the end user who is looking for an answer to his or her query. In this case, you need to release or expose features of the QA system as a REST API. Now, follow below steps to release features of QA system as REST API.

Let’s go through the steps to set up a REST API and public URL for that API (use ngrok to generate a public URL if you are inside the private network) for a question answering system on both Windows and Linux Server.

For Windows Server

Prerequisite: Python 3.6.x and Pip need to be installed on your system.

Creation of REST API

1. Install Flask-RESTful

Flask-RESTful is an extension of the micro-framework Flask for building REST APIs.

For installation, run the following command at the Windows command prompt, as shown in Figure 5-7.
pip install flask-restful
../images/498208_1_En_5_Chapter/498208_1_En_5_Fig7_HTML.jpg
Figure 5-7

Installation of Flask-RESTful

This command will install the package and all its dependencies.

2. Build the REST API

A RESTful API uses HTTP requests to GET and POST data.

First create a QuestionAnswering.py file that will have the question answering code that you have downloaded from GitHub.

3. Deploy Flask REST API

Using Flask, deploy the REST API service and run the following command at the Windows command prompt as shown in Figure 5-8.
python QuestionAnswering.py
../images/498208_1_En_5_Chapter/498208_1_En_5_Fig8_HTML.jpg
Figure 5-8

Service deployment

4. Response from REST API

Now the service has been hosted at the URL http://127.0.0.1:5000/getResponse. We want the features of the question answering system to be publicly available. Therefore, we will use ngrok to generate a public URL corresponding to the local URL that we configured earlier.

Let’s look at the steps to generate a public URL using ngrok.
  1. 1.

    To configure ngrok, download it from https://ngrok.com/download .

     
  2. 2.

    The public URL is only available when the auth token is downloaded from https://dashboard.ngrok.com after signing up at https://ngrok.com/signup.

     
  3. 3.

    The auth token must be specified to ngrok so that the client is tied to this account. ngrok saves the auth token in ~/.ngrok2/ngrok.yml so that there is no need to repeat the preceding steps.

     
  4. 4.

    Unzip the downloaded ngrok folder and run the ngrok.exe application.

     
  5. 5.

    Copy the auth token from the user account mentioned in the command and run this command on the ngrok terminal prompt, as shown in Figure 5-9.

     
../images/498208_1_En_5_Chapter/498208_1_En_5_Fig9_HTML.jpg
Figure 5-9

Token generation

"ngrok authtoken <AUTHTOKEN>"
  1. 6.

    After the previous step, authtoken gets saved to the configuration file, as shown in Figure 5-10.

     
../images/498208_1_En_5_Chapter/498208_1_En_5_Fig10_HTML.jpg
Figure 5-10

ngrok configuration

  1. 7.

    ngrok is a command-line application, so type ngrok http https://<IP>:<PORT> at this terminal prompt to expose the HTTPS URL. Here the IP and port settings correspond to the question answering API host and port on which the API is hosted, as shown in Figure 5-11.

     
../images/498208_1_En_5_Chapter/498208_1_En_5_Fig11_HTML.jpg
Figure 5-11

Generate public URL

  1. 8.

    A new terminal will open after the execution of the command that will show the public URL https://44e2f215.ngrok.io corresponding to the local server URL, as shown in Figure 5-12.

     
../images/498208_1_En_5_Chapter/498208_1_En_5_Fig12_HTML.jpg
Figure 5-12

Public URL

Now, you can use the URL highlighted in Figure 5-12. That is, <URL>/getResponse Flask is good for a development environment, but not for production. For a production environment, the API should be hosted on Apache Server. Refer to the following URL to deploy a service on Apache Server in Windows.
https://medium.com/@madumalt/flask-app-deployment-in-windows-apache-server-mod-wsgi-82e1cfeeb2ed

For Linux Server

Prerequisite: Python 3.6.x and Pip need to be installed on your system.

Creation of REST API

1. Install Flask-RESTful

To install, run the following command on Linux Shell as shown in Figure 5-13.
$ pip install flask-restful
../images/498208_1_En_5_Chapter/498208_1_En_5_Fig13_HTML.jpg
Figure 5-13

Installation of flask-restful

This will install the package and its dependencies.

2. Build the REST API

Create an QuestionAnswering.py file that will have the question answering system code that you downloaded from GitHub.

3. Deploy Flask REST API

To deploy the REST API service using Flask, run the following command on Linux Shell, as shown in Figure 5-14.
$ python QuestionAnswering.py
../images/498208_1_En_5_Chapter/498208_1_En_5_Fig14_HTML.jpg
Figure 5-14

Service deployment

4. Response from REST API

Now the service has been hosted at the URL http://127.0.0.1:5000/getResponse. Because we want features of the question answering system to be publicly available, we use ngrok to generate a public URL corresponding to the local URL that we have configured previously.

Let’s look at the steps to generate a public URL using ngrok.
  1. 1.

    To expose a local HTTPS server, download ngrok for Linux server from https://bin.equinox.io/c/4VmDzA7iaHb/ngrok-stable-linux-amd64.zip.

     
  2. 2.

    The public URL is only available when the auth token is downloaded from https://dashboard.ngrok.com after signing up at https://ngrok.com/signup.

     
  3. 3.

    The auth token must be specified to ngrok so that the client is bound to this account. ngrok saves the auth token in ~/.ngrok2/ngrok.yml so that there is no need to repeat this step.

     
  4. 4.

    To unzip the downloaded ngrok files, run the following command on the terminal as shown in Figure 5-15.

     
../images/498208_1_En_5_Chapter/498208_1_En_5_Fig15_HTML.jpg
Figure 5-15

Unzip ngrok

$ unzip /path/to/ngrok.zip
  1. 5.

    Copy the auth token from the user account and add in the command. Run this command at the ngrok terminal prompt, as shown in Figure 5-16.

    "ngrok authtoken <AUTHTOKEN>"

     
../images/498208_1_En_5_Chapter/498208_1_En_5_Fig16_HTML.jpg
Figure 5-16

ngrok configuration

  1. 6.

    After the previous step the auth token will be saved to the configuration file.

     
  2. 7.

    ngrok is a command-line application, so type ngrok http https://<IP>:<PORT> at this terminal prompt to expose the HTTPS URL. Here the IP and port settings correspond to the question answering API host and port on which the API is hosted, as shown in Figure 5-17.

     
../images/498208_1_En_5_Chapter/498208_1_En_5_Fig17_HTML.jpg
Figure 5-17

Generate public URL

  1. 8.

    After the execution of the command, the terminal will be displayed with the public URL https://44e2f215.ngrok.io corresponding to the local server URL as shown in Figure 5-18.

     
../images/498208_1_En_5_Chapter/498208_1_En_5_Fig18_HTML.jpg
Figure 5-18

Public URL

For more details, please refer to the ngrok documentation at https://ngrok.com/docs.

Now, you can use the URL as highlighted in Figure 5-18. That is, <URL>/getResponse Flask is good for a development environment but not for production. For a production environment, the API should be hosted on Apache Server. Refer to the following URL for guidance on deploying a service on Apache Server in Linux.

https://www.codementor.io/abhishake/minimal-apache-configuration-for-deploying-a-flask-app-ubuntu-18-04-phu50a7ft

Follow the steps given next to release features of a question answering system as a REST API.
  1. 1.

    Create a file named QuestionAnsweringAPI.py.

     
  2. 2.

    Copy the following code and paste it in that file, then save it.

     
from flask import Flask, request
import json
from QuestionAnsweringSystem.QuestionAnswer import get_answer_using_bert
app=Flask(__name__)
@app.route ("/questionAnswering", methods=['POST'])
def questionAnswering():
    try:
        json_data = request.get_json(force=True)
       query = json_data['query']
       context_list = json_data['context_list']
       result = []
       for val in context_list:
            context = val['context']
            context = context.replace(" "," ")
            answer_json_final = dict()
            answer = get_answer_using_bert(context, query)
            answer_json_final['answer'] = answer
            answer_json_final['id'] = val['id']
            answer_json_final['question'] = query
            result.append(answer_json_final)
           result={'results':result}
        result = json.dumps(result)
        return result
    except Exception as e:
        return {"Error": str(e)}
if __name__ == "__main__" :
    app.run(port="5000")
  1. 3.

    That code processes input passed to an API, calls the function get_answer_using_bert, and sends a response from this function as an API response.

     
  2. 4.

    Open a command prompt and run the following command.

    Python QuestionAnsweringAPI.py

    This will start a service on http://127.0.0.1:5000/ as shown in Figure 5-19.

     
../images/498208_1_En_5_Chapter/498208_1_En_5_Fig19_HTML.jpg
Figure 5-19

Service deployment.

  1. 5.

    Now to test the Rest API, we are going to use Postman. This is a REST API client that is used to test the API URL. We can test any complex HTTP/s requests and can also read their responses. First, go to https://www.postman.com/downloads/ to download the Postman tool and install it on your system.

     
  2. 6.

    After the installation, test following URL and sample request JSON that is being sent to the question answering API end and response JSON that will be received as a response from the API as shown in Figure 5-20.

     

URL: http://127.0.0.1:5000/questionAnswering

Question answering system sample input request JSON:
{
    "query": "Where was the Football league founded?",
    "context_list": [
        {
            "id": 1,
            "context": "In 1888, The Football League was founded in England, becoming the first of many professional football competitions. During the 20th century, several of the various kinds of football grew to become some of the most popular team sports in the world"
        }
    ]
}
Question answering system sample output response JSON:
{
    "results": [
        {
            "answer": "england",
            "id": 1,
            "question": "Where was the Football leagure founded?"
        }
    ]
}
../images/498208_1_En_5_Chapter/498208_1_En_5_Fig20_HTML.jpg
Figure 5-20

Calling question answering API

s

The codebase for this exercise can be downloaded from GitHub at https://github.com/bertbook/Python_code/tree/master/Chapter5/QuestionAnsweringSystem.

Open-Domain Question Answering System

An ODQA system aims to find an exact answer to any question from Wikipedia articles. Thus, for a question, this system will provide a relevant answer. The default implementation of an ODQA system processes a batch of queries as an input and returns the answer.

Model Architecture

The architecture of the DeepPavlov ODQA system consists of two components: a ranker and a reader. To find an answer to any question, the ranker first retrieves a list of relevant articles from the collection of documents, and then the reader scans them to identify an answer.

The ranker component is based on the DrQA architecture proposed by Facebook Research. Specifically, the DrQA approach uses unigram-bigram hashing and a TF-IDF algorithm to efficiently return a subset of relevant articles based on a question. The reader component is based on R-NET proposed by Microsoft Research Asia and implemented by Wenxuan Zhou. The R-NET architecture is an end-to-end neural network model that aims to answer questions based on a given document. R-NET first matches the question and the document via gated attention-based recurrent networks to obtain a question-aware document representation. Then, the self-matching attention mechanism refines the representation by matching the document against itself, which effectively encodes information from the whole document. Finally, pointer networks locate the start and end index of the answer in the article. Figure 5-21 shows the logical flow of a DeepPavlov ODQA system.
../images/498208_1_En_5_Chapter/498208_1_En_5_Fig21_HTML.jpg
Figure 5-21

The DeepPavlov-based ODQA system architecture

To use this model for an ODQA system, we have used the deeppavlov library in Python. Please note that an ODQA system uses a corpus of Wikipedia articles or documents. Follow these steps to configure and use an ODQA system.
  1. 1.

    Create a new Jupyter notebook and run the following command to install the deeppavlov library, as shown in Figure 5-22.

     
../images/498208_1_En_5_Chapter/498208_1_En_5_Fig22_HTML.jpg
Figure 5-22

Installing deeppavlov library

pip install deeppavlov
  1. 2.

    Run the following command to install all required models, vocabulary, and so on, trained on the Wikipedia corpus in the English language, as shown in Figure 5-23.

     
! python -m deeppavlov install en_odqa_infer_wiki
Note

Please use the ‘!’ symbol before the installation command as just shown if you are working with Colab Notebook.

../images/498208_1_En_5_Chapter/498208_1_En_5_Fig23_HTML.jpg
Figure 5-23

Installing required packages for deeppavlov

  1. 3.

    Perform the necessary imports required for this implementation as shown here.

     
from deeppavlov import configs
from deeppavlov.core.commands.infer import build_model
  1. 4.
    Then we will get an ODQA model using the build_model class of the deeppavlov library. It takes two arguments:

    • config file path: Define the name of the config file that contains details of the relevant NLP model to be used. For this case, we will use en_odqa_infer_wiki. This name implies the ODQA model from Wikipedia.

    • download: This will be True if the model needs to be downloaded and False otherwise.

     
odqa = build_model(configs.odqa.en_odqa_infer_wiki, download = True)
  1. 5.

    Once the ODQA model has been loaded, you can test this model by providing questions such as “Who is Virat Kohli?” as shown here.

     
questions = ["Where did guinea pigs originate?", "Who is virat kohli?"]
answers = odqa(questions)
The output of this code will be the answer to questions asked from Wikipedia documents. Here is the complete code for the ODQA system.
from deeppavlov import configs
from deeppavlov.core.commands.infer import build_model
def odqa_deeppavlov(questions):
    odqa = build_model(configs.odqa.en_odqa_infer_wiki, download = True)
    results = odqa(questions)
    return results
if __name__ == "__main__" :
       questions = ["Where did guinea pigs originate?", "Who is virat kohli?"]
answers = odqa_deeppavlov(questions)
print(answers)
Here is the ouput:
['Andes of South America',  'Indian international cricketer who currently captains the India national team']
Now, we have seen how an ODQA system can be used for research or development purposes. Next, consider a scenario where you need to deploy this feature to be consumed by some website or conversation system to serve the end user who is looking for an answer to his or her query. In this case, you need to release or expose features of the ODQA system as a REST API. Now, follow these steps to release features of the question answering system as a REST API.
  1. 1.

    Create a file named OpenDomainQuestionAnsweringAPI.

     
  2. 2.

    Copy the following code and paste it in that file, then save it.

     
from flask import Flask, request
import json
from OpenDomainQuestionAnsweringSystem.OpenDomainQA import odqa_deeppavlov
app=Flask(__name__)
@route ("/opendomainquestionAnswering", methods=['POST'])
def opendomainquestionAnswering():
    try:
        json_data = request.get_json(force=True)
        questions = json_data['questions']
        answers_list = odqa_deeppavlov(questions)
        index = 0
        result = []
        for answer in answers_list:
            qa_dict = dict()
            qa_dict['answer']=answer
            qa_dict['question']=questions[index]
            index = index+1
            result.append(qa_dict)
        results = {'results':result}
        results = json.dumps(results)
        return results
    except Exception as e:
        return {"Error": str(e)}
if __name__ == "__main__" :
    app.run(debug=True,port="5000")
  1. 3.

    This code processes input passed to an API, calls a function odqa_deeppavlov, and sends a response from this function as an API response.

     
  2. 4.

    Open a command prompt and run the following command.

    Python OpenDomainQuestionAnsweringAPI.py

    This will start a service on http://127.0.0.1:5000/ as shown in Figure 5-24.

     
../images/498208_1_En_5_Chapter/498208_1_En_5_Fig24_HTML.jpg
Figure 5-24

Service deployment

  1. 5.

    Now, to test this API, Postman can be used. Please refer to the following URL and sample request JSON that is being provided to the question answering API and response JSON that will be received as a response from API, as shown in Figure 5-25.

     

URL: http://127.0.0.1:5000/opendomainquestionAnswering

ODQA system sample input request JSON:
{
    "questions": [
       {
            "question": "Where did guinea pigs originate?"
        },
{
             "question": "Who is virat kohli?"
        }
    ]
}
ODQA system sample output response JSON:
{
    "results": [
        {
            "answer": "Andes of South America",
            "question": "Where did guinea pigs originate?"
        },
       {
            "answer": "Indian international cricketer who currently captains the India national team",
            "question": "Who is virat kohli?"
        }
    ]
}
../images/498208_1_En_5_Chapter/498208_1_En_5_Fig25_HTML.jpg
Figure 5-25

Calling the ODQA system API

The codebase for this exercise can be downloaded from GitHub at https://github.com/bertbook/Python_code/tree/master/Chapter5/OpenDomainQuestionAnsweringSystem.

DeepPavlov QA System

In the previous section , we discussed how an ODQA system that has been trained on Wikipedia documents can be used to answer factoid and non-factoid questions. Next, we look at how DeepPavlov can be used for an implementation of a contextual-based question answering system where an answer to the question exists in context. As an example, consider the following context and question from a Wikipedia article.

Context: In 1888, The Football League was founded in England, becoming the first of many professional football competitions. During the 20th century, several of the various kinds of football grew to become some of the most popular team sports in the world.

Question: In which year was the Football League founded?

Answer: 1888

Please follow these steps to implement a contextual-based question answering system.
  1. 1.

    Create a new Jupyter notebook and run the following command to install the deeppavlov library.

     
pip install deeppavlov
  1. 2.

    Run the following command to install all required models, vocabulary, and so on.

     
! python -m deeppavlov install squad_bert
Note

Please use the ‘!’ symbol before the installation command as just shown if you are working with Colab Notebook.

  1. 3.

    Import the required packages as shown here.

     
from deeppavlov import configs, build_model
  1. 4.
    Then we will get the BERT model using the build_model class of the deeppavlov library. It takes two arguments:

    • config file path: Define the name of the config file that contains details of the relevant NLP model to be used. For this case, we will use squad_bert. This configuration contains all details for the specific BERT model that has been trained over the SQuAD dataset.

    • download: This is True if the model needs to be downloaded and False otherwise.

     
odqa = build_model(configs.squad.squad_bert, download = True)
  1. 5.

    Once the BERT model has been loaded, you can test it by providing a question along with the context to extract an answer, as shown here.

     
context = " In 1888, The Football League was founded in England, becoming the first of many professional football competitions. During the 20th century, several of the various kinds of football grew to become some of the most popular team sports in the world."
question = "In which year the Football league was founded?"
answers = qa_ deeppavlov (context, question)
  1. 6.

    The output of this code snippet will be the answer extracted from the context for the question asked.

     
Here is the complete Python code that shows an implementation of a contextual CDQA system.
from deeppavlov import build_model, configs
def qa_deeppavlov(question, context):
    model = build_model(configs.squad.squad_bert, download=True)
    result = model([context], [question])
    return result [0]
if __name__=="__main__":
context = "In 1888, The Football League was founded in England, becoming the first of many professional football competitions. During the 20th century, several of the various kinds of football grew to become some of the most popular team sports in the world."
question = "In which year the Football league was founded?"
answers = qa_deeppavlov (context, question)
             print(answers)
Here is the output:
1888
Now, we have seen how a contextual-based question answering system (another variation of BERT) can be used for research or development purposes. Next, consider a scenario where you need to deploy this feature to be consumed by some website or conversation system to serve the end user who is looking for an answer to his or her query. In this case, you need to release or expose features of the ODQA system as a REST API. Follow the steps given here to release features of the question answering system as a REST API.
  1. 1.

    Create a file named DeepPavlovQASystemAPI.

     
  2. 2.

    Copy the following code and paste in that file, then save it.

     
from flask import Flask, request
from DeeppavlovQASystem.QA_Deepplavlov import qa_deeppavlov
import json
app=Flask(__name__)
@app.route ("/qaDeepPavlov", methods=['POST'])
def qaDeepPavlov():
    try:
        json_data = request.get_json(force=True)
        query = json_data['query']
        context_list = json_data['context_list']
        result = []
        for val in context_list:
            context = val['context']
            context = context.replace(" "," ")
            answer_json_final = dict()
            answer = qa_deeppavlov(context, query)
            answer_json_final['answer'] = answer
            answer_json_final['id'] = val['id']
            answer_json_final['question'] = query
            result.append(answer_json_final)
        result = json.dumps(result)
        return result
    except Exception as e:
        return {"Error": str(e)}
  1. 3.

    This code processes input passed to an API, calls a function qa_deeppavlov, and sends a response from this function as an API response.

     
  2. 4.

    Open a command prompt and run the following command.

     
Python DeepPavlovQASystemAPI.py
This will start a service on http://127.0.0.1:5000/ as shown in Figure 5-26.
../images/498208_1_En_5_Chapter/498208_1_En_5_Fig26_HTML.jpg
Figure 5-26

Service deployment

  1. 5.

    Now, to test this API, Postman can be used. Please refer to the following URL and sample request JSON that is being provided to the DeepPavlov QA API end and response JSON that will be received as a response from the API as shown in Figure 5-27.

     

URL: http://127.0.0.1:5000/qaDeepPavlov

DeepPavlov QA system sample input request JSON:
{
    "query": "In which year the Football league was founded?",
    "context_list": [
        {
            "id": 1,
            "context": "In 1888, The Football League was founded in England, becoming the first of many professional football competitions. During the 20th century, several of the various kinds of football grew to become some of the most popular team sports in the world"
        }
    ]
}
DeepPavlov QA system sample output response JSON:
{
    "results": [
        {
            "answer": "1888",
            "id": 1,
            "question": "In which year the Football league was founded?"
        }
    ]
}
../images/498208_1_En_5_Chapter/498208_1_En_5_Fig27_HTML.jpg
Figure 5-27

Calling DeepPavlov QA system API

The codebase for this exercise can be downloaded from GitHub at https://github.com/bertbook/Python_code/tree/master/Chapter5/DeeppavlovQASystem.

Conclusion

This chapter covered the question answering system, which is one of the important applications of the BERT model. We learned about types of question answering Systems like CDQA and ODQA. We built a question answering system using BERT and deployed it as an API for use by a third-party system. In the next chapter, we look at how BERT is used in other NLP tasks.

..................Content has been hidden....................

You can't read the all page of ebook, please click here login for view all page.
Reset
3.17.23.130