Simple and fast Question Answering system using HuggingFace DistilBERT — single & batch inf...

Simple and fast Question Answering system using HuggingFace DistilBERT — single & batch inference examples provided.

Question Answering systems have many use cases like automatically responding to a customer’s query by reading through the company’s documents and finding a perfect answer.

In this blog post, we will see how we can implement a state-of-the-art, super-fast, and lightweight question answering system using DistilBERT from Huggingface transformers library.

Input

The input will be a short paragraph what we call context, and a question –

Context :
The US has passed the peak on new coronavirus cases, President Donald Trump said and predicted that some states would reopen this month.The US has over 637,000 confirmed Covid-19 cases and over 30,826 deaths, the highest for any country in the world.Question:
What was President Donald Trump's prediction?

Output

The output from our question-answering system will be an answer from the context paragraph as shown below —

Answer: 
some states would reopen this month.

Let’s get started :

First, let’s see how we can answer a single question as shown above. Then we will see how we can leverage batch processing to answer multiple questions at once from the context.

Single Inference :

Here is the code to do single inference with DistilBERT :

The output will be :

Question What was President Donald Trump's prediction?Answer Tokens:
['some', 'states', 'would', 're', '##open', 'this', 'month']Answer : some states would reopen this month

Batch Inference :

Now let’s try to do the same with batch inference where we try to pass three questions and get answers for them as a batch –

Three questions are –

1. What was President Donald Trump's prediction?
2. How many deaths have been reported from the virus?
3. How many cases have been reported in the United States?

Here is the code to do batch inference with DistilBERT :

The output will be —

Context : The US has passed the peak on new coronavirus cases, President Donald Trump said and predicted that some states would reopen this month.The US has over 637,000 confirmed Covid-19 cases and over 30,826 deaths, the highest for any country in the world.Question: What was President Donald Trump's prediction?
Answer: some states would reopen this monthQuestion: How many deaths have been reported from the virus?
Answer: 30 , 826Question: How many cases have been reported in the United States?
Answer: over 637 , 000

If you are interested in automatic question generation from context rather than question answering , you can check out my various algorithms and open-sourced code in the links below –

1. True or False Question Generation
2. Multiple Choice Question Generation
3. Generate pronoun questions for English language learning
4. Grammar MCQ generation

Happy coding! If you have any questions or just wanted to say hi, you can reach out to me on ramsrigouthamg@gmail.com