Convolutional Neural Network in Natural Language Processing

栏目: IT技术 · 发布时间: 4年前

内容简介:Neural networks are a set of algorithms designed to recognize patterns. These patterns are numbers contained in vectors that are translated from real-world data such as images, sound, text or time series. A convolutional neural network is a neural network

What Convolutional Neural Network is and How to Utilize it for Sentiment Analysis?

Apr 21 ·4min read

What is Convolutional Neural Network?

Neural networks are a set of algorithms designed to recognize patterns. These patterns are numbers contained in vectors that are translated from real-world data such as images, sound, text or time series. A convolutional neural network is a neural network that applies convolutional layers to local features.

To understand how a small kernel can transform the large input data, see the gif below.

Convolutional Neural Network in Natural Language Processing

Retrieved from Stack Exchange

As we can see, every kernel is small spatially (along width and height), but extends through the full depth of the input volume and transform the input data as it slides.

But what if we want to different outputs? No problem. We just need to apply different filters.

Convolutional Neural Network in Natural Language Processing

Explore CNN in Setosa.io

These filters can also be applied in multi-dimensional output. The below image is input in 3D, with the image of size 7x7 and the last dimension represents the 3 channels of color (red, blue, green). This makes the input size 7x7x3. Then we apply 2 filters w0 and w1, each with size 3x3x3 to match the input size. What should we expect for the dimension of the output? 3x3x2 with 2 the number of filters.

Convolutional Neural Network in Natural Language Processing

Retrieved from gitbook

Convolutional Neural Network in Natural Language Processing

So we understand what convolutional neural network is and get an idea of how CNN can be applied to images. But how does CNN really work in NLP? For example, if we have a sentence “I love my new iphone” how can we use CNN to classify if this sentence is negative, positive, or neutral?

Convolutional Neural Network in Natural Language Processing

Image retrieved from Stack Exchange

Short explanations of the image above from left to right:

  • The inputs are words. Each word is represented by a vector of size 7.
  • Apply 4 different filters on the word vectors to create convolutional feature map
  • Choose the maximum value of the result from each filter vector for pooled representation
  • Apply softmax to transform a vector of size 1x4 to a vector of size 1x3 for classification

Learn by Example: Sentiment Analysis with PyTorch

PyTorch is a library for Python programs that facilitates building deep learning projects. In case you don’t know about PyTorch, check my article on the topic:

If you don’t have GPU in your machine, I encourage you to use Google Colab to try out with the codes. We will use this library to perform sentiment analysis on Kera’s IMDb dataset of movie reviews. Our task is to classify whether the review is positive or negative.

To build model, we use 2D convolution with nn.Conv2d(in_channels, out_channels, kernel_size) and a layer of linear neural networks for classification with nn.Linear(in_channels, out_channels).

Training steps

Visualize our loss function

import matplotlib.pyplot as pltplt.plot(LOSS)

Convolutional Neural Network in Natural Language Processing

print("F1_test: %.5f"%(get_f1(X_test, y_test)))
Convolutional Neural Network in Natural Language Processing

Awesome! Our CNN model gives us an F1 score of 0.87!

Conclusion

Congratulations! You have learned what convolutional neural network is and how to apply for natural language processing with PyTorch. I hope this gives you a general understanding of CNN and the motivation to utilize this method for your deep learning project. In case you want to gain a better understanding of CNN, t his website provides a cool interactive visualization for how the images change when applying with filters of CNN. You can try out the code of this article here .

I like to write about basic data science concepts and play with different data science tools. Follow me on Medium to get updated about my latest articles. You could also connect with me on LinkedIn and Twitter .

Check out my other blogs on data science topics:


以上就是本文的全部内容,希望本文的内容对大家的学习或者工作能带来一定的帮助,也希望大家多多支持 码农网

查看所有标签

猜你喜欢:

本站部分资源来源于网络,本站转载出于传递更多信息之目的,版权归原作者或者来源机构所有,如转载稿涉及版权问题,请联系我们

写给Web开发人员看的HTML5教程

写给Web开发人员看的HTML5教程

2012-3 / 45.00元

《写给Web开发人员看的HTML5教程》通过结合大量实际案例和源代码对HTML5的重要特性进行了详细讲解,内容全面丰富,易于理解。全书共分为12章,从HTML5的历史故事讲起,涉及了文档结构和语义、智能表单、视频与音频、画布、SVG与MathML、地理定位、Web存储与离线Web应用程序、WebSockets套接字、WebWorker多线程、微数据以及以拖曳为代表的一些全局属性,涵盖了HTML5所......一起来看看 《写给Web开发人员看的HTML5教程》 这本书的介绍吧!

HTML 压缩/解压工具
HTML 压缩/解压工具

在线压缩/解压 HTML 代码

HTML 编码/解码
HTML 编码/解码

HTML 编码/解码

正则表达式在线测试
正则表达式在线测试

正则表达式在线测试