Convolutional Neural Network in Natural Language Processing

栏目: IT技术 · 发布时间: 4年前

内容简介:Neural networks are a set of algorithms designed to recognize patterns. These patterns are numbers contained in vectors that are translated from real-world data such as images, sound, text or time series. A convolutional neural network is a neural network

What Convolutional Neural Network is and How to Utilize it for Sentiment Analysis?

Apr 21 ·4min read

What is Convolutional Neural Network?

Neural networks are a set of algorithms designed to recognize patterns. These patterns are numbers contained in vectors that are translated from real-world data such as images, sound, text or time series. A convolutional neural network is a neural network that applies convolutional layers to local features.

To understand how a small kernel can transform the large input data, see the gif below.

Convolutional Neural Network in Natural Language Processing

Retrieved from Stack Exchange

As we can see, every kernel is small spatially (along width and height), but extends through the full depth of the input volume and transform the input data as it slides.

But what if we want to different outputs? No problem. We just need to apply different filters.

Convolutional Neural Network in Natural Language Processing

Explore CNN in Setosa.io

These filters can also be applied in multi-dimensional output. The below image is input in 3D, with the image of size 7x7 and the last dimension represents the 3 channels of color (red, blue, green). This makes the input size 7x7x3. Then we apply 2 filters w0 and w1, each with size 3x3x3 to match the input size. What should we expect for the dimension of the output? 3x3x2 with 2 the number of filters.

Convolutional Neural Network in Natural Language Processing

Retrieved from gitbook

Convolutional Neural Network in Natural Language Processing

So we understand what convolutional neural network is and get an idea of how CNN can be applied to images. But how does CNN really work in NLP? For example, if we have a sentence “I love my new iphone” how can we use CNN to classify if this sentence is negative, positive, or neutral?

Convolutional Neural Network in Natural Language Processing

Image retrieved from Stack Exchange

Short explanations of the image above from left to right:

  • The inputs are words. Each word is represented by a vector of size 7.
  • Apply 4 different filters on the word vectors to create convolutional feature map
  • Choose the maximum value of the result from each filter vector for pooled representation
  • Apply softmax to transform a vector of size 1x4 to a vector of size 1x3 for classification

Learn by Example: Sentiment Analysis with PyTorch

PyTorch is a library for Python programs that facilitates building deep learning projects. In case you don’t know about PyTorch, check my article on the topic:

If you don’t have GPU in your machine, I encourage you to use Google Colab to try out with the codes. We will use this library to perform sentiment analysis on Kera’s IMDb dataset of movie reviews. Our task is to classify whether the review is positive or negative.

To build model, we use 2D convolution with nn.Conv2d(in_channels, out_channels, kernel_size) and a layer of linear neural networks for classification with nn.Linear(in_channels, out_channels).

Training steps

Visualize our loss function

import matplotlib.pyplot as pltplt.plot(LOSS)

Convolutional Neural Network in Natural Language Processing

print("F1_test: %.5f"%(get_f1(X_test, y_test)))
Convolutional Neural Network in Natural Language Processing

Awesome! Our CNN model gives us an F1 score of 0.87!

Conclusion

Congratulations! You have learned what convolutional neural network is and how to apply for natural language processing with PyTorch. I hope this gives you a general understanding of CNN and the motivation to utilize this method for your deep learning project. In case you want to gain a better understanding of CNN, t his website provides a cool interactive visualization for how the images change when applying with filters of CNN. You can try out the code of this article here .

I like to write about basic data science concepts and play with different data science tools. Follow me on Medium to get updated about my latest articles. You could also connect with me on LinkedIn and Twitter .

Check out my other blogs on data science topics:


以上就是本文的全部内容,希望本文的内容对大家的学习或者工作能带来一定的帮助,也希望大家多多支持 码农网

查看所有标签

猜你喜欢:

本站部分资源来源于网络,本站转载出于传递更多信息之目的,版权归原作者或者来源机构所有,如转载稿涉及版权问题,请联系我们

超级IP

超级IP

吴声 / 中信出版集团 / 2016-7 / 49.00元

一切商业皆内容,一切内容皆IP! 从迪士尼、airbnb、YouTube、Instagram到微信、Papi酱、芈月传、鹿晗,IP浪潮席卷全球,这不仅仅是互联网领域的革命,更是未来商业的游戏新规则。 IP从泛娱乐形态快速渗透新商业生态全维度,正深化为不同行业共同的战略方法,甚至是一种全新的商业生存方式,即IP化生存。 超级IP的内核,是辨识度极高的可认同的商业符号,它意味着一种对......一起来看看 《超级IP》 这本书的介绍吧!

JS 压缩/解压工具
JS 压缩/解压工具

在线压缩/解压 JS 代码

HTML 编码/解码
HTML 编码/解码

HTML 编码/解码

UNIX 时间戳转换
UNIX 时间戳转换

UNIX 时间戳转换