Dating Algorithms using Machine Learning and AI

栏目: IT技术 · 发布时间: 4年前

内容简介:To begin, we must first import all the necessary libraries we will need in order for this clustering algorithm to run properly. We will also load in the Pandas DataFrame, which we created when we forged the fake dating profiles.With our dataset good to go,

Preparing the Profile Data

To begin, we must first import all the necessary libraries we will need in order for this clustering algorithm to run properly. We will also load in the Pandas DataFrame, which we created when we forged the fake dating profiles.

The DataFrame containing all our data for each fake dating profile

With our dataset good to go, we can begin the next step for our clustering algorithm.

Scaling the Data

The next step, which will assist our clustering algorithm’s performance, is scaling the dating categories ( Movies, TV, religion, etc ). This will potentially decrease the time it takes to fit and transform our clustering algorithm to the dataset.

# Instantiating the Scaler
scaler = MinMaxScaler()# Scaling the categories then replacing the old values
df = df[['Bios']].join(
 pd.DataFrame(
 scaler.fit_transform(
 df.drop('Bios',axis=1)), 
 columns=df.columns[1:], 
 index=df.index))

Vectorizing the Bios

Next, we will have to vectorize the bios we have from the fake profiles. We will be creating a new DataFrame containing the vectorized bios and dropping the original ‘ Bio ’ column. With vectorization we will implementing two different approaches to see if they have significant effect on the clustering algorithm. Those two vectorization approaches are: Count Vectorization and TFIDF Vectorization . We will be experimenting with both approaches to find the optimum vectorization method.

Here we have the option of either using CountVectorizer() or TfidfVectorizer() for vectorizing the dating profile bios. When the Bios have been vectorized and placed into their own DataFrame, we will concatenate them with the scaled dating categories to create a new DataFrame with all the features we need.

Our DF that includes the vectorized bios and scaled dating categories

Based on this final DF, we have more than 100 features. Because of this, we will have to reduce the dimensionality of our dataset by using Principal Component Analysis (PCA).

PCA on the DataFrame

In order for us to reduce this large feature set, we will have to implement Principal Component Analysis (PCA) . This technique will reduce the dimensionality of our dataset but still retain much of the variability or valuable statistical information.

What we are doing here is fitting and transforming our last DF, then plotting the variance and the number of features. This plot will visually tell us how many features account for the variance.

# of Features accounting for % of the Variance

After running our code, the number of features that account for 95% of the variance is 74 . With that number in mind, we can apply it to our PCA function to reduce the number of Principal Components or Features in our last DF to 74 from 117 . These features will now be used instead of the original DF to fit to our clustering algorithm.


以上就是本文的全部内容,希望对大家的学习有所帮助,也希望大家多多支持 码农网

查看所有标签

猜你喜欢:

本站部分资源来源于网络,本站转载出于传递更多信息之目的,版权归原作者或者来源机构所有,如转载稿涉及版权问题,请联系我们

PHP项目开发全程实录

PHP项目开发全程实录

清华大学出版社 / 2008 / 56.00元

《软件项目开发全程实录丛书•PHP项目开发全程实录:DVD17小时语音视频讲解(附光盘1张)》主要特色: (1)12-32小时全程语音同步视频讲解,目前市场上唯一的“全程语音视频教学”的案例类 图书,培训数千元容,尽在一盘中! (2)10套“应用系统”并公开全部“源代码”,誓将案例学习进行到底! (3)丛书总计80个应用系统300个应用模块。 (4)含5000页SQL se......一起来看看 《PHP项目开发全程实录》 这本书的介绍吧!

HTML 压缩/解压工具
HTML 压缩/解压工具

在线压缩/解压 HTML 代码

XML、JSON 在线转换
XML、JSON 在线转换

在线XML、JSON转换工具

UNIX 时间戳转换
UNIX 时间戳转换

UNIX 时间戳转换