Building the “Hello World” of Kaggle projects using AutoAI

栏目: IT技术 · 发布时间: 4年前

内容简介:I always get asked what’s the best way to get started with data science? Well, my response to that is always the same, do a lot of data science courses to get familiar with the concepts and then move on to building projects by participating in Kaggle compe

I always get asked what’s the best way to get started with data science? Well, my response to that is always the same, do a lot of data science courses to get familiar with the concepts and then move on to building projects by participating in Kaggle competitions. In this blog, I will show how I started my Kaggle journey by completing the titanic survival Kaggle competition using AutoAI in just 30 minutes.

What are Kaggle competitions?

Building the “Hello World” of Kaggle projects using AutoAI

Photo by Justin Chan on Data-Driven Investor

Kaggle provides data science enthusiasts a platform for analytics competitions in which companies and researchers post data to allow enthusiasts to compete to produce the best models for predicting and describing the data. With over 13,000 datasets at present, Kaggle offers a veritable gold mine of data to work with.

Why IBM Watson AutoAI?

AutoAI in Watson Studio automates tasks that typically take data scientists days or weeks. All you need to do is submit your data and leave the rest on the tool to decide the best pipeline for your project.

Building the “Hello World” of Kaggle projects using AutoAI

Photo by Greg Filla on IBM Watson

Let the Kaggling begin!

The titanic project is known as the ‘hello world’ of Kaggle projects since beginners can get hands-on Kaggle experience before attempting complex projects.

Preparation:

  1. Set up AutoAI: — Create an IBM Cloud account. — Create a Watson Studio Instance . — Create a project — Add AutoAI to the project.
  2. Create an account on Kaggle
  3. Participate in the titanic survival competition

Step 1 — Data Collection:

We will have to download the dataset from the competition page . The zip file would contain 3 files- the train.csv file is the one we will use for training the model, the test is the file we will use to batch score our model and use it for submission purposes and the gender_ submission file shows how the submission file for Kaggle should look like (its the template we will fill out).

Step 2 — Data preparation:

The data is pretty much clean but there are null values that need to be taken care of in the training dataset and the testing dataset. So first, let’s replace the null values in the Age and Fare columns with the average of their values. I used the Excel formula ‘AVERAGE’ to find and replace the values. And I left the Cabin column to be null. While testing the model, this is being taken care of (to be continued …)

Step 3 — Model Building using AutoAI:

Although this step may seem hard, it's the easiest because we are using AutoAI. Simply create an AutoAI project in Watson Studio, give your experiment a name, and upload your train.csv file. Select ‘Survived to be your predicted variable, Run the experiment, and wait for at least 15 minutes.

Building the “Hello World” of Kaggle projects using AutoAI

AutoAI takes the dataset and the target variable to design pipelines (these are different models) using various HPO (hyperparameter optimization) parameters and enhanced feature engineering for each pipeline to get the best model.

Building the “Hello World” of Kaggle projects using AutoAI

As you might already know, there are different ways to evaluate and select the best model such as Accuracy, F1 score, precision, etc. You can edit this to suit your needs as well, but we will select accuracy for this case (you are free to try other evaluators as well). In the diagram below, you can see how each pipeline performed against different evaluators. This is how the best pipeline (leader pipeline) is selected and recommended to be deployed.

Building the “Hello World” of Kaggle projects using AutoAI

Now to deploy the model click the first pipeline (with a star), save as, and click model. This will let you deploy your model as a Watson machine learning model.

Go back to your project, click the WML model and deploy it as a web service and once it’s deployed you may test it and grab the scoring link which we will use in the next step.

Building the “Hello World” of Kaggle projects using AutoAI

Step4 — Batch scoring the model

Now that we have our model, let's create a python script to batch score the AutoAI model against our test.csv to submit our results.

Below is the code to run a batch score test for all the records in the test file.

The code basically stores each record in variables one by one so it can pass it as a payload to be scored by the model. Since some of the Cabin values are empty we replace them with None. Once we get the JSON result from the model, we parse it to get the predicted value for the record and store it in the array which would be written back to the Results.csv sheet.

Building the “Hello World” of Kaggle projects using AutoAI

Results.csv file

Final Step— Submitting the results to Kaggle

Go to the competition page and click on submit results to submit your Results file. Wait until you appear on the leadership board and your score(my score is 77%) along with the rank will be shown on the screen. The score determines your rank. You can keep improving your model and keep submitting multiple times to reach the top.


以上所述就是小编给大家介绍的《Building the “Hello World” of Kaggle projects using AutoAI》,希望对大家有所帮助,如果大家有任何疑问请给我留言,小编会及时回复大家的。在此也非常感谢大家对 码农网 的支持!

查看所有标签

猜你喜欢:

本站部分资源来源于网络,本站转载出于传递更多信息之目的,版权归原作者或者来源机构所有,如转载稿涉及版权问题,请联系我们

长尾理论

长尾理论

[美] 克里斯·安德森 / 乔江涛 / 中信出版社 / 2006-12 / 35.00元

书中阐述,商业和文化的未来不在于传统需求曲线上那个代表“畅销商品”(hits)的头部; 而是那条代表“冷门商品”(misses)经常为人遗忘的长尾。 举例来说, 一家大型书店通常可摆放10万本书,但亚马逊网络书店的图书销售额中,有四分之一来自排名10万以后的书籍。这些“冷门”书籍的销售比例正以高速成长,预估未来可占整体书市的一半。 这意味着消费者在面对无限的选择时,真正想要的东西、和想要取得......一起来看看 《长尾理论》 这本书的介绍吧!

SHA 加密
SHA 加密

SHA 加密工具

XML、JSON 在线转换
XML、JSON 在线转换

在线XML、JSON转换工具

UNIX 时间戳转换
UNIX 时间戳转换

UNIX 时间戳转换