MXNet 实现 Self-normalizing networks

栏目: 数据库 · 发布时间: 7年前

内容简介:MXNet 实现 Self-normalizing networks

MXNET-Scala Self-Norm Nets

MXNet-scala module implementation of Self-normalizing networks[1].

Based on: https://github.com/bioinf-jku/SNNs

Building

Tested on Ubuntu 14.04

Requirements

  • sbt 0.13
  • Mxnet

steps

1, compile Mxnet with CUDA, then compile the scala-pkg;

2,

cd Mxnet-Scala/SelfNormNets
mkdir lib

3, copy your compiled mxnet-full_2.11-linux-x86_64-gpu-0.10.1-SNAPSHOT.jar into lib folder;

4, run sbt, compile the project

Running

1, use datas/get_mnist_data.sh script to download the mnist dataset

2, run trainSNN_CNN_MNIST.sh or trainSNN_MLP_MNIST.sh under scripts folder

Training logs

Train MLP

logs

bash trainSNN_MLP_MNIST.sh

Epoch[0] Train-accuracy=0.86191666
Epoch[0] Time cost=1646
Epoch[0] Validation-accuracy=0.9341
Epoch[1] Train-accuracy=0.9213667
Epoch[1] Time cost=1478
Epoch[1] Validation-accuracy=0.9485
Epoch[2] Train-accuracy=0.9421667
Epoch[2] Time cost=1428
Epoch[2] Validation-accuracy=0.9402
Epoch[3] Train-accuracy=0.9501
Epoch[3] Time cost=1415
Epoch[3] Validation-accuracy=0.9669
Epoch[4] Train-accuracy=0.9571667
Epoch[4] Time cost=1604
Epoch[4] Validation-accuracy=0.9623
Epoch[5] Train-accuracy=0.96195
Epoch[5] Time cost=1457
Epoch[5] Validation-accuracy=0.9614
Epoch[6] Train-accuracy=0.9679667
Epoch[6] Time cost=1591
Epoch[6] Validation-accuracy=0.9673
Epoch[7] Train-accuracy=0.97048336
Epoch[7] Time cost=1629
Epoch[7] Validation-accuracy=0.9639
Epoch[8] Train-accuracy=0.9719333
Epoch[8] Time cost=1668
Epoch[8] Validation-accuracy=0.9703
Epoch[9] Train-accuracy=0.9753
Epoch[9] Time cost=1662
Epoch[9] Validation-accuracy=0.9728
Epoch[10] Train-accuracy=0.9769
Epoch[10] Time cost=1526
Epoch[10] Validation-accuracy=0.9752
Epoch[11] Train-accuracy=0.9784333
Epoch[11] Time cost=1487
Epoch[11] Validation-accuracy=0.9709
Epoch[12] Train-accuracy=0.98066664
Epoch[12] Time cost=1609
Epoch[12] Validation-accuracy=0.9753
Epoch[13] Train-accuracy=0.98113334
Epoch[13] Time cost=1475
Epoch[13] Validation-accuracy=0.9725
Epoch[14] Train-accuracy=0.98215
Epoch[14] Time cost=1477
Epoch[14] Validation-accuracy=0.9749

Compare selu with relu

logs

bash trainSNN_CNN_MNIST.sh

Epoch[0] SNN Train-accuracy=0.88266224
Epoch[0] ReLU Train-accuracy=0.807926
Epoch[1] SNN Train-accuracy=0.9415899
Epoch[1] ReLU Train-accuracy=0.8241854
Epoch[2] SNN Train-accuracy=0.95097154
Epoch[2] ReLU Train-accuracy=0.8243189
Epoch[3] SNN Train-accuracy=0.95880073
Epoch[3] ReLU Train-accuracy=0.833734
Epoch[4] SNN Train-accuracy=0.9629741
Epoch[4] ReLU Train-accuracy=0.82568777
Epoch[5] SNN Train-accuracy=0.96793205
Epoch[5] ReLU Train-accuracy=0.8318643
Epoch[6] SNN Train-accuracy=0.9703693
Epoch[6] ReLU Train-accuracy=0.8342181
Epoch[7] SNN Train-accuracy=0.97163796
Epoch[7] ReLU Train-accuracy=0.83628803
Epoch[8] SNN Train-accuracy=0.9741086
Epoch[8] ReLU Train-accuracy=0.8316807
Epoch[9] SNN Train-accuracy=0.9753105
Epoch[9] ReLU Train-accuracy=0.8397269
SNN Validation-accuracy=0.96334136
ReLU Validation-accuracy=0.9423077

Referneces

[1] Klambauer, Günter, et al. "Self-Normalizing Neural Networks." arXiv preprint arXiv:1706.02515 (2017).


以上就是本文的全部内容,希望本文的内容对大家的学习或者工作能带来一定的帮助,也希望大家多多支持 码农网

查看所有标签

猜你喜欢:

本站部分资源来源于网络,本站转载出于传递更多信息之目的,版权归原作者或者来源机构所有,如转载稿涉及版权问题,请联系我们

信息检索导论

信息检索导论

Christopher D.Manning、Hinrich Schütze、Prabhakar Raghavan / 王斌 / 人民邮电出版社 / 201008 / 69.00元

封面图片为英国伯明翰塞尔福瑞吉百货大楼,其极具线条感的轮廓外型优美,犹如水波的流动。其外表悬挂了1.5万个铝碟,创造出一种极具现代气息的纹理装饰效果,有如夜空下水流的波光粼粼,闪烁于月光之下,使建筑的商业氛围表现到极致。设计该建筑的英国“未来系统建筑事物所”,将商场内部围合成一个顶部采光的中庭,配以交叉的自动扶梯,使购物环境呈现出一种凝聚的向心力和商业广告的展示效应。作为英国第二商业城市伯明翰的建......一起来看看 《信息检索导论》 这本书的介绍吧!

JSON 在线解析
JSON 在线解析

在线 JSON 格式化工具

SHA 加密
SHA 加密

SHA 加密工具

XML 在线格式化
XML 在线格式化

在线 XML 格式化压缩工具