内容简介:Simpfly implementation of Quantization Aware Training[1][2] with MXNet-scala module.Tested on Ubuntu 14.041, compile MXNet with CUDA, then compile the scala-pkg,doc:
MXNET-Scala TrainQuantization
Simpfly implementation of Quantization Aware Training[1][2] with MXNet-scala module.
Setup
Tested on Ubuntu 14.04
Requirements
- sbt 0.13 http://www.scala-sbt.org/
- Mxnet v1.4 https://github.com/dmlc/mxnet
Build steps
1, compile MXNet with CUDA, then compile the scala-pkg,doc: https://github.com/dmlc/mxnet/tree/master/scala-package
2, under the Mxnet-Scala/TrainQuantization folder:
mkdir lib; ln -s $MXNET_HOME/scala-package/assembly/linux-x86_64-gpu/target/mxnet-full_2.11-linux-x86_64-gpu-1.5.0-SNAPSHOT.jar lib
3, run sbt
and then compile the project
Train vgg on Cifar10
Using the script train_vgg16_cifar10.sh
under the scripts folder to train vgg from scratch on Cifar10:
FINETUNE_MODEL_EPOCH=-1 FINETUNE_MODEL_PREFIX=$ROOT/models/
Or you can finetune with the provided pretrain model:
FINETUNE_MODEL_EPOCH=46 FINETUNE_MODEL_PREFIX=$ROOT/models/cifar10_vgg16_acc_0.8772035
I did not use any data augmentation and carefully tune the hyper-parameters during training, the best accuracy I got was 0.877, worse than the best accracy 0.93 reported on Cifar10.
Train vgg with fake quantization on Cifar10
Using the script train_quantize_vgg16_cifar10.sh
under the scripts folder to train vgg with fake quantization on Cifar10,
you must provide the pretrained model:
FINETUNE_MODEL_EPOCH=46 FINETUNE_MODEL_PREFIX=$ROOT/models/cifar10_vgg16_acc_0.8772035
If everything goes right, you should get almost the same accuray with pretrained model after serveral epoch.
Test vgg with simulated quantization on Cifar10
Using the script test_quantize_vgg16_cifar10.sh
under the scripts folder to test pretrained fake quantization vgg with simulated quantization on Cifar10, you must provide the pretrained model:
FINETUNE_MODEL_EPOCH=57 FINETUNE_MODEL_PREFIX=$ROOT/models/cifar10_quantize_vgg16_acc_0.877504
Warning
Currently there is memory leak some where in the code, but I can't figure out the reason. You will see the memory usage keep increasing when you run the tranining script. So remenber to stop the traning script when memory usage is too high, and you can resume the training process with saved model previously.
Reference
[1] Quantizing deep convolutional networks for efficient inference: A whitepaper. https://arxiv.org/pdf/1806.08342.pdf
[2] Quantization and Training of Neural Networks for Efficient Integer-Arithmetic-Only Inference. https://arxiv.org/pdf/1712.05877.pdf
以上就是本文的全部内容,希望对大家的学习有所帮助,也希望大家多多支持 码农网
猜你喜欢:- Pytorch实现卷积神经网络训练量化(QAT)
- TensorFlow 模型优化工具包 — 训练后整型量化
- 【邢不行|量化小讲堂系列05-Python量化入门】计算创业板平均市盈率
- 量化交易的尬舞
- 抄底的艺术:量化交易之路
- 用R语言开始量化投资
本站部分资源来源于网络,本站转载出于传递更多信息之目的,版权归原作者或者来源机构所有,如转载稿涉及版权问题,请联系我们。
机器学习系统设计
[德] Willi Richert、Luis Pedro Coelho / 刘峰 / 人民邮电出版社 / 2014-7-1 / CNY 49.00
如今,机器学习正在互联网上下掀起热潮,而Python则是非常适合开发机器学习系统的一门优秀语言。作为动态语言,它支持快速探索和实验,并且针对Python的机器学习算法库的数量也与日俱增。本书最大的特色,就是结合实例分析教会读者如何通过机器学习解决实际问题。 本书将向读者展示如何从原始数据中发现模式,首先从Python与机器学习的关系讲起,再介绍一些库,然后就开始基于数据集进行比较正式的项目开......一起来看看 《机器学习系统设计》 这本书的介绍吧!
HTML 编码/解码
HTML 编码/解码
XML、JSON 在线转换
在线XML、JSON转换工具