MXNet 实现 TensorFlow 训练模拟量化方法

栏目: 数据库 · 发布时间: 5年前

内容简介:Simpfly implementation of Quantization Aware Training[1][2] with MXNet-scala module.Tested on Ubuntu 14.041, compile MXNet with CUDA, then compile the scala-pkg,doc:

MXNET-Scala TrainQuantization

Simpfly implementation of Quantization Aware Training[1][2] with MXNet-scala module.

Setup

Tested on Ubuntu 14.04

Requirements

Build steps

1, compile MXNet with CUDA, then compile the scala-pkg,doc: https://github.com/dmlc/mxnet/tree/master/scala-package

2, under the Mxnet-Scala/TrainQuantization folder:

mkdir lib;
 ln -s $MXNET_HOME/scala-package/assembly/linux-x86_64-gpu/target/mxnet-full_2.11-linux-x86_64-gpu-1.5.0-SNAPSHOT.jar lib

3, run sbt and then compile the project

Train vgg on Cifar10

Using the script train_vgg16_cifar10.sh under the scripts folder to train vgg from scratch on Cifar10:

FINETUNE_MODEL_EPOCH=-1
FINETUNE_MODEL_PREFIX=$ROOT/models/

Or you can finetune with the provided pretrain model:

FINETUNE_MODEL_EPOCH=46
FINETUNE_MODEL_PREFIX=$ROOT/models/cifar10_vgg16_acc_0.8772035

I did not use any data augmentation and carefully tune the hyper-parameters during training, the best accuracy I got was 0.877, worse than the best accracy 0.93 reported on Cifar10.

Train vgg with fake quantization on Cifar10

Using the script train_quantize_vgg16_cifar10.sh under the scripts folder to train vgg with fake quantization on Cifar10, you must provide the pretrained model:

FINETUNE_MODEL_EPOCH=46
FINETUNE_MODEL_PREFIX=$ROOT/models/cifar10_vgg16_acc_0.8772035

If everything goes right, you should get almost the same accuray with pretrained model after serveral epoch.

Test vgg with simulated quantization on Cifar10

Using the script test_quantize_vgg16_cifar10.sh under the scripts folder to test pretrained fake quantization vgg with simulated quantization on Cifar10, you must provide the pretrained model:

FINETUNE_MODEL_EPOCH=57
FINETUNE_MODEL_PREFIX=$ROOT/models/cifar10_quantize_vgg16_acc_0.877504

Warning

Currently there is memory leak some where in the code, but I can't figure out the reason. You will see the memory usage keep increasing when you run the tranining script. So remenber to stop the traning script when memory usage is too high, and you can resume the training process with saved model previously.

Reference

[1] Quantizing deep convolutional networks for efficient inference: A whitepaper. https://arxiv.org/pdf/1806.08342.pdf

[2] Quantization and Training of Neural Networks for Efficient Integer-Arithmetic-Only Inference. https://arxiv.org/pdf/1712.05877.pdf


以上就是本文的全部内容,希望对大家的学习有所帮助,也希望大家多多支持 码农网

查看所有标签

猜你喜欢:

本站部分资源来源于网络,本站转载出于传递更多信息之目的,版权归原作者或者来源机构所有,如转载稿涉及版权问题,请联系我们

剑指Offer

剑指Offer

何海涛 / 电子工业出版社 / 2014-6-1 / CNY 55.00

《剑指Offer——名企面试官精讲典型编程题(纪念版)》是为纪念本书英文版全球发行而推出的特殊版本,在原版基础上新增大量本书英文版中的精选题目,系统整理基础知识、代码质量、解题思路、优化效率和综合能力这5个面试要点。全书分为8章,主要包括面试流程:讨论面试每一环节需要注意的问题;面试需要的基础知识:从编程语言、数据结构及算法三方面总结程序员面试知识点;高质量代码:讨论影响代码质量的3个要素(规范性......一起来看看 《剑指Offer》 这本书的介绍吧!

JS 压缩/解压工具
JS 压缩/解压工具

在线压缩/解压 JS 代码

URL 编码/解码
URL 编码/解码

URL 编码/解码

Markdown 在线编辑器
Markdown 在线编辑器

Markdown 在线编辑器