DefaultQuantization, AccuracyAwareQuantization by OpenVINO's post training optimization toolkit, INT8 (Integer Quantization)

１． Introduction

Organize the procedure for INT8 Quantification of EfficientNet by "post training optimization toolkit" of OpenVINO.

２． Environment

• Ubuntu 18.04 x86_64
• Corei7-8750H (No AVX-512 instruction set)
• Tensorflow-gpu v1.15.2
• PyTorch
• Python 3.6
• OpenVINO 2020 R1
• EfficientNet D0 (Pytorch)
• Numpy 1.18.5 ... Only at build time
• Numpy 1.20.1 ... Only at runtime

３． Procedure

３−１．Install AccuracyChecker and post_training_optimization_toolkit

https://docs.openvinotoolkit.org/latest/_README.html

Install_AccuracyChecker_and_post_training_optimization_toolkit

### Install the AccuracyChecker component
$ cd $INTEL_OPENVINO_DIR/deployment_tools/open_model_zoo/tools/accuracy_checker/
$ sudo python3 setup.py install

### Install the toolkit
$ cd $INTEL_OPENVINO_DIR/deployment_tools/tools/post_training_optimization_toolkit/
$ sudo python3 setup.py install

３−２．Download a trained model of EfficientNet_D0

Download_a_trained_model_of_EfficientNet_D0

$ cd $INTEL_OPENVINO_DIR/deployment_tools/tools/post_training_optimization_toolkit
$ cd libs/open_model_zoo/tools/downloader

$ sudo python3 downloader.py \
  --name efficientnet-b0-pytorch

３−３．Convert Pytorch model to IR model

Convert_Pytorch_model_to_IR_model

$ sudo python3 converter.py \
  --name efficientnet-b0-pytorch \
  --mo $INTEL_OPENVINO_DIR/deployment_tools/model_optimizer/mo.py

Log

========= Converting efficientnet-b0-pytorch to ONNX
Conversion to ONNX command: /usr/bin/python3 /opt/intel/openvino_2020.1.023/deployment_tools/open_model_zoo/tools/downloader/pytorch_to_onnx.py --model-path=/opt/intel/openvino_2020.1.023/deployment_tools/open_model_zoo/tools/downloader/public/efficientnet-b0-pytorch --model-name=efficientnet_b0 --import-module=model --weights=/opt/intel/openvino_2020.1.023/deployment_tools/open_model_zoo/tools/downloader/public/efficientnet-b0-pytorch/efficientnet-b0.pth --input-shape=1,3,224,224 --input-names=data --output-names=prob --output-file=/opt/intel/openvino_2020.1.023/deployment_tools/open_model_zoo/tools/downloader/public/efficientnet-b0-pytorch/efficientnet-b0.onnx

ONNX check passed successfully.

========= Converting efficientnet-b0-pytorch to IR (FP16)
Conversion command: /usr/bin/python3 -- /opt/intel/openvino/deployment_tools/model_optimizer/mo.py --framework=onnx --data_type=FP16 --output_dir=/opt/intel/openvino_2020.1.023/deployment_tools/open_model_zoo/tools/downloader/public/efficientnet-b0-pytorch/FP16 --model_name=efficientnet-b0-pytorch --reverse_input_channels '--input_shape=[1,3,224,224]' --input=data '--mean_values=data[123.675,116.28,103.53]' '--scale_values=data[58.395,57.12,57.375]' --output=prob --input_model=/opt/intel/openvino_2020.1.023/deployment_tools/open_model_zoo/tools/downloader/public/efficientnet-b0-pytorch/efficientnet-b0.onnx

Model Optimizer arguments:
Common parameters:
    - Path to the Input Model:  /opt/intel/openvino_2020.1.023/deployment_tools/open_model_zoo/tools/downloader/public/efficientnet-b0-pytorch/efficientnet-b0.onnx
    - Path for generated IR:    /opt/intel/openvino_2020.1.023/deployment_tools/open_model_zoo/tools/downloader/public/efficientnet-b0-pytorch/FP16
    - IR output name:   efficientnet-b0-pytorch
    - Log level:    ERROR
    - Batch:    Not specified, inherited from the model
    - Input layers:     data
    - Output layers:    prob
    - Input shapes:     [1,3,224,224]
    - Mean values:  data[123.675,116.28,103.53]
    - Scale values:     data[58.395,57.12,57.375]
    - Scale factor:     Not specified
    - Precision of IR:  FP16
    - Enable fusing:    True
    - Enable grouped convolutions fusing:   True
    - Move mean values to preprocess section:   False
    - Reverse input channels:   True
ONNX specific parameters:
Model Optimizer version:    2020.1.0-61-gd349c3ba4a

[ SUCCESS ] Generated IR version 10 model.
[ SUCCESS ] XML file: /opt/intel/openvino_2020.1.023/deployment_tools/open_model_zoo/tools/downloader/public/efficientnet-b0-pytorch/FP16/efficientnet-b0-pytorch.xml
[ SUCCESS ] BIN file: /opt/intel/openvino_2020.1.023/deployment_tools/open_model_zoo/tools/downloader/public/efficientnet-b0-pytorch/FP16/efficientnet-b0-pytorch.bin
[ SUCCESS ] Total execution time: 14.41 seconds. 
[ SUCCESS ] Memory consumed: 189 MB. 

========= Converting efficientnet-b0-pytorch to IR (FP32)
Conversion command: /usr/bin/python3 -- /opt/intel/openvino/deployment_tools/model_optimizer/mo.py --framework=onnx --data_type=FP32 --output_dir=/opt/intel/openvino_2020.1.023/deployment_tools/open_model_zoo/tools/downloader/public/efficientnet-b0-pytorch/FP32 --model_name=efficientnet-b0-pytorch --reverse_input_channels '--input_shape=[1,3,224,224]' --input=data '--mean_values=data[123.675,116.28,103.53]' '--scale_values=data[58.395,57.12,57.375]' --output=prob --input_model=/opt/intel/openvino_2020.1.023/deployment_tools/open_model_zoo/tools/downloader/public/efficientnet-b0-pytorch/efficientnet-b0.onnx

Model Optimizer arguments:
Common parameters:
    - Path to the Input Model:  /opt/intel/openvino_2020.1.023/deployment_tools/open_model_zoo/tools/downloader/public/efficientnet-b0-pytorch/efficientnet-b0.onnx
    - Path for generated IR:    /opt/intel/openvino_2020.1.023/deployment_tools/open_model_zoo/tools/downloader/public/efficientnet-b0-pytorch/FP32
    - IR output name:   efficientnet-b0-pytorch
    - Log level:    ERROR
    - Batch:    Not specified, inherited from the model
    - Input layers:     data
    - Output layers:    prob
    - Input shapes:     [1,3,224,224]
    - Mean values:  data[123.675,116.28,103.53]
    - Scale values:     data[58.395,57.12,57.375]
    - Scale factor:     Not specified
    - Precision of IR:  FP32
    - Enable fusing:    True
    - Enable grouped convolutions fusing:   True
    - Move mean values to preprocess section:   False
    - Reverse input channels:   True
ONNX specific parameters:
Model Optimizer version:    2020.1.0-61-gd349c3ba4a

[ SUCCESS ] Generated IR version 10 model.
[ SUCCESS ] XML file: /opt/intel/openvino_2020.1.023/deployment_tools/open_model_zoo/tools/downloader/public/efficientnet-b0-pytorch/FP32/efficientnet-b0-pytorch.xml
[ SUCCESS ] BIN file: /opt/intel/openvino_2020.1.023/deployment_tools/open_model_zoo/tools/downloader/public/efficientnet-b0-pytorch/FP32/efficientnet-b0-pytorch.bin
[ SUCCESS ] Total execution time: 14.62 seconds. 
[ SUCCESS ] Memory consumed: 190 MB.

３−４．Check model definition information

$INTEL_OPENVINO_DIR/deployment_tools/open_model_zoo/tools/accuracy_checker/configs/efficientnet-b0-pytorch.yml

models:
  - name: efficientnet-b0-pytorch

    launchers:
      - framework: onnx_runtime
        model: public/efficientnet-b0-pytorch/efficientnet-b0.onnx
        adapter: classification
        inputs:
          - name: data
            type: INPUT
            shape: 1,3,224,224


    datasets:
      - name: imagenet_1000_classes
        reader: pillow_imread

        preprocessing:
          - type: resize
            size: 256
            aspect_ratio_scale: greater
            use_pillow: True
            interpolation: BICUBIC

          - type: crop
            use_pillow: True
            size: 224

          - type: normalization
            mean: (123.675,116.28,103.53)
            std: (58.395,57.12,57.375)

  - name: efficientnet-b0-pytorch

    launchers:
      - framework: dlsdk
        tags:
          - FP32
        model:   public/efficientnet-b0-pytorch/FP32/efficientnet-b0-pytorch.xml
        weights: public/efficientnet-b0-pytorch/FP32/efficientnet-b0-pytorch.bin
        adapter: classification

      - framework: dlsdk
        tags:
          - FP16
        model:   public/efficientnet-b0-pytorch/FP16/efficientnet-b0-pytorch.xml
        weights: public/efficientnet-b0-pytorch/FP16/efficientnet-b0-pytorch.bin
        adapter: classification

    datasets:
      - name: imagenet_1000_classes
        reader: pillow_imread

        preprocessing:
          - type: rgb_to_bgr

          - type: resize
            size: 256
            aspect_ratio_scale: greater
            use_pillow: True
            interpolation: BICUBIC

          - type: crop
            use_pillow: True
            size: 224

global_definitions: ../dataset_definitions.yml

３−５．Confirmation of definition information of calibration data set (ImageNet)

$INTEL_OPENVINO_DIR/deployment_tools/open_model_zoo/tools/accuracy_checker/dataset_definitions.yml

  - name: imagenet_1000_classes
    annotation_conversion:
      converter: imagenet
      annotation_file: val.txt
    annotation: imagenet1000.pickle
    data_source: ILSVRC2012_img_val
    metrics:
      - name: accuracy@top1
        type: accuracy
        top_k: 1
      - name: accuracy@top5
        type: accuracy
        top_k: 5

３−６．Check the contents of the sample definition file of the quantization setting file

https://docs.openvinotoolkit.org/latest/_configs_README.html
https://docs.openvinotoolkit.org/latest/_compression_algorithms_quantization_README.html
https://docs.openvinotoolkit.org/latest/_compression_algorithms_quantization_default_README.html
https://docs.openvinotoolkit.org/latest/_compression_algorithms_quantization_accuracy_aware_README.html

"target_device"_Specifiable_parameters

        "target_device": "CPU", # ---> [CPU or CPU_VNNI or GPU or VPU]

"algorithms"_Specifiable_parameters

        "algorithms": [
            {
                "name": "DefaultQuantization", # ---> optimization algorithm name [DefaultQuantization, AccuracyAwareQuantization]
                "preset": "performance", # ---> Preset to control quantization mode [performance(Default), accuracy]
                "params": {
                    "stat_subset_size": 300 # ---> The size of the subset for calculating the activation statistics used for quantization. If no parameters are specified, the entire dataset will be used. example）300 images
                }
            }

３−７．Creating a quantization setting file

$ cd ~
$ mkdir -p openvino_quantization
$ nano openvino_quantization/efficientnet-b0-pytorch.json

efficientnet-b0-pytorch.json

{
    "model": {
        "model_name": "efficientnet-b0-pytorch",
        "model": "/opt/intel/openvino_2020.1.023/deployment_tools/open_model_zoo/tools/downloader/public/efficientnet-b0-pytorch/FP32/efficientnet-b0-pytorch.xml",
        "weights": "/opt/intel/openvino_2020.1.023/deployment_tools/open_model_zoo/tools/downloader/public/efficientnet-b0-pytorch/FP32/efficientnet-b0-pytorch.bin"
    },
    "engine": {
        "config": "/opt/intel/openvino/deployment_tools/open_model_zoo/tools/accuracy_checker/configs/efficientnet-b0-pytorch.yml"
    },
    "compression": {
        "target_device": "CPU",
        "algorithms": [
            {
                "name": "DefaultQuantization",
                "preset": "performance",
                "params": {
                    "stat_subset_size": 300
                }
            }
        ]
    }
}

３−８．Preparing the Imagenet dataset

https://hiro99ma.blogspot.com/2015/10/aicaffe_7.html
https://github.com/BVLC/caffe/tree/master/examples/imagenet
https://github.com/BVLC/caffe/tree/master/data/ilsvrc12

$ cd ~
$ git clone --depth 1 https://github.com/BVLC/caffe.git
$ cd caffe
$ ./data/ilsvrc12/get_ilsvrc_aux.sh

$ ls -1 data/ilsvrc12/
det_synset_words.txt
get_ilsvrc_aux.sh
imagenet.bet.pickle
imagenet_mean.binaryproto
synset_words.txt
synsets.txt
test.txt
train.txt
val.txt

view_pickle.py

import pickle
import base64
import json
import numpy as np

picklepath = 'data/ilsvrc12/imagenet.bet.pickle'
jsonpath = 'data/ilsvrc12/imagenet.bet.json'

class NumpyEncoder(json.JSONEncoder):
    def default(self, obj):
        if isinstance(obj, np.ndarray):
            return obj.tolist()
        return json.JSONEncoder.default(self, obj)

with open(picklepath, 'rb') as file:
    data = pickle.load(file, encoding='latin1')
    print(data)

json_string = json.dumps(data, cls=NumpyEncoder)
with open(jsonpath, 'w') as j:
    json.dump(json_string, j)

$ python3 data/ilsvrc12/view_pickle.py

３−９．Preparation of annotation data

https://github.com/opencv/open_model_zoo/blob/master/tools/accuracy_checker/accuracy_checker/annotation_converters/README.md

usage: convert_annotation [-o OUTPUT_DIR] [-m META_NAME] [-a ANNOTATION_NAME]
                          [-ss SUBSAMPLE] [--subsample_seed SUBSAMPLE_SEED]
                          [--analyze_dataset]
                          {market1501_reid,mars,voc_segmentation,voc_detection,
                           sample,wider,detection_opencv_storage,lfw,vgg_face,
                           super_resolution,multi_frame_super_resolution,imagenet,
                           icdar_detection,icdar13_recognition,mscoco_detection,
                           mscoco_keypoints,mscoco_segmentation,mscoco_mask_rcnn,
                           mscoco_single_keypoints,cityscapes,movie_lens_converter,
                           brats,brats_numpy,cifar10,mnist_csv,wmt,
                           common_semantic_segmentation,camvid,lpr_txt,
                           image_retrieval,cvat_object_detection,
                           cvat_attributes_recognition,cvat_age_gender,
                           cvat_facial_landmarks,cvat_text_recognition,
                           cvat_multilabel_binary_attributes_recognition,
                           cvat_pose_estimation,cvat_person_detection_action_recognition,
                           squad,xnli,bert_xnli_tf_record,cmu_panoptic_keypoints,
                           clip_action_recognition,continuous_clip_action_recognition}

$ convert_annotation \
  --output_dir ${HOME}/caffe/data/ilsvrc12 \
  --annotation_name imagenet1000.pickle \
  imagenet \
  --annotation_file ${HOME}/caffe/data/ilsvrc12/val.txt \
  --labels_file ${HOME}/caffe/data/ilsvrc12/synset_words.txt \
  --has_background False

$ cp ${HOME}/caffe/data/ilsvrc12/imagenet1000.pickle ${HOME}/openvino_quantization && \
  cp ${HOME}/caffe/data/ilsvrc12/val.txt ${HOME}/openvino_quantization

３−１０．Perform quantization

Commandline_tool：POT=post_training_optimization_toolkit

$ cd ~/openvino_quantization

$ pot \
  --config efficientnet-b0-pytorch.json \
  --save-model

or

$ pot \
  --config efficientnet-b0-pytorch.json \
  --evaluate \
  --save-model

Log

/usr/local/lib/python3.6/dist-packages/pot-1.0-py3.6.egg/compression/algorithms/quantization/optimization/algorithm.py:39: UserWarning: Nevergrad package could not be imported. Consider installing itusing pip. Note that nevergrad is compatible only with Python 3.6+
  warnings.warn('Nevergrad package could not be imported. Consider installing it'
Output log dir: ./results/efficientnet-b0-pytorch_DefaultQuantization/2020-03-22_23-09-11
Creating pipeline:
Algorithm: DefaultQuantization
Parameters:
    stat_subset_size           : 300
    target_device              : CPU
===========================================================================
INFO:compression.statistics.collector:Start computing statistics for algorithms : DefaultQuantization
INFO:compression.statistics.collector:Computing statistics finished
INFO:compression.pipeline.pipeline:Start algorithm: DefaultQuantization
INFO:compression.statistics.collector:Start computing statistics for algorithms : ActivationChannelAlignment
INFO:compression.statistics.collector:Computing statistics finished
INFO:compression.statistics.collector:Start computing statistics for algorithms : MinMaxQuantization,BiasCorrection
INFO:compression.statistics.collector:Computing statistics finished
INFO:compression.pipeline.pipeline:Finished: DefaultQuantization
 ===========================================================================
INFO:compression.pipeline.pipeline:Evaluation of generated model
INFO:compression.engines.ac_engine:Start inference on the whole dataset
Total dataset size: 50000
1000 / 50000 processed in 23.321s
2000 / 50000 processed in 23.383s
3000 / 50000 processed in 24.053s
　：
49000 / 50000 processed in 27.800s
50000 / 50000 processed in 26.584s
50000 objects processed in 1258.780 seconds
INFO:compression.engines.ac_engine:Inference finished
accuracy@top1              : 0.76102
accuracy@top5              : 0.9277

$ pot -h
usage: pot [-h] -c CONFIG [-e] [--output-dir OUTPUT_DIR] [-sm] [-d]
           [--log-level {CRITICAL,ERROR,WARNING,INFO,DEBUG}] [--progress-bar]

Post-training Compression Toolkit

optional arguments:
  -h, --help            show this help message and exit
  -c CONFIG, --config CONFIG
                        Path to a config file with task/model-specific
                        parameters
  -e, --evaluate        Whether to evaluate model on whole dataset
  --output-dir OUTPUT_DIR
                        The directory where models are saved. Default:
                        ./results
  -sm, --save-model     Saves fp32 model into output directory
  -d, --direct-dump     Flag to save files without sub folders with algo names
  --log-level {CRITICAL,ERROR,WARNING,INFO,DEBUG}
                        Log level to print: ['CRITICAL', 'ERROR', 'WARNING',
                        'INFO', 'DEBUG']
  --progress-bar        Disable CL logging and enable progress bar

３−１１．Model benchmark

Build_benchmark_app

$ cd $INTEL_OPENVINO_DIR/deployment_tools/inference_engine/samples/cpp
$ ./build_samples.sh

Build_Log

Setting environment variables for building samples...
[setupvars.sh] OpenVINO environment initialized
-- The C compiler identification is GNU 7.4.0
-- The CXX compiler identification is GNU 7.4.0
-- Check for working C compiler: /usr/bin/cc
-- Check for working C compiler: /usr/bin/cc -- works
-- Detecting C compiler ABI info
-- Detecting C compiler ABI info - done
-- Detecting C compile features
-- Detecting C compile features - done
-- Check for working CXX compiler: /usr/bin/c++
-- Check for working CXX compiler: /usr/bin/c++ -- works
-- Detecting CXX compiler ABI info
-- Detecting CXX compiler ABI info - done
-- Detecting CXX compile features
-- Detecting CXX compile features - done
-- Looking for C++ include unistd.h
-- Looking for C++ include unistd.h - found
-- Looking for C++ include stdint.h
-- Looking for C++ include stdint.h - found
-- Looking for C++ include sys/types.h
-- Looking for C++ include sys/types.h - found
-- Looking for C++ include fnmatch.h
-- Looking for C++ include fnmatch.h - found
-- Looking for strtoll
-- Looking for strtoll - found
-- Found InferenceEngine: /opt/intel/openvino_2020.1.023/deployment_tools/inference_engine/lib/intel64/libinference_engine.so (Required is at least version "2.1") 
-- Configuring done
-- Generating done
-- Build files have been written to: /home/b920405/inference_engine_cpp_samples_build
Scanning dependencies of target gflags_nothreads_static
Scanning dependencies of target format_reader
[ 10%] Building CXX object thirdparty/gflags/CMakeFiles/gflags_nothreads_static.dir/src/gflags_reporting.cc.o
[ 10%] Building CXX object thirdparty/gflags/CMakeFiles/gflags_nothreads_static.dir/src/gflags_completions.cc.o
[ 10%] Building CXX object thirdparty/gflags/CMakeFiles/gflags_nothreads_static.dir/src/gflags.cc.o
[ 16%] Building CXX object common/format_reader/CMakeFiles/format_reader.dir/MnistUbyte.cpp.o
[ 16%] Building CXX object common/format_reader/CMakeFiles/format_reader.dir/bmp.cpp.o
[ 20%] Building CXX object common/format_reader/CMakeFiles/format_reader.dir/format_reader.cpp.o
[ 23%] Building CXX object common/format_reader/CMakeFiles/format_reader.dir/opencv_wraper.cpp.o
[ 26%] Linking CXX shared library ../../intel64/Release/lib/libformat_reader.so
[ 30%] Linking CXX static library ../../intel64/Release/lib/libgflags_nothreads.a
[ 30%] Built target gflags_nothreads_static
[ 30%] Built target format_reader
Scanning dependencies of target hello_classification
Scanning dependencies of target hello_reshape_ssd
Scanning dependencies of target hello_nv12_input_classification
Scanning dependencies of target hello_query_device
Scanning dependencies of target speech_sample
Scanning dependencies of target classification_sample_async
Scanning dependencies of target benchmark_app
Scanning dependencies of target style_transfer_sample
[ 33%] Building CXX object hello_query_device/CMakeFiles/hello_query_device.dir/main.cpp.o
[ 36%] Building CXX object hello_nv12_input_classification/CMakeFiles/hello_nv12_input_classification.dir/main.cpp.o
[ 40%] Building CXX object speech_sample/CMakeFiles/speech_sample.dir/main.cpp.o
[ 43%] Building CXX object classification_sample_async/CMakeFiles/classification_sample_async.dir/main.cpp.o
[ 46%] Building CXX object style_transfer_sample/CMakeFiles/style_transfer_sample.dir/main.cpp.o
[ 50%] Building CXX object benchmark_app/CMakeFiles/benchmark_app.dir/inputs_filling.cpp.o
[ 53%] Building CXX object hello_reshape_ssd/CMakeFiles/hello_reshape_ssd.dir/main.cpp.o
[ 56%] Building CXX object hello_classification/CMakeFiles/hello_classification.dir/main.cpp.o
[ 60%] Building CXX object benchmark_app/CMakeFiles/benchmark_app.dir/main.cpp.o
[ 63%] Linking CXX executable ../intel64/Release/hello_query_device
[ 63%] Built target hello_query_device
[ 66%] Building CXX object benchmark_app/CMakeFiles/benchmark_app.dir/statistics_report.cpp.o
[ 70%] Linking CXX executable ../intel64/Release/classification_sample_async
[ 70%] Built target classification_sample_async
[ 73%] Building CXX object benchmark_app/CMakeFiles/benchmark_app.dir/utils.cpp.o
[ 76%] Linking CXX executable ../intel64/Release/hello_reshape_ssd
[ 80%] Linking CXX executable ../intel64/Release/hello_nv12_input_classification
[ 80%] Built target hello_nv12_input_classification
[ 80%] Built target hello_reshape_ssd
Scanning dependencies of target object_detection_sample_ssd
[ 83%] Linking CXX executable ../intel64/Release/hello_classification
[ 86%] Building CXX object object_detection_sample_ssd/CMakeFiles/object_detection_sample_ssd.dir/main.cpp.o
[ 90%] Linking CXX executable ../intel64/Release/style_transfer_sample
[ 90%] Built target hello_classification
[ 90%] Built target style_transfer_sample
[ 93%] Linking CXX executable ../intel64/Release/speech_sample
[ 93%] Built target speech_sample
[ 96%] Linking CXX executable ../intel64/Release/object_detection_sample_ssd
[ 96%] Built target object_detection_sample_ssd
[100%] Linking CXX executable ../intel64/Release/benchmark_app
[100%] Built target benchmark_app
Scanning dependencies of target ie_samples
[100%] Built target ie_samples

Build completed, you can find binaries for all samples in the ${HOME}/inference_engine_cpp_samples_build/intel64/Release subfolder.

Installing_additional_packages

${HDDL_INSTALL_DIR}/install_IVAD_VPU_dependencies.sh
$ sudo reboot
$ cd ${HOME}/inference_engine_cpp_samples_build/intel64/Release

Benchmark_FP32_(Before_Integer_Quantization)

$ ./benchmark_app \
  -m ${HOME}/openvino_quantization/results/efficientnet-b0-pytorch_DefaultQuantization/2020-03-28_00-48-49/fp32/efficientnet-b0-pytorch.xml \
  -i ${HOME}/Pictures \
  -d CPU \
  -nthreads 8

Count:      7636 iterations
Duration:   60037.62 ms
Latency:    26.61 ms
Throughput: 127.19 FPS
Peak Virtual Memory (VmPeak) Size, kBytes: 1174064
Peak Resident Memory (VmHWM) Size, kBytes:  276328

Benchmark_Integer_Quantization_(Fake_Quantize)

$ ./benchmark_app \
  -m ${HOME}/openvino_quantization/results/efficientnet-b0-pytorch_DefaultQuantization/2020-03-28_00-48-49/optimized/efficientnet-b0-pytorch.xml \
  -i ${HOME}/Pictures \
  -d CPU \
  -nthreads 8

Count:      3284 iterations
Duration:   60132.62 ms
Latency:    63.86 ms
Throughput: 54.61 FPS
Peak Virtual Memory (VmPeak) Size, kBytes: 1718448
Peak Resident Memory (VmHWM) Size, kBytes:  280536

４．Reference articles

https://docs.openvinotoolkit.org/latest/_docs_Workbench_DG_Install_from_Docker_Hub.html

５．Model structure

５−１． Model structure visualization website

https://lutzroeder.github.io/netron/

５−２． Part of the structure of EfficientNet D0

EfficientNet D0 FP32

EfficientNet D0 INT8