Overview

NNTC supports the direct conversion of the TFLite quantization model. Here, the ResNet50 model is used as an example to illustrate the compilation method.

This example model ResNet50.tflite download address:https://disk.sophgo.vip/sharing/ibo9YyxgT.

The TFLite model is compiled using the bmtflite front end in TPU-NNTC.

Initial tpu-nntc environment

Tool chain software can be used in docker environment.The docker of the latest version can be installed by referring to official tutorial.After installation, execute the following script and add the present user into docke group to obtain docker execution authority.

sudo usermod -aG docker $USER
newgrp docker

Download the Compressed package of tpu-nntc from official website and name it according to the following rules tpu-nntc_vx.y.z-<hash>-<date>.tar.gz

mkdir tpu-nntc
# decompress it to tpu-nntc.
tar zxvf tpu-nntc_vx.y.z-<hash>-<date>.tar.gz --strip-components=1 -C tpu-nntc

The docker used by tpu-nntc is sophgo/tpuc_dev:v2.1. The docker mirroring is bound with tpu-nntc. In few cases, tpu-nntc may be updated and a new mirroring is required.

cd tpu-nntc

# If the current system has no corresponding mirroring after entering docker, a mirroring will be downloaded automatically from docker hub.
# Map the upper-level directory of tpu-nntc to the /workspace directory in docker.
# 8001 - 8001 port mapping is used here. It will be used when using ufw visual tool thereafter.
# If port has been occupied, please change other unoccupied one and then make change and adjustment based on needs.
docker run -v $PWD/..:/workspace -p 8001:8001 -it sophgo/tpuc_dev:v2.1

# At this time, it has entered docker and is under /workspace catalogue.
# Initial software environment below.
cd /workspace/tpu-nntc
source scripts/envsetup.sh

Initialization mode of other dockers:

Downloading can be made in advance under an offline environment and then it can be imported into target machine for importing according to the following commands:
# Save mirror on connected machine.
docker pull sophgo/tpuc_dev:v2.1
docker save sophgo/tpuc_dev:v2.1 -o sophon_tpuc_dev.docker

# After copying sophon_tpuc_dev_docker.tar to target machine.
docker load -i sophon_tpuc_dev.docker
In addition, tpu-nntc provides the docker_setup.sh script in the scrtips directory for reference and convenient basic use. In actual use, it can be managed according to the needs. The method of use is as follows:
# Parameters behind docker_setup.sh are used for designating the specific catalogue that will be used as working catalogue. It will be mapped in /workspace.
# Here is the upper level directory of tpu-nntc.
scripts/docker_setup.sh ..

# The following information will be output. If 8001 port is occupied, other unoccupied ports will be chosen automatically
# ...
# OPEN 'http://localhost:8001' IN WEB BROWSER WHEN 'python3 -m ufw.tools.app' RUNNING IN DOCKER

Compile BMODEL

Get model parameters

You can open the ResNet50.tflite file with netron or other software to get the input_names and shapes parameters.

Compile at the command line

# ResNet50.tflite needs to be in the current directory.
# Since the TFLite model is converted, the bmtflite front end is required.
# There is only one input here, if there are multiple, you can separate them with ",".
python3 -m bmtflite --model ResNet50.tflite \
                --input_names "serving_default_input_1:0" \
                --shapes "[1,224,224,3]" \
                --target BM1684X \
                --outdir cmd_bmodel

View the information of the following bmodel:

tpu_model --info cmd_bmodel/compilation.bmodel

The output model information is as follows:

bmodel version: B.2.2
chip: BM1684X
create time: Mon Aug 22 21:12:23 2022

==========================================
net 0: [network]  static
------------
stage 0:
input: serving_default_input_1:0, [1, 224, 224, 3], float32, scale: 1
output: StatefulPartitionedCall:0, [1, 1000], float32, scale: 1

device mem size: 28124160 (coeff: 26162176, instruct: 0, runtime: 1961984)
host mem size: 0 (coeff: 0, runtime: 0)

Compile with Python script

Create a python compilation script compile_resnet50.py, the content is as follows:

import bmtflite as bm

bm.compile(
  model = "ResNet50.tflite",
  input_names = ["serving_default_input_1:0"],
  shapes = [[1,224,224,3]],
  target = "BM1684X",
  outdir = "py_bmodel",
  net_name = "resnet50",
  )

Execute compilation:

# ResNet50.tflite 需要在当前目录下
python3 compile_resnet50.py

View the information of the following bmodel:

tpu_model --info py_bmodel/compilation.bmodel

The output is as follows:

bmodel version: B.2.2
chip: BM1684X
create time: Mon Aug 22 21:19:34 2022

==========================================
net 0: [resnet50]  static
------------
stage 0:
input: serving_default_input_1:0, [1, 224, 224, 3], float32, scale: 1
output: StatefulPartitionedCall:0, [1, 1000], float32, scale: 1

device mem size: 28124160 (coeff: 26162176, instruct: 0, runtime: 1961984)
host mem size: 0 (coeff: 0, runtime: 0)