TPU-NNTC Quick Start Guide

Legal Statement

All rights reserved © SOPHGO 2022. All rights reserved.

Without the written permission of the Company, neither entity nor individual shall extract or reproduce part or all of the contents in this document or disseminate such contents in any form.

Note

The products, services or features you purchase shall be bound by the commercial contracts and clauses of Sophon. All or part of the products, services or features described herein may not fall within the scope of your purchase or use. Unless otherwise specified in the contract, Sophon makes no representations or warranties, express or implied, with respect to the contents of this document. The contents herein may be updated on a regular basis due to product version upgrades or other reasons. Unless otherwise agreed on, this document can only be used as a reference and all statements, information and suggestions in this document do not constitute any express or implied warranties.

Technical Support

Add

Building 1, IC Park, No. 9 Fenghao East Road, Haidian District, Beijing

Post Code

100094

Website

https://www.sophgo.com/

Email

sales@sophgo.com

Tel

+86-10-57590723 +86-10-57590724

Process Overview

• Complete implanting application process
  • Model compilation quantization
    • FP32 BMODELgeneration
    • Generation of quantization BMODEL
  • Application development & deployment

Complete Example: Classified Application of Resnet18

• Overview
• Compilation and quantization model
  • Initial tpu-nntc environment
  • Compilation of FP32 BMODEL
  • INT8 BMODEL generation by automatic quantization
  • INT8 BMODEL generation by step-based quantization (optional)
    • Generate input data of LMDB
    • Export FP32 UMODEL
    • Quantify FP32 UMODEL as INT8 UMODEL
    • Compile INT8 BMODEL
  • Clearing environment
• Program development & deployment
  • Install libsophon environment
  • Inspect the correctness and performance of BMODEL
  • Application development
    • Initialize equipment and load bmodel
    • Pre-processing
    • Model reasoning
    • Post-processing
    • Release equipment
  • Program compilation script
  • Packaging deployment

Model Performance and Precision Test Process

• Decompress test package
• Prepare dataset
  • ImageNet
  • COCO (optional)
• Run performance and accuracy tests in SOC
• Prepare running environment
• Install the tpu_perf tool
• Prepare tool chain compilation environment
• One example
• Performance test
• Precision verification
  • Prepare quantization and test LMDB
  • Model quantization and compilation
  • Precision test

TFLite Quantized Model Compilation Example

• Overview
• Initial tpu-nntc environment
• Compile BMODEL
  • Get model parameters
  • Compile at the command line
  • Compile with Python script