2. Introduction
2.1. Explanation of Terms
Terminology |
Description |
BM1684X |
Sophgo Technology’s Fourth Generation Tensor Processor for Deep Learning |
BM1684 |
Sophgo Technology’s Third Generation Tensor Processor for Deep Learning |
TPU |
Neural Network Unit in the BM1684/BM1684X Chips |
VPU |
Decoding Unit in BM1684 Chip |
VPP |
Graphics Acceleration Unit in the BM1684/BM1684X Chips |
JPU |
Image jpeg Codec Unit in the BM1684/BM1684X Chips |
SophonSDK |
Sophgo Technology’s Original Deep Learning Development Kit based on BM1684/BM1684X Chips |
PCIe Mode |
A Working Form of BM1684/BM1684X, where the chip is used as an acceleration device and the client algorithm runs on x86 host |
SoC Mode |
A Working Form of BM1684/BM1684X, where the chip itself runs independently as a host and the client algorithm can run directly on it |
arm_pcie Mode |
A Working Form of BM1684/BM1684X, where the board with the chip is plugged into the server of ARM cpu as PCIe from the device and the client algorithm runs on the host of ARM cpu |
BMCompiler |
The Optimized Compiler for Deep Neural Networks developed by the TPU Processor of Sophgo Technology, which can transform various deep neural networks defined by the deep learning framework into instruction streams running on TPU |
BMRuntime |
TPU Inference Interface Library |
BMCV |
Hardware Accelerated Interface Library for Graphical Computing |
BMLib |
A Layer of Underlying Software Libraries wrapped on top of the kernel driver, which can carry out device management, memory management, Data handling, API sending, A53 enable, power control |
UFramework(ufw) |
A Sophgo Technology’s Self-defined Deep Learning Inference Framework based on Caffe, which is used to decouple the model from the original framework in order to verify the model conversion accuracy and quantify |
BMNetC |
BMCompiler Front-end Tool for Caffe |
BMNetD |
BMCompiler Front-end Tool for Darknet |
BMNetM |
BMCompiler Front-end Tool for MxNet |
BMNetO |
BMCompiler Front-end Tool for ONNX |
BMNetP |
BMCompiler Front-end Tool for PyTorch |
BMNetT |
BMCompiler Front-end Tool for TensorFlow |
BMNetU |
BMCompiler Front-end Tool for INT8 Quantitative Model |
BMPaddle |
BMCompiler Front-end Tool for Paddle Paddle |
Umodel |
The Model Format under UFamework defined by the Sophgo Technology, as well as the intermediate model format used when quantifying the model |
BModel |
Deep Neural Network Model File Format for Sophon TPU Processor, including target network weight, TPU instruction flow, etc |
BMLang |
Advanced Programming Model for TPUs. Users can develop without understanding the underlying TPU Hardware information |
TPUKernel |
Development Library based on TPU Atomic Operation(a set of APIs packaged according to the chip’s instruction set), which requires users to be familiar with chip architecture and storage details |
SAIL |
The Sophon Inference Inference Library, supporting the Python/C++ interfaces, which is a further encapsulation of BMCV, BMDecoder, BMLib, BMRuntime, etc |
TPU-MLIR |
TPU Compiler Project, which can transform pre-trained neural networks under different frameworks into bmodel which can operate efficiently on Sophon TPU |
winograd |
An Accelerated Algorithm for Convolution |
2.2. Upgrade Description
There are some differences between the upgraded SophonSDK and the previous BMNNSDK2 and SophonSDK3. The main differences include:
Difference |
Sophon SDK |
SophonSDK3 |
BMNNSDK2 |
Adaptor Chip |
BM1684, BM1684X |
BM1684, BM1684X |
BM1684 |
Release Mode |
Decouple each module and provide tar and deb packages respectively. |
Provides a unified tar development package that includes Compiler and Library |
Provides a unified tar development package that includes Compiler,Library and Examples |
Driver Installation |
Install the sophon-driver_x.y.z_amd64.deb package |
Install using ‘install_driver_{target}.sh’ |
|
NNToolchain, Quantization-Tools |
Provide tpu-nntc_vx.y.z- < hash >-< data > .tar.gz installation |
Included in SophonSDK3 |
Included in BMNNSDK2 |
BMCV,BMRuntime, BMLib |
Provide sophon-libsophon_x.x.x_amd64.deb package, located at /opt/sophon/libsophon-current after installation |
Included in SophonSDK3 |
Included in BMNNSDK2 |
Multimedia Library |
Provide sophon-mw-x.x.x_amd64.deb package,located at /opt/sophon/sophon-ffmpeg-latest,/opt/sophon/sophon-opencv-latest,/opt/sophon/opencv-bmcpu-latest after installation |
Included in SophonSDK3 |
Included in BMNNSDK2 |
pipeline Routine |
Open source to github repository, sophon-pipeline |
||
Algorithm Routine |
Open source to github repository (not released yet) |
${BMNNSDK}/examples |
|
注解
Where x.y.z is the version number
2.3. Precautions
The new Sophon SDK no longer provides the official docker, you can install the library package directly on the host using the deb package, or you can refer to Chapter 6 of the LIBSOPHON User Manual to build your own image.
In PCIE mode, if you have already installed the BM1684 SDK driver, please uninstall the old BM1684 SDK driver before installing the sophon-driver_x.y.z_amd64.deb package. For more information, please refer to Chapter 3 of the LIBSOPHON User Manual.
Sophon-mw-sophon-ffmpeg relies on the sophon-libsophon package, while sophon-mw-sophon-opencv depends on sophon-mw-sophon-ffmpeg, so you must install libsophon first, then sophon-mw-sophon-ffmpeg, and finally sophon-mw-sophon-opencv in the installation order.
It is recommended that you use find_package in cmake to find library packages for libsophon and sophon-mw and link to your own programs.
Some header files are not included in libsophon, such as bm_wrapper.hpp, utils.hpp.
In SOC mode, the library package originally under /system is now located under /opt/sophon/.
In SOC mode, the ko kernel module under /opt/sophon/libsophon-current/data is automatically loaded through the systemd service at boot time. If you have a user-defined self-startup service and rely on BM168X chips, be sure to start after the ko is loaded.
The new TPU-NNTC replaces the original bm_model.bin with tpu_model, using the same method.
2.5. Help and support
If you have any questions or comments or suggestions regarding the use of SophonSDK, please contact the relevant technical support.