4.3. Key Concepts

4.3.1. Working Mode

Sophon BM168X series chip products cover a variety of product forms from the end to the edge to the hub, and can support two different working modes, corresponding to different product forms. The details are as follows:

BM168X

SoC Mode

PCIe Mode

Independent Operation

Yes, BM168X is the independent host and the algorithm runs on BM168X

The algorithm is deployed on the X86 or ARM host and the inference runs on the PCI card

External IO Mode

Gigabit Ethernet

PCIe Interface

Corresponding Product

Microservers/Modules

PCIe Accelerator Cards

4.3.2. Hardware Memory

Memory is an important concept often involved in debugging BM168X applications. In particular, there are three concepts that need to be distinguished: Global Memory、Host Memory、Device Memory.

  • Global Memory: refers to the off-chip storage DDR of the BM168X, typically 12GB for the BM1684, with a maximum supported customisation of 16GB.

  • Device Memory and Host Memory: Device memory and system memory have different meanings depending on the BM168X product type or mode of operation:

Mode

Soc Mode

PCIE Mode

Products

SM5/SE5

SC5/SC5H/SC5+

Global Memory

4GB A53 dedicated +

4GB TPU dedicated +

4GB VPP/VPU dedicated

4GB TPU dedicated +

4GB VPU dedicated +

4GB VPP/A53 dedicated

Host Memory

On-chip memory for the master Cortex A53

Mainframe memory

Device Memory

Device memory allocated to TPU/VPP/VPU

Physical memory on PCIE boards (Global Memory)

Memory synchronisation is an important and hidden issue that is often encountered in subsequent application debugging. We provide functions for memory synchronisation in both BM-OpenCV and BM-FFmpeg frameworks; BMCV API is only oriented towards device memory operations, so there is no memory synchronisation problem. The input data needs to be prepared on the device memory before calling the BMCV API. we provide interfaces in BMLib for data handling between Host Memory and Global Memory, within Global Memory and between Global Memory of different devices. For more details please refer to BMLib Development Reference Manual and Multimedia Development Reference Manual .

4.3.3. BModel

BModel: a deep neural network model file format for Sophon TPU processors, containing the weight of the target network, TPU instruction stream, etc.

Stage: It supports combining different batch size models of the same network into one BModel. Different batch size inputs of the same network correspond to different stages, and the BMRuntime will automatically select the model of the corresponding stage according to the size of the input shape during inference. It is also possible to combine different networks into one BModel, and to obtain different networks by their names.

Dynamic and Static Compilation： Support the dynamic and static compilation of the model, which can be determined by parameter setting when converting the model. For dynamically compiled BModels, any input shape smaller than the shape set at compile time is supported at Runtime; for statically compiled BModels, only the shape set at compile time is supported at Runtime.

注解

Preference is given to statically compiled models: dynamically compiled models require the involvement of the ARM9 microcontroller within the BM168X to dynamically generate TPU run instructions in real time, based on the actual input shape. As a result, the dynamically compiled model is less efficient than the statically compiled model. If possible, a statically compiled model or a statically compiled model that supports multiple input shapes should be used in preference.

4.3.4. bm_image

 1typedef enum bm_image_format_ext_{
 2    FORMAT_YUV420P,
 3    FORMAT_YUV422P,
 4    FORMAT_YUV444P,
 5    FORMAT_NV12,
 6    FORMAT_NV21,
 7    FORMAT_NV16,
 8    FORMAT_NV61,
 9    FORMAT_RGB_PLANAR,
10    FORMAT_BGR_PLANAR,
11    FORMAT_RGB_PACKED,
12    FORMAT_BGR_PACKED,
13    PORMAT_RGBP_SEPARATE,
14    PORMAT_BGRP_SEPARATE,
15    FORMAT_GRAY,
16    FORMAT_COMPRESSED
17} bm_image_format_ext;
18
19typedef enum bm_image_data_format_ext_{
20    DATA_TYPE_EXT_FLOAT32,
21    DATA_TYPE_EXT_1N_BYTE,
22    DATA_TYPE_EXT_4N_BYTE,
23    DATA_TYPE_EXT_1N_BYTE_SIGNED,
24    DATA_TYPE_EXT_4N_BYTE_SIGNED,
25}bm_image_data_format_ext;
26
27// bm_image structure is defined as follows
28struct bm_image {
29    int width;
30    int height;
31    bm_image_format_ext image_format;
32    bm_data_format_ext data_type;
33    bm_image_private* image_private;
34};

BMCV API is all around bm_image. A bm_image object corresponds to a picture. Users build bm_image objects through bm_image_create, attach to device memory, and call bm_image_destroy to destroy after using it; manually attach to device memory also need to manually release; different BMCV APIs support different image_format and data_type, pay attention to the format requirements, if necessary, format conversion; for more details, please refer to BMCV Development Reference Manual .

Bm_image is encapsulated as BMImage in the SAIL library. For more information, please see SAIL User Development Manual .