4. API Reference

4.1. Handle

【Syntax】

typedef void *cvitdl_handle_t;
typedef void *cvitdl_service_handle_t;

【Description】

In TDL SDK, each module has its own handle, but when creating the cvitdl_service_handle_t module, the cvitdl_handle_t is required as an input.

4.2. CVI_TDL_Core

4.2.1. Common

4.2.1.1. CVI_TDL_CreateHandle

【Syntax】

CVI_S32 CVI_TDL_CreateHandle(cvitdl_handle_t *handle);

【Description】

Create handle to use TDL SDK. The TDL SDK automatically creates a VPSS Group.

【Parameter】

Data Type

Parameter

Description

Input/Output

cvitdl_handle_t*

handle

Input handle pointer

4.2.1.2. CVI_TDL_CreateHandle2

【Syntax】

CVI_S32 CVI_TDL_CreateHandle2(cvitdl_handle_t *handle, const VPSS_GRP vpssGroupId, const CVI_U8 vpssDev);

【Description】

Create a handle to use the TDL SDK and use the specified VPSS Group ID and Dev ID to create a VPSS.

【Parameter】

Data Type

Parameter

Description

Output

cvitdl_handle_t*

handle

Input handle pointer

Input

VPSS_GRP

vpssGroupId

Group ID used by VPSS

Input

CVI_U8

vpssDev

VPSS Device id

4.2.1.3. CVI_TDL_DestroyHandle

【Syntax】

CVI_S32 CVI_TDL_DestroyHandle(cvitdl_handle_t handle);

【Description】

Destroy the created handle cvitdl_handle_t. Destroy all open models at the same time

【Parameter】

Data Type

Parameter

Description

Input

cvitdl_handle_t

handle

Input handle pointer

4.2.1.4. CVI_TDL_GetModelPath

const char *CVI_TDL_GetModelPath(cvitdl_handle_t handle, CVI_TDL_SUPPORTED_MODEL_E model);

【Description】

Gets the model path of the model that has been set up internally to support. Release the filepath variable after use.

【Parameter】

Data Type

Parameter

Description

Input

cvitdl_handle_t

handle

Input handle pointer

Input

CVI_TDL_SUPPORTED_MODEL_E

model

Model ID

【Output】

Data Type

Description

Output

char*

Model path pointer

4.2.1.5. CVI_TDL_OpenModel

【Syntax】

CVI_S32 CVI_TDL_OpenModel(cvitdl_handle_t handle, CVI_TDL_SUPPORTED_MODEL_E model, const char *filepath);

【Description】

Enable and initialize the model.

【Parameter】

Data Type

Parameter

Description

Input

cvitdl_handle_t

handle

Input handle pointer

Input

CVI_TDL_SUPPORTED_MODEL_E

model

Model index

Input

const char*

filepath

cvimodel model path

4.2.1.6. CVI_TDL_SetSkipVpssPreprocess

【Syntax】

CVI_S32 CVI_TDL_SetSkipVpssPreprocess(cvitdl_handle_t handle, CVI_TDL_SUPPORTED_MODEL_E model, bool skip);

【Description】

Disable preprocessing for a specified model.

By default, TDL SDK uses the internally created VPSS to preprocess the model (skip = false).

When skip is set to true, TDL SDK will not preprocess the model.

The model input must be preprocessed externally before being input to the model.

This is usually used when directly binding VPSS to VI and using only a single model.

CVI_TDL_GetVpssChnConfig can be used to obtain the VPSS preprocessing parameters for the model.

【Parameter】

Data Type

Parameter

Description

Input

cvitdl_handle_t

handle

Handel

Input

CVI_TDL_SUPPORTED_MODEL_E

model

Model ID

Input

bool

skip

Whether to skip pre-processing

4.2.1.7. CVI_TDL_GetSkipVpssPreprocess

【Syntax】

CVI_S32 CVI_TDL_GetSkipVpssPreprocess(cvitdl_handle_t handle, CVI_TDL_SUPPORTED_MODEL_E model, bool *skip);

【Description】

Inquire whether the model will be preprocessed within TDL SDK.

【Parameter】

Data Type

Parameter

Description

Input

cvitdl_handle_t

handle

Handel

Input

CVI_TDL_SUPPORTED_MODEL_E

model

Model ID

Output

bool*

skip

Inquire whether the model will be preprocessed within TDL SDK.

4.2.1.8. CVI_TDL_SetVpssThread

【Syntax】

CVI_S32 CVI_TDL_SetVpssThread(cvitdl_handle_t handle, CVI_TDL_SUPPORTED_MODEL_E model, const uint32_t thread);

【Description】

Set the thread ID used by a specific model. In TDL SDK, a VPSS thread represents a set of VPSS group settings. Thread 0 is used by default for the VPSS group used by the model. When multiple threads use the same TDL SDK handle for model inference, this API must be used to specify different VPSS threads to avoid race conditions.

【Parameter】

Data Type

Parameter

Description

Input

cvitdl_handle_t

handle

Handel

Input

CVI_TDL_SUPPORTED_MODEL_E

model

Model ID

Input

uint32_t

thread

Thread ID

4.2.1.9. CVI_TDL_SetVpssThread2

【Syntax】

CVI_S32 CVI_TDL_SetVpssThread2(cvitdl_handle_t handle, CVI_TDL_SUPPORTED_MODEL_E model, const uint32_t thread, const VPSS_GRP vpssGroupId, const CVI_U8 dev);

【Description】

Same as CVI_TDL_SetVpssThread. You can specify the Vpss Group ID.

【Parameter】

Data Type

Parameter

Description

Input

cvitdl_handle_t

handle

Handel

Input

CVI_TDL_SUPPORTED_MODEL_E

model

Model ID

Input

uint32_t

thread

Thread ID

Input

VPSS_GRP

vpssGroupId

VPSS Group id

Input

const CVI_U8

dev

VPSS Device id

4.2.1.10. CVI_TDL_GetVpssThread

【Syntax】

CVI_S32 CVI_TDL_GetVpssThread(cvitdl_handle_t handle, CVI_TDL_SUPPORTED_MODEL_E model, uint32_t *thread);

【Description】

Gets the thread id used by the model.

【Parameter】

Data Type

Parameter

Description

Input

cvitdl_handle_t

handle

Handel

Input

CVI_TDL_SUPPORTED_MODEL_E

model

Model ID

Output

uint32_t*

thread

VPSS thread ID

4.2.1.11. CVI_TDL_GetVpssGrpIds

【Syntax】

CVI_S32 CVI_TDL_GetVpssGrpIds(cvitdl_handle_t handle, VPSS_GRP **groups, uint32_t *num);

【Description】

Get all the VPSS group IDs used in the handle. After use, the groups must be released manually.

【Parameter】

Data Type

Parameter

Description

Input

cvitdl_handle_t

handle

Handel

Output

VPSS_GRP **

groups

Reference to a null pointer

Output

uint32_t*

num

Length of groups

4.2.1.12. CVI_TDL_SetVpssTimeout

【Syntax】

CVI_S32 CVI_TDL_SetVpssTimeout(cvitdl_handle_t handle, uint32_t timeout);

【Description】

Set the TDL SDK to wait for VPSS hardware timeout, with a default setting of 100ms. This setting applies to all VPSS threads within TDL SDK.

【Parameter】

Data Type

Parameter

Description

Input

cvitdl_handle_t

handle

Handel

Input

uint32_t

timeout

timeout

4.2.1.13. CVI_TDL_SetVBPool

【Syntax】

CVI_S32 CVI_TDL_SetVBPool(cvitdl_handle_t handle, uint32_t thread, VB_POOL pool_id);

【Description】

Specify a VBPool for the internal VPSS in TDL SDK. After being specified, the internal VPSS in TDL SDK will directly obtain memory from this pool. If this API is not used to specify a pool, the system will allocate one automatically.

【Parameter】

Data Type

Parameter

Description

Input

cvitdl_handle_t

handle

Handle

Input

uint32_t

thread

VPSS thread ID

Input

VB_POOL

pool_id

VB Pool Id. If INVALID_POOLID is set, the Pool is not specified and is automatically allocated by the system.

4.2.1.14. CVI_TDL_GetVBPool

【Syntax】

CVI_S32 CVI_TDL_GetVBPool(cvitdl_handle_t handle, uint32_t thread, VB_POOL *pool_id);

【Description】

Get the VBPool ID used by the specified VPSS. If CVI_TDL_SetVBPool is not used to specify a pool, INVALID_POOLID will be returned.

【Parameter】

Data Type

Parameter

Description

Input

cvitdl_handle_t

handle

Handle

Input

uint32_t

thread

VPSS thread ID

Output

VB_POOL*

pool_id

The current VB Pool Id is used.

4.2.1.15. CVI_TDL_CloseAllModel

【Syntax】

CVI_S32 CVI_TDL_CloseAllModel(cvitdl_handle_t handle);

【Description】

Unload all models that have been loaded in the handle.

【Parameter】

Data Type

Parameter

Description

Input

cvitdl_handle_t

handle

Handle

4.2.1.16. CVI_TDL_CloseModel

【Syntax】

CVI_S32 CVI_TDL_CloseModel(cvitdl_handle_t handle, CVI_TDL_SUPPORTED_MODEL_E model);

【Description】

Dismount the specific model that has been loaded in the handle.

【Parameter】

Data Type

Parameter

Description

Input

cvitdl_handle_t

handle

Handle

Input

CVI_TDL_SUPPORTED_MODEL_E

model

Model index

4.2.1.17. CVI_TDL_Dequantize

【Syntax】

CVI_S32 CVI_TDL_Dequantize(const int8_t *quantizedData, float *data, const uint32_t bufferSize, const float dequantizeThreshold);

【Description】

Dequantize int8 values to float.

【Parameter】

Data Type

Parameter

Description

Input

const int8_t*

quantizedData

Int8 data

Output

float*

data

Float output data

Input

const uint32_t

bufferSize

Int8 data quantity

Input

const float

dequantizeThreshold

Quantization threshold

4.2.1.18. CVI_TDL_ObjectNMS

【Syntax】

CVI_S32 CVI_TDL_ObjectNMS(const cvtdl_object_t *obj, cvtdl_object_t *objNMS, const float threshold, const char method);

【Description】

Run the Non-Maximum Suppression algorithm for bboxes in cvtdl_object_t.

【Parameter】

Data Type

Parameter

Description

Input

const cvtdl_object_t*

obj

Object Meta of the NMS

Output

cvtdl_object_t*

objNMS

Result after NMS

Input

const float

threshold

IOU threshold

Input

const char

method

‘u’: Intersection over Union

‘m’: Intersection over min area

4.2.1.19. CVI_TDL_FaceNMS

【Syntax】

CVI_S32 CVI_TDL_ObjectNMS(const cvtdl_face_t *face, cvtdl_face_t *faceNMS, const float threshold, const char method);

【Description】

Run the Non-Maximum Suppression algorithm for bboxes in cvtdl_face_t.

【Parameter】

Data Type

Parameter

Description

Input

const cvtdl_face_t*

face

face meta of the NMS

Output

cvtdl_face_t*

faceNMS

Result after NMS

Input

const float

threshold

IOU threshold

Input

const char

method

‘u’: Intersection over Union

‘m’: Intersection over min area

4.2.1.20. CVI_TDL_FaceAlignment

【Syntax】

CVI_S32 CVI_TDL_FaceAlignment(VIDEO_FRAME_INFO_S *inFrame, const uint32_t metaWidth, const uint32_t metaHeight, const cvtdl_face_info_t *info, VIDEO_FRAME_INFO_S *outFrame, const bool enableGDC);

【Description】

InsightFace Alignment parameter was used to perform Face Alignment for inFrame image Face.

【Parameter】

Data Type

Parameter

Description

Input

VIDEO_FRAME_INFO_S*

inFrame

Input image

Input

const uint32_t metaWidth

metaWidth

The width of the frame in Info

Input

const uint32_t metaHeight

metaHeight

The height of the frame in Info

Input

const cvtdl_face_info_t*

info

Face info

Output

VIDEO_FRAME_INFO_S*

outFrame

The face image after face alignment.

Input

const bool

enableGDC

Whether to use GDC hardware

4.2.1.21. CVI_TDL_CropImage

【Syntax】

CVI_S32 CVI_TDL_CropImage(VIDEO_FRAME_INFO_S *srcFrame, cvtdl_image_t *dst, cvtdl_bbox_t *bbox, bool cvtRGB888);

【Description】

Retrieves the Bbox-specified region image from the srcFrame image.

【Parameter】

Data Type

Parameter

Description

Input

VIDEO_FRAME_INFO_S*

srcFrame

Input image, currently only supports RGB packed format.

Output

cvtdl_image_t*

dst

Output image.

Input

cvtdl_bbox_t*

bbox

Bounding box

Input

bool

cvtRGB888

Whether to convert to RGB888 format for output.

4.2.1.22. CVI_TDL_CropImage_Face

【Syntax】

CVI_S32 CVI_TDL_CropImage_Face(VIDEO_FRAME_INFO_S *srcFrame, cvtdl_image_t *dst, cvtdl_face_info_t *face_info, bool align, bool cvtRGB888);

【Description】

Extract the image within the specified face bbox from the srcFrame image.

【Parameter】

Data Type

Parameter

Description

Input

VIDEO_FRAME_INFO_S*

srcFrame

Input image, currently only supports RGB packed format.

Output

cvtdl_image_t*

dst

Output image.

Input

cvtdl_face_info_t*

face_info

Specified face info.

Input

bool

align

Whether to perform face alignment. Using InsightFace alignment parameters.

Input

bool

cvtRGB888

Whether to convert the output format to RGB888.

4.2.1.23. CVI_TDL_SoftMax

【Syntax】

CVI_S32 CVI_TDL_SoftMax(const float *inputBuffer, float *outputBuffer, const uint32_t bufferSize);

【Description】

Compute Softmax for the inputBuffer.

【Parameter】

Data Type

Parameter

Description

Input

const float*

inputBuffer

Buffer to be softmaxed

Output

const float*

outputBuffer

resulting softmax buffer

Input

const uint32_t

bufferSize

buffer size

4.2.1.24. CVI_TDL_GetVpssChnConfig

【Syntax】

CVI_S32 CVI_TDL_GetVpssChnConfig(cvitdl_handle_t handle, CVI_TDL_SUPPORTED_MODEL_E model, const CVI_U32 frameWidth, const CVI_U32 frameHeight, const CVI_U32 idx, cvtdl_vpssconfig_t *chnConfig);

【Description】

Get the VPSS parameters used in model pre-processing.

【Parameter】

Data Type

Parameter

Description

Input

cvitdl_handle_t

handle

Handle

Input

CVI_TDL_SUPPORTED_MODEL_E

model

Model ID

Input

CVI_U32

frameWidth

Input image width

Input

CVI_U32

fra meHeight

Input image height

Input

CVI_U32

idx

The input index of the model

Output

cvtdl_vpssconfig_t*

chnConfig

The specified parameter value to be returned.

4.2.1.25. CVI_TDL_Free

CVI_TDL_Free(X)

【Description】

Free the data structure generated by the model results. Some data structures may contain sub-items allocated by malloc, so they need to be released.

【Parameter】

The following are the supported input variables:

4.2.1.26. CVI_TDL_CopyInfo

CVI_TDL_CopyInfo(IN, OUT)

【Description】

Generic API for copying CVI_TDL structures. This API will allocate the pointer memory for the copied structure and perform a complete copy.

【Parameter】

Data Type

Parameter

Description

Input

Support type:

cvtdl_face_info_t

cvtdl_object_info_t

cvtdl_image_t

IN

Replication source

Output

Support type:

cvtdl_face_info_t

cvtdl_object_info_t

cvtdl_image_t

OUT

Replication purpose

4.2.1.27. CVI_TDL_RescaleMetaCenter

【Description】

Rescale the coordinates in the structure to the size of the input image with padding on top, bottom, left, and right.

【Parameter】

Here are the supported input variables:

4.2.1.28. CVI_TDL_RescaleMetaRB

【Description】

In the padding image, the coordinates in the structure are restored to the same size as the input image.

【Parameter】

Here are the supported input variables:

4.2.1.29. getFeatureTypeSize

int getFeatureTypeSize(feature_type_e type);

【Description】

Gets the unit size of the eigenvalue.

【Parameter】

Data Type

Parameter

Description

Input

feature_type_e

type

Unit

【Output】

Data Type

Parameter

Description

Output

int

X

The unit size is byte

4.2.1.30. CVI_TDL_SetModelThreshold

【Syntax】

CVI_S32 CVI_TDL_SetModelThreshold(cvitdl_handle_t handle, CVI_TDL_SUPPORTED_MODEL_E model, float threshold);

【Description】

Set the threshold value for the model, currently only supported for models of the detection type.

【Parameter】

Data Type

Parameter

Description

Input

cvitdl_handle_t

handle

Handle

Input

CVI_TDL_SUPPORTED_MODEL_E

model

Model index

Input

float

threshold

Threshold (0.0~1.0)

4.2.1.31. CVI_TDL_GetModelThreshold

【Syntax】

CVI_S32 CVI_TDL_GetModelThreshold(cvitdl_handle_t handle, CVI_TDL_SUPPORTED_MODEL_E model, float *threshold);

【Description】

Take out the model threshold, and only the Detection type model is supported at present.

【Parameter】

Data Type

Parameter

Description

Input

cvitdl_handle_t

handle

Handle

Input

CVI_TDL_SUPPORTED_MODEL_E

model

Model index

Output

float*

threshold

Threshold

4.2.2. Object Detection

4.2.2.1. CVI_TDL_Detection

【Syntax】

CVI_S32 CVI_TDL_Detection(const cvitdl_handle_t handle, VIDEO_FRAME_INFO_S *frame, CVI_TDL_SUPPORTED_MODEL_E model_id, cvtdl_object_t *obj)

【Description】

Use the specified model ID to call the corresponding target detection algorithm.

【Parameter】

Data Type

Parameter

Description

Input

cvitdl_handle_t

handle

Handle

Input

VIDEO_FRAME_INFO_S*

frame

Input image

Input

CVI_TDL_SUPPORTED_MODEL_E

model_id

model id

Output

cvtdl_object_t*

obj

Detected objects

【Optional model ID list】

model_id

description

CVI_TDL_SUPPORTED_MODEL_YOLOV3

YoloV3

CVI_TDL_SUPPORTED_MODEL_YOLOV5

YoloV5

CVI_TDL_SUPPORTED_MODEL_YOLOV6

YoloV6

CVI_TDL_SUPPORTED_MODEL_YOLOV7

YoloV7

CVI_TDL_SUPPORTED_MODEL_YOLOV8

_DETECTION

YoloV8

CVI_TDL_SUPPORTED_MODEL_YOLOX

Yolo X

CVI_TDL_SUPPORTED_MODEL_PPYOLOE

PP-YoloE

CVI_TDL_SUPPORTED_MODEL_YOLOV10

_DETECTION

YoloV10

CVI_TDL_SUPPORTED_MODEL_YOLOV8

_HARDHAT

YOLOV8 Ciffic detection

CVI_TDL_SUPPORTED_MODEL_HAND

_DETECTION

hand detection

CVI_TDL_SUPPORTED_MODEL_PERSON

_PETS_DETECTION

Person, cat, dog target detection

CVI_TDL_SUPPORTED_MODEL_PERSON

_VEHICLE_DETECTION

Person and car target detection

CVI_TDL_SUPPORTED_MODEL_HAND

_FACE_PERSON_DETECTION

Person, face, hand target detection

CVI_TDL_SUPPORTED_MODEL_HEAD

_PERSON_DETECTION

Person, face, head target detection

CVI_TDL_SUPPORTED_MODEL_MOBILEDETV2

_COCO80

Mobiledet model trained on the COCO80 dataset

CVI_TDL_SUPPORTED_MODEL_MOBILEDETV2

_VEHICLE

MOBILEDET model of vehicle detection

CVI_TDL_SUPPORTED_MODEL_MOBILEDETV2

_PEDESTRIAN

Pedestrian detection MobileDET model

4.2.3. Face detection

4.2.3.1. CVI_TDL_FaceDetection

【Syntax】

CVI_S32 CVI_TDL_FaceDetection(const cvitdl_handle_t handle, VIDEO_FRAME_INFO_S *frame, CVI_TDL_SUPPORTED_MODEL_E model_id, cvtdl_face_t *face_meta)

【Description】

Use the specified model to detect the face.

【Parameter】

Data Type

Parameter

Description

Input

cvitdl_handle_t

handle

Handle

Input

VIDEO_FRAME_INFO_S*

frame

Input image

Input

CVI_TDL_SUPPORTED_MODEL_E

model_id

model id

Output

cvtdl_face_t*

faces

The faces we detected

【Optional model ID list】

model_id

description

CVI_TDL_SUPPORTED_MODEL_RETINAFACE

Face detection (RetinaFace), Caffe reasoning framework

CVI_TDL_SUPPORTED_MODEL_SCRFDFACE

ScrFDFace

CVI_TDL_SUPPORTED_MODEL_RETINAFACE

_IR

Infrared face detection (Reti-naFace),PYTORCH reasoning framework

CVI_TDL_SUPPORTED_MODEL

_FACEMASKDETECTION

RetinafaceYolox,Detect the face of the mask.Face score stored in “faces->info[i].bbox.score”,Wearing mask face score stored in “faces->info[i].mask_score”。

4.2.3.2. CVI_TDL_FaceLandmarkerDet2

【Syntax】

CVI_S32 CVI_TDL_FaceLandmarkerDet2(cvitdl_handle_t handle, VIDEO_FRAME_INFO_S *frame, cvtdl_face_t *faces);

【Description】

Evaluate the face punctuation of the person in the Faces structure.

【Parameter】

Data Type

Parameter

Description

Input

cvitdl_handle_t

handle

Handle

Input

VIDEO_FRAME_INFO_S*

frame

The faces we detected

Output

cvtdl_face_t*

faces

faces punctuation

4.2.3.3. CVI_TDL_FaceQuality

【Syntax】

CVI_S32 CVI_TDL_FaceQuality(cvitdl_handle_t handle, VIDEO_FRAME_INFO_S *frame, cvtdl_face_t *faces, bool *skip);

【Description】

Evaluate the quality of the faces in the given “faces” structure and detect their angles. The quality is affected by the clarity of the face and whether it is occluded. The quality score of a face is stored in “faces->info[i].face_quality”, while the angle is stored in “faces->info[i].head_pose”.

【Parameter】

Data Type

Parameter

Description

Input

cvitdl_handle_t

handle

handle

Input

VIDEO_FRAME_INFO_S*

frame

Input image

Input

cvtdl_face_t*

face

The faces we detected

Input

bool*

skip

Bool array: Specifies the face quality required for the face. NULL indicates that all faces do this.

4.2.3.4. CVI_TDL_MaskClassification

【Syntax】

CVI_S32 CVI_TDL_MaskClassification(cvitdl_handle_t handle, VIDEO_FRAME_INFO_S *frame, cvtdl_face_t *face);

【Description】

Check if all faces in the input faces are masked faces. A face detection must be performed before calling this interface. The score of masked face is stored in faces->info[i].mask_score.

【Parameter】

Data Type

Parameter

Description

Input

cvitdl_handle_t

handle

handle

Input

VIDEO_FRAME_INFO_S*

frame

Input image

Input

cvtdl_face_t*

faces

The faces we detected

4.2.4. Face Recognition.

4.2.4.1. CVI_TDL_FaceRecognition

【Syntax】

CVI_S32 CVI_TDL_FaceRecognition(cvitdl_handle_t handle, VIDEO_FRAME_INFO_S *frame, cvtdl_face_t *faces);

【Description】

Extract facial features. This interface will extract features for all faces in the face variable and store them in faces->info[i].feature.

【Parameter】

Data Type

Parameter

Description

Input

cvitdl_handle_t

handle

handle

Input

VIDEO_FRAME_INFO_S*

frame

Input image

Input/Output

cvtdl_face_t*

faces

The faces we detected

4.2.4.2. CVI_TDL_FaceAttribute_cls

【Syntax】

CVI_S32 CVI_TDL_FaceAttribute_cls(cvitdl_handle_t handle, VIDEO_FRAME_INFO_S *frame, cvtdl_face_t *faces);

【Description】

Face attribute classification. This interface will classify the attributes of the face in the picture.

Face attributes include: gender, age, whether to wear glasses and whether to wear a mask, the results are placed infaces->info[i].gender_score, faces->info[i].age, faces->info[i].glass, faces->info[i].mask_score。

【Parameter】

Data Type

Parameter

Description

Input

cvitdl_handle_t

handle

handle

Input

VIDEO_FRAME_INFO_S*

frame

input image

Input/Output

cvtdl_face_t*

faces

face attribute

4.2.5. Gesture Recognition

4.2.5.1. CVI_TDL_HandClassification

【Syntax】

CVI_S32 CVI_TDL_HandClassification(const cvitdl_handle_t handle, VIDEO_FRAME_INFO_S *frame, cvtdl_object_t *meta);

【Description】

This gesture classification algorithm is used for recognizing gestures on the specified ‘frame’. The results will be stored in ‘meta->info[i].name’ and ‘meta->info[i].bbox.score’.

【Parameter】

Data Type

Parameter

Description

Input

cvitdl_handle_t

handle

The handle for the TDL model.

Input

VIDEO_FRAME_INFO_S*

frame

The input image.

Input/Output

cvtdl_object_t*

meta

Detected hand bounding boxes and gesture classifications.

4.2.5.2. CVI_TDL_HandKeypoint

【Syntax】

CVI_S32 CVI_TDL_HandKeypoint(const cvitdl_handle_t handle, VIDEO_FRAME_INFO_S *frame, cvtdl_handpose21_meta_ts *meta);

【Description】

This function is used for outputting hand key points and storing them in ‘meta->info[i]’.

【Parameter】

Data Type

Parameter

Description

Input

cvitdl_handle_t

handle

The handle for the TDL model.

Input

VIDEO_FRAME_INFO_S*

frame

Input image.

Input/Output

cvtdl_handpose21_meta_ts*

meta

Detected hand bounding boxes and 21 hand key points.

4.2.5.3. CVI_TDL_HandKeypointClassification

【Syntax】

CVI_S32 CVI_TDL_HandKeypointClassification(const cvitdl_handle_t handle, VIDEO_FRAME_INFO_S *frame, cvtdl_handpose21_meta_t *meta);

【Description】

This function is used for extracting hand keypoint information. The keypoint information will be stored in ‘meta->info[i]’.

【Parameter】

Data Type

Parameter

Description

Input

cvitdl_handle_t

handle

The handle for the TDL model.

Input

VIDEO_FRAME_INFO_S*

frame

Input 21 key points of the hand. The x and y coordinates are sequentially stored in ‘frame->stVFrame.pu8VirAddr[0]’, ‘frame->stVFrame.u32Height=1’, ‘frame->stVFrame.u32Width=42*sizeof(float)’.

Output

cvtdl_handpose21_meta_t*

meta

The output of hand gestures, including the gesture label in ‘meta->label’ and the gesture score in ‘meta->score’.

4.2.6. Object Tracking

4.2.6.1. CVI_TDL_DeepSORT_Init

【Syntax】

CVI_S32 CVI_TDL_DeepSORT_Init(const cvitdl_handle_t handle, bool use_specific_counter);

【Description】

Initialize the DeepSORT algorithm.

【Parameter】

Data Type

Parameter

Description

Input

cvitdl_handle_t

handle

handle

Input

bool

use_specific_counter

Whether each object class is assigned its own id

4.2.6.2. CVI_TDL_DeepSORT_GetDefaultConfig

【Syntax】

CVI_S32 CVI_TDL_DeepSORT_GetDefaultConfig(cvtdl_deepsort_config_t *ds_conf);

【Description】

Get the default parameters for DeepSORT.

【Parameter】

Data Type

Parameter

Description

Input

cvtdl_deepsort_config_t*

ds_conf

DeepSORT parameter

4.2.6.3. CVI_TDL_DeepSORT_SetConfig

【Syntax】

CVI_S32 CVI_TDL_DeepSORT_SetConfig(const cvitdl_handle_t handle , cvtdl_deepsort_config_t *ds_conf, int cvi_tdl_obj_type, bool show_config);

【Description】

Setting DeepSORT parameters.

【Parameter】

Data Type

Parameter

Description

Input

cvitdl_handle_t

handle

Handle

Input

cvtdl_deepsort_config_t*

ds_conf

DeepSORT parameter

Input

int

cvi_tdl_obj_type

“-1” indicates default settings. Non-negative values indicate setting parameters for cvi_tdl_obj_type.

Input

bool

show_config

Display Settings.

4.2.6.4. CVI_TDL_DeepSORT_GetConfig

【Syntax】

CVI_S32 CVI_TDL_DeepSORT_GetConfig(const cvitdl_handle_t handle , cvtdl_deepsort_config_t *ds_conf, int cvi_tdl_obj_type);

【Description】

To inquire about the DeepSORT parameters that have been set, please call the corresponding function or method provided by the DeepSORT implementation being used in your system. The specific method or function name may vary depending on the DeepSORT implementation you are using.

【Parameter】

Data Type

Parameter

Description

Input

cvitdl_handle_t

handle

TDL SDK handle

Output

cvtdl_deepsort_config_t*

ds_conf

DeepSORT parameter

Input

int

cvi_tdl_obj_type

-1 indicates that the default parameter is obtained. Non-1 values represent parameters set against the category of cvi_tdl_obj_type

4.2.6.5. CVI_TDL_DeepSORT_CleanCounter

【Syntax】

CVI_S32 CVI_TDL_DeepSORT_CleanCounter(const cvitdl_handle_t handle);

【Description】

Reset the ID counter recorded in DeepSORT.

【Parameter】

Data Type

Parameter

Description

Input

cvitdl_handle_t

handle

Handle

4.2.6.6. CVI_TDL_DeepSORT_Obj

【Syntax】

CVI_S32 CVI_TDL_DeepSORT_Obj(const cvitdl_handle_t handle, cvtdl_object_t *obj, cvtdl_tracker_t *tracker, bool use_reid);

【Description】

Track objects and update the tracker status. This interface will assign a unique ID to each Object, which can be obtained from obj->info[i].unique_id. The tracker_t will record the tracking status of each object and the current predicted Bounding Box. If you want to use object appearance features for tracking, set use_reid to true, and use CVI_TDL_OSNet for feature extraction before tracking. Currently, feature extraction only supports human-shaped objects.

【Parameter】

Data Type

Parameter

Description

Input

cvitdl_handle_t

handle

handle

Input

cvtdl_object_t*

obj

The object to be tracked

Output

cvtdl_tracker_t*

tracker

The trace state of the object

Input

bool

use_reid

Whether object appearance characteristics are used for tracing

4.2.6.7. CVI_TDL_DeepSORT_Face

【Syntax】

CVI_S32 CVI_TDL_DeepSORT_Face(const cvitdl_handle_t handle, cvtdl_face_t *face, cvtdl_tracker_t *tracker, bool use_reid);

【Description】

Track faces and update the Tracker state. This function assigns a unique ID to each face, which can be retrieved from face->info[i].unique_id. The tracker_t will record the tracking status and the predicted bounding box of each face. If you want to use face features for tracking, set use_reid to true and call CVI_TDL_FaceRecognition to compute face features before tracking.

【Parameter】

Data Type

Parameter

Description

Input

cvitdl_handle_t

handle

handle

Input

cvtdl_face_t*

face

The face you want to track

Output

cvtdl_tracker_t*

tracker

The tracking state of a face

Input

bool

use_reid

Whether to use appearance characteristics for tracing. Currently, only false can be set

4.2.7. Motion Detection

4.2.7.1. CVI_TDL_Set_MotionDetection_Background

【Syntax】

CVI_S32 CVI_TDL_Set_MotionDetection_Background(const cvitdl_handle_t handle, VIDEO_FRAME_INFO_S *frame);

【Description】

Set the background for Motion Detection. The first time this function is called, it will initialize the Motion Detection module. Subsequent calls will only update the background. The Motion Detection module in TDL SDK uses frame difference method.

【Parameter】

Data Type

Parameter

Description

Input

cvitdl_handle_t

handle

Handle

Input

VIDEO_FRAME_INFO_S*

frame

Background

4.2.7.2. CVI_TDL_MotionDetection

【Syntax】

CVI_S32 CVI_TDL_MotionDetection(const cvitdl_handle_t handle, VIDEO_FRAME_INFO_S *frame, cvtdl_object_t *objects, uint8_t threshold, double min_area);

【Description】

Detect objects using frame difference method. The detection results will be stored in “objects”.

【Parameter】

Data Type

Parameter

Description

Input

cvitdl_handle_t

handle

handle

Input

VIDEO_FRAME_INFO_S*

frame

image

Output

cvtdl_object_t*

object

Motion detection result

Input

uint8_t

threshold

The frame difference threshold must be 0-255

Input

double

min_area

Minimum object area (Pixels). Filters out objects smaller than this value.

4.2.7.3. CVI_TDL_Set_MotionDetection_ROI

【Syntax】

CVI_S32 CVI_TDL_Set_MotionDetection_ROI(const cvitdl_handle_t handle, MDROI_t *roi_s);

【Description】

This function is used for object detection using frame difference method. The detection results will be stored in ‘objects’.

【Parameter】

Data Type

Parameter

Description

Input

cvitdl_handle_t

handle

The handle for the TDL model.

Input

MDROI_t*

roi_s

Set the region of interest for motion detection.

4.2.8. License Plate Recognition

4.2.8.1. CVI_TDL_LicensePlateDetection

【Syntax】

CVI_S32 CVI_TDL_LicensePlateDetection(cvitdl_handle_t handle, VIDEO_FRAME_INFO_S *frame, cvtdl_object_t *vehicle_meta);

【Description】

License plate detection. Before calling this API, vehicle detection must be performed first. This algorithm will perform license plate detection on the already detected objects. The location of the license plate will be stored in obj->info[i].vehicle_properity->license_pts.

【Parameter】

Data Type

Parameter

Description

Input

cvitdl_handle_t

handle

handle

Input

VIDEO_FRAME_INFO_S*

frame

image

Input

cvtdl_object_t*

obj

Object (vehicle) detection result

4.2.8.2. CVI_TDL_LicensePlateRecognition_TW

CVI_S32 CVI_TDL_LicensePlateRecognition_TW(const cvitdl_handle_t handle, VIDEO_FRAME_INFO_S *frame, cvtdl_object_t *obj);

【Description】

Perform license plate recognition (Taiwan) on all vehicles in the input obj. Prior to calling this API, CVI_TDL_LicensePlateDetection must be called once for license plate detection. The license plate number is stored in obj->info[i].vehicle_properity->license_char.

【Parameter】

Data Type

Parameter

Description

Input

cvitdl_handle_t

handle

handle

Input

VIDEO_FRAME_INFO_S*

frame

The input image

Input

cvtdl_object_t*

obj

License plate detection results

4.2.8.3. CVI_TDL_LicensePlateRecognition_CN

CVI_S32 CVI_TDL_LicensePlateRecognition_CN(const cvitdl_handle_t handle, VIDEO_FRAME_INFO_S *frame, cvtdl_object_t *obj);

【Description】

Perform license plate recognition (Mainland China) for all vehicles in the input obj. CVI_TDL_LicensePlateDetection must be called before calling this API for license plate detection. The license plate number is stored in obj->info[i].vehicle_properity->license_char.

【Parameter】

Data Type

Parameter

Description

Input

cvitdl_handle_t

handle

handle

Input

VIDEO_FRAME_INFO_S*

frame

image

Input/Output

cvtdl_object_t*

obj

icense plate detection results

4.2.9. Liveness Detection

4.2.9.1. CVI_TDL_IrLiveness

【Syntax】

CVI_S32 CVI_TDL_IrLiveness(const cvitdl_handle_t handle, VIDEO_FRAME_INFO_S *irFrame, cvtdl_face_t *ir_faces);

【Description】

This function performs IR (infrared) monocular liveness detection.

It determines whether the detected faces in ‘ir_faces’ are real or not.

The liveness scores for each face will be stored in ‘ir_faces->info[i].liveness_score’.

【Parameter】

Data Type

Parameter

Description

Input

cvitdl_handle_t

handle

Handle

Input

VIDEO_FRAME_INFO_S*

irFrame

The ir image

Input/Output

cvtdl_face_t*

ir_faces

Detected IR face/ liveness score

4.2.10. Pose Estimation

4.2.10.1. CVI_TDL_PoseDetection

【Syntax】

CVI_S32 CVI_TDL_PoseDetection(const cvitdl_handle_t handle, VIDEO_FRAME_INFO_S *frame, CVI_TDL_SUPPORTED_MODEL_E model_id, cvtdl_object_t *obj_meta)

【Description】

Use the model ID to call the attitude detection algorithm to detect 17 key points of the human body.

【Parameter】

Data Type

Parameter

Description

Input

cvitdl_handle_t

handle

handle

Input

VIDEO_FRAME_INFO_S*

frame

input image

Input

CVI_TDL_SUPPORTED_MODEL_E

model_id

model id

Output

cvtdl_object_t*

obj_meta

The detected objects and key points

【Optional model ID list】

id

name

description

34

CVI_TDL_SUPPORTED_MODEL_YOLOV8POSE

Optional model ID list

35

CVI_TDL_SUPPORTED_MODEL_SIMCC_POSE

simcc model

4.2.11. Semantic Segmentation

4.2.11.1. CVI_TDL_YoloV8_Seg

【Syntax】

CVI_S32 CVI_TDL_YoloV8_Seg(const cvitdl_handle_t handle, VIDEO_FRAME_INFO_S *frame, cvtdl_object_t *obj_meta);

【Description】

Use the YOLOV8 model for example segmentation.

【Parameter】

Data Type

Parameter

Description

Input

cvitdl_handle_t

handle

handle

Input

VIDEO_FRAME_INFO_S*

frame

input image

Output

cvtdl_object_t*

obj_meta

The detected object and boundary point

4.2.11.2. CVI_TDL_Topformer_Seg

【Syntax】

CVI_S32 CVI_TDL_Topformer_Seg(const cvitdl_handle_t handle, VIDEO_FRAME_INFO_S *frame, cvtdl_seg_t *seg_ann);

【Description】

Use the TopFormer model for semantic segmentation.

【Parameter】

Data Type

Parameter

Description

Input

cvitdl_handle_t

handle

handle

Input

VIDEO_FRAME_INFO_S*

frame

input image

Output

cvtdl_seg_t *

seg_ann

the id of input image

4.2.12. Fall Detection

4.2.12.1. CVI_TDL_Fall_Monitor

【Syntax】

CVI_S32 CVI_TDL_Fall_Monitor(cvitdl_handle_t handle, cvtdl_object_t *obj);

【Description】

To judge the fall state, you need to call the attitude detection model before use.

【Parameter】

Data Type

Parameter

Description

Input

cvitdl_handle_t

handle

handle

Output

cvtdl_object_t*

obj

the result of fall detection

4.2.13. Driver Fatigue Detection

4.2.13.1. CVI_TDL_FaceLandmarker

【Syntax】

CVI_S32 CVI_TDL_FaceLandmarker(cvitdl_handle_t handle, VIDEO_FRAME_INFO_S *frame, cvtdl_face_t *faces);

【Description】

Before using this API, the face detection API should be called to obtain the detection result of 106 facial landmarks, which should be placed in face->dms[i].landmarks_106 and update the 5 facial landmarks in face->dms[i].landmarks_5.

【Parameter】

Data Type

Parameter

Description

Input

cvitdl_handle_t

handle

handle

Input

VIDEO_FRAME_INFO_S*

frame

Input image

Input

cvtdl_face_t*

face

Human face

4.2.13.2. CVI_TDL_EyeClassification

【Syntax】

CVI_S32 CVI_TDL_EyeClassification (cvitdl_handle_t handle, VIDEO_FRAME_INFO_S *frame, cvtdl_face_t *faces);

【Description】

The closed state of the eyes is determined based on the results of face detection and facial landmark detection. The scores of the right and left eyes are stored in face->dms[i].reye_score and face->dms[i].leye_score, respectively.

【Parameter】

Data Type

Parameter

Description

Input

cvitdl_handle_t

handle

handle

Input

VIDEO_FRAME_INFO_S*

frame

Input image

Input

cvtdl_face_t*

face

Human face

4.2.13.3. CVI_TDL_YawnClassification

【Syntax】

CVI_S32 CVI_TDL_YawnClassification (cvitdl_handle_t handle, VIDEO_FRAME_INFO_S *frame, cvtdl_face_t *faces);

【Description】

Detect yawning based on the results of face detection and facial landmark detection. CVI_FaceRecognition must be called first to obtain the results of face detection and facial landmark detection. The yawning score will be placed in face->dms[i].yawn_score, ranging from 0.0 to 1.0.

【Parameter】

Data Type

Parameter

Description

Input

cvitdl_handle_t

handle

handle

Input

VIDEO_FRAME_INFO_S*

frame

Input image

Input

cvtdl_face_t*

face

Human face

4.2.14. Sound Classification

4.2.14.1. CVI_TDL_SoundClassification

【Syntax】

CVI_S32 CVI_TDL_SoundClassification(cvitdl_handle_t handle, VIDEO_FRAME_INFO_S *frame, int *index);

【Description】

Classify the audio in the frame into different categories and output the scores of each category in sorted order.

【Parameter】

Data Type

Parameter

Description

Input

cvitdl_handle_t

handle

handle

Input

VIDEO_FRAME_INFO_S*

frame

Input image

Input

int*

index

Scores in each category

4.2.14.2. CVI_TDL_Get_SoundClassification_ClassesNum

【Syntax】

CVI_S32 CVI_TDL_Get_SoundClassification_ClassesNum(cvitdl_handle_t handle);

【Description】

Get the number of audio categories.

【Parameter】

Data Type

Parameter

Description

Input

cvitdl_handle_t

handle

Handle

【Output】

Data Type

Description

Output

int

The number of category

4.2.14.3. CVI_TDL_Set_SoundClassification_Threshold

【Syntax】

CVI_S32 CVI_TDL_Set_SoundClassification_Threshold(cvitdl_handle_t handle, const float th);

【Description】

Set the audio category threshold.

【Parameter】

Data Type

Parameter

Description

Input

cvitdl_handle_t

handle

Handle

Input

const float

th

Similarity threshold, the similarity higher than this threshold will be taken out.

4.3. CVI_TDL_Service

4.3.1. Common

4.3.1.1. CVI_TDL_Service_CreateHandle

【Syntax】

CVI_S32 CVI_TDL_Service_CreateHandle(cvitdl_service_handle_t *handle, cvtdl_handle tdl_handle);

【Description】

Create a Service handle

【Parameter】

Data Type

Parameter

Description

Input

cvitdl_service_handle_t*

handle

handle

Input

cvitdl_handle_t

tdl_handle

cvi_tdl_core handle

4.3.1.2. CVI_TDL_Service_DestroyHandle

【Syntax】

CVI_S32 CVI_TDL_Service_DestroyHandle(cvitdl_service_handle_t *handle);

【Description】

Destroy the Service handle

【Parameter】

Data Type

Parameter

Description

Input

cvitdl_service_handle_t*

handle

Handle

4.3.1.3. CVI_TDL_Service_Polygon_SetTarget

【Syntax】

CVI_S32 CVI_TDL_Service_Polygon_SetTarget(cvitdl_service_handle_t handle, const cvtdl_pts_t *pts);

【Description】

Set up a region of interest for intrusion detection with pts as the coordinates of the convex polygon. The order of the points should be clockwise or counterclockwise. Call CVI_TDL_Service_Polygon_Intersect to check if a bounding box intersects with the defined region.

【Parameter】

Data Type

Parameter

Description

Input

cvitdl_service_handle_t

handle

handle

Input

cvtdl_pts_t*

pts

Convex polygon point

4.3.1.4. CVI_TDL_Service_Polygon_Intersect

【Syntax】

CVI_S32 CVI_TDL_Service_Polygon_Intersect(cvitdl_service_handle_t handle, const cvtdl_bbox_t *bbox, bool *has_intersect);

【Description】

Detect if a given bounding box is within the targeted intrusion area previously set by calling CVI_TDL_Service_Polygon_SetTarget.

【Parameter】

Data Type

Parameter

Description

Input

cvitdl_service_handle_t

handle

handle

Input

cvtdl_bbox_t*

bbox

Bounding box

Output

bool*

ha s_intersect

Intrusion or not

4.3.1.5. CVI_TDL_Service_RegisterFeatureArray

【Syntax】

CVI_S32 CVI_TDL_Service_RegisterFeatureArray(cvitdl_service_handle_t handle, const cvtdl_service_feature_array_t featureArray, const cvtdl_service_feature_matching_e method);

【Description】

Register a feature database by precomputing and loading the features in the featureArray into memory.

【Parameter】

Data Type

Parameter

Description

Input

cvitdl_service_handle_t

handle

handle

Input

const cvtdl_service_feature_array_t

featureArray

Feature array structure

Input

const cvtdl_service_feature_matching_e

method

Currently, only COS_SIMILARITY is supported

4.3.1.6. CVI_TDL_Service_CalculateSimilarity

【Syntax】

CVI_S32 CVI_TDL_Service_CalculateSimilarity(cvitdl_service_handle_t handle, const cvtdl_feature_t *feature_rhs, const cvtdl_feature_t *feature_lhs, float *score);

【Description】

Compute the cosine similarity between two feature vectors using the RISC-V. The cosine similarity is calculated using the following formula:

\[sim(\theta) = \frac{A \bullet B}{\left\| A \right\| \bullet \left\| B \right\|} = \frac{\sum_{i = 1}^{n}{A_{i}B_{i}}}{\sqrt{\sum_{i = 1}^{n}A_{i}^{2}}\sqrt{\sum_{i = 1}^{n}B_{i}^{2}}}\]

The Cosine Similarity between two features of length n is calculated using the following formula. Currently, only INT8 features are supported.

【Parameter】

Data Type

Parameter

Description

Input

cvitdl_service_handle_t

handle

handle

Input

const cvtdl_feature_t*

feature_rhs

The first feature

Input

const cvtdl_feature_t*

feature_lhs

Second feature

Output

float*

score

Similarity

4.3.1.7. CVI_TDL_Service_ObjectInfoMatching

【Syntax】

CVI_S32 CVI_TDL_Service_ObjectInfoMatching(cvitdl_service_handle_t handle, const cvtdl_object_info_t *object_info, const uint32_t topk, float threshold, uint32_t *indices, float *sims, uint32_t *size);

【Description】

Compute the cosine similarity between the object feature in object_info and the pre-calculated object features in the registered object feature library. Retrieve the top-K similarities that are greater than threshold. The calculation formula is as follows:

\[sim(\theta) = \frac{A \bullet B}{\left\| A \right\| \bullet \left\| B \right\|} = \frac{\sum_{i = 1}^{n}{A_{i}B_{i}}}{\sqrt{\sum_{i = 1}^{n}A_{i}^{2}}\sqrt{\sum_{i = 1}^{n}B_{i}^{2}}}\]

Calculate the Cosine Similarity between the object feature in object_info and the registered object feature in the feature library.

The top-K similarities greater than the given threshold will be returned.

If the number of features in the library is less than 1000, RISC-V will be used for calculation, otherwise TPU will be launched for computation.

Feature registration can be done through CVI_TDL_Service_RegisterFeatureArray.

Note that the feature length n must be consistent and INT8 features are currently supported.

【Parameter】

Data Type

Parameter

Description

Input

cvitdl_service_handle_t

handle

handle

Input

const cvtdl_object_info_t*

object_info

Object Info

Input

const uint32_t

topk

Take topk similarity

Output

float

threshold

Similarity threshold, above which the similarity will be taken out

Output

uint32_t*

indices

The Index of the similarity that satisfies the condition in the library

Output

float*

sims

The similarity of the condition

Output

uint32_t*

size

The number of similarities that are finally extracted

4.3.1.8. CVI_TDL_Service_FaceInfoMatching

【Syntax】

CVI_S32 CVI_TDL_Service_FaceInfoMatching(cvitdl_service_handle_t handle, const cvtdl_face_info_t *face_info, const uint32_t topk, float threshold, uint32_t *indices, float *sims, uint32_t *size);

【Description】

Calculate the Cosine Similarity between the face features in face_info and the registered face feature library, and output the top-K similarities that are greater than the given threshold.

The calculation formula is as follows:

\[sim(\theta) = \frac{A \bullet B}{\left\| A \right\| \bullet \left\| B \right\|} = \frac{\sum_{i = 1}^{n}{A_{i}B_{i}}}{\sqrt{\sum_{i = 1}^{n}A_{i}^{2}}\sqrt{\sum_{i = 1}^{n}B_{i}^{2}}}\]

Where n is the feature length.

If the number of features in the library is less than 1000, the calculation will be performed on the RISC-V.

Otherwise, the TPU will be used for calculation.

Feature registration requires calling CVI_TDL_Service_RegisterFeatureArray.

Currently, only INT8 features are supported.

【Parameter】

Data Type

Parameter

Description

Input

cvitdl_service_handle_t

handle

handle

Input

const cvtdl_face_info_t*

face_info

Face info

Input

const uint32_t

topk

Take topk similarity

Output

float

threshold

Similarity threshold, above which the similarity will be taken out

Output

uint32_t*

indices

The Index of the similarity that satisfies the condition in the library

Output

float*

sims

The similarity of the condition

Output

uint32_t*

size

The number of similarities that are finally extracted

4.3.1.9. CVI_TDL_Service_RawMatching

【Syntax】

CVI_S32 CVI_TDL_Service_RawMatching(cvitdl_service_handle_t handle, const void *feature, const feature_type_e type, const uint32_t topk, float threshold, uint32_t *indices, float *scores, uint32_t *size);

【Description】

Compute the Cosine Similarity between a given feature and the registered feature library.

Retrieve the top-K similarities greater than the specified threshold.

The computation is done using the following formula:

\[sim(\theta) = \frac{A \bullet B}{\left\| A \right\| \bullet \left\| B \right\|} = \frac{\sum_{i = 1}^{n}{A_{i}B_{i}}}{\sqrt{\sum_{i = 1}^{n}A_{i}^{2}}\sqrt{\sum_{i = 1}^{n}B_{i}^{2}}}\]

where n is the length of the feature.

If the number of registered feature arrays is less than 1000, the calculation will be performed on RISC-V.

Otherwise, TPU will be launched for calculation.

The feature arrays need to be registered using CVI_TDL_Service_RegisterFeatureArray.

Unlike CVI_TDL_Service_FaceInfoMatching and CVI_TDL_Service_ObjectInfoMatching, this API directly uses the feature array for comparison and does not require cvtdl_face_info_t or cvtdl_object_info_t to be passed in.

This API limits the feature type to be the same as the feature type in the feature arrays.

Currently, only INT8 features are supported.

【Parameter】

Data Type

Parameter

Description

Input

cvitdl_service_handle_t

handle

handle

Input

const void*

feature

Feature array

Input

const feature_type_e

type

Feature type, currently only TYPE_INT8 is supported

Input

const uint32_t

topk

Take topk similarity

Output

float

threshold

Similarity threshold, above which the similarity will be taken out

Output

uint32_t*

indices

The Index of the similarity that satisfies the condition in the library

Output

float*

scores

The similarity of the condition

Output

uint32_t*

size

The number of similarities that are finally extracted

4.3.1.10. CVI_TDL_Service_FaceAngle

【Syntax】

CVI_S32 CVI_TDL_Service_FaceAngle(const cvtdl_pts_t *pts, cvtdl_head_pose_t *hp);

【Description】

Estimate the pose of a single face.

【Parameter】

Data Type

Parameter

Description

Input

cvtdl_pts_t*

pts

Face landmark

Output

cvtdl_head_pose_t*

hp

Face posture

4.3.1.11. CVI_TDL_Service_FaceAngleForAll

【Syntax】

CVI_S32 CVI_TDL_Service_FaceAngleForAll(const cvtdl_face_t *meta);

【Description】

Calculate pose for multiple faces

【Parameter】

Data Type

Parameter

Description

Input

cvtdl_face_t*

meta

Face data

4.3.2. Image Scaling

4.3.2.1. CVI_TDL_Service_FaceDigitalZoom

【Syntax】

CVI_S32 CVI_TDL_Service_FaceDigitalZoom(

cvitdl_service_handle_t handle,

const VIDEO_FRAME_INFO_S *inFrame,

const cvtdl_face_t *meta,

const float face_skip_ratio,

const float trans_ratio,

const float padding_ratio,

VIDEO_FRAME_INFO_S *outFrame);

【Description】

Resizing (zooming in) the detected face.

【Parameter】

Data Type

Parameter

Description

Input

cvitdl_service_handle_t

handle

handle

Input

VIDEO_FRAME_INFO_S*

inFrame

Input image

Input

cvtdl_face_t*

meta

Face data

Input

float

face_skip_ratio

Neglect ratio

Input

float

trans_ratio

Amplification ratio

Input

float

padding_ratio

Expand bounding box ratio

Output

VIDEO_FRAME_INFO_S*

outFrame

Output image

4.3.2.2. CVI_TDL_Service_ObjectDigitalZoom

【Syntax】

CVI_S32 CVI_TDL_Service_ObjectDigitalZoom(cvitdl_service_handle_t handle,

const VIDEO_FRAME_INFO_S *inFrame, const cvtdl_object_t *meta, const float obj_skip_ratio, const float trans_ratio, const float padding_ratio,

VIDEO_FRAME_INFO_S *outFrame);

【Description】

The object detected in the detection result is zoomed in.

【Parameter】

Data Type

Parameter

Description

Input

cvitdl_service_handle_t

handle

handle

Input

const VIDEO_FRAME_INFO_S*

inFrame

Input image

Input

const cvtdl_object_t*

meta

Object data

Input

const float

obj_skip_ratio

Neglect ratio

Input

const float

trans_ratio

Amplification ratio

Input

const float

padding_ratio

Expand bounding box ratio

Output

VIDEO_FRAME_INFO_S*

outFrame

Output image

4.3.2.3. CVI_TDL_Service_ObjectDitgitalZoomExt

【Syntax】

CVI_S32 CVI_TDL_Service_ObjectDigitalZoomExt(cvitdl_service_handle_t handle, const VIDEO_FRAME_INFO_S *inFrame, const cvtdl_object_t *meta,

const float obj_skip_ratio, const float trans_ratio, const float pad_ratio_left, const float pad_ratio_right, const float pad_ratio_top,

const float pad_ratio_bottom, VIDEO_FRAME_INFO_S *outFrame);

【Description】

Zoom in on the object in the detection results

【Parameter】

Data Type

Parameter

Description

Input

cvitdl_service_handle_t

handle

handle

Input

VIDEO_FRAME_INFO_S*

inFrame

Input image

Input

const cvtdl_object_t*

meta

Object data

Input

const float

obj_skip_ratio

Neglect ratio

Input

const float

trans_ratio

Amplification ratio

Input

const float

pad_ratio_left

Expansion rate (left)

Input

const float

pad_ratio_right

Expansion rate (right)

Input

const float

pad_ratio_top

Expansion rate (top)

Input

const float

pad_ratio_bottom

Expansion rate (lower)

Output

VIDEO_FRAME_INFO_S*

outFrame

Output image

4.3.3. Image drawing

4.3.3.1. CVI_TDL_Service_FaceDrawPts

【Syntax】

CVI_S32 CVI_TDL_Service_FaceDrawPts(cvtdl_pts_t *pts, VIDEO_FRAME_INFO_S *frame);

【Description】

Drawing face landmarks.

【Parameter】

Data Type

Parameter

Description

Input

cvtdl_pts_t*

pts

Face landmark

Input

VIDEO_FRAME_INFO_S*

hp

Input/output image

4.3.3.2. CVI_TDL_Service_FaceDrawRect

【Syntax】

CVI_S32 CVI_TDL_Service_FaceDrawRect(cvitdl_service_handle_t handle, const cvtdl_face_t *meta, VIDEO_FRAME_INFO_S *frame, const bool drawText, cvtdl_service_brush_t brush);

【Description】

Drawing a bounding box around a detected face.

【Parameter】

Data Type

Parameter

Description

Input

cvitdl_service_handle_t

handle

handle

Input

cvtdl_face_t*

meta

Face data

Input

VIDEO_FRAME_INFO_S*

frame

Input/output image

Input

bool

drawText

Whether to draw face names

Input

cvtdl_service_brush_t

brush

color

4.3.3.3. CVI_TDL_Service_ObjectDrawPose

【Syntax】

CVI_S32 CVI_TDL_Service_ObjectDrawPose(const cvtdl_object_t *meta, VIDEO_FRAME_INFO_S *frame);

【Description】

Draw 17 skeletal points detected by pose estimation.

【Parameter】

Data Type

Parameter

Description

Input

cvtdl_object_t*

meta

Skeletal points detection result

Input

VIDEO_FRAME_INFO_S*

frame

Input image

4.3.3.4. CVI_TDL_Service_ObjectDrawRect

【Syntax】

CVI_S32 CVI_TDL_Service_ObjectDrawRect(cvitdl_service_handle_t handle, const cvtdl_object_t *meta, VIDEO_FRAME_INFO_S *frame, const bool drawText);

【Description】

Draw bounding box of detected objects

【Parameter】

Data Type

Parameter

Description

Input

cvitdl_service_handle_t

handle

handle

Input

cvtdl_object_t*

meta

Object detection result

Input

VIDEO_FRAME_INFO_S*

frame

Input/output image

Input

const bool

drawText

Whether to draw class text

4.3.3.5. CVI_TDL_Service_ObjectWriteText

【Syntax】

CVI_S32 CVI_TDL_Service_ObjectWriteText(char *name, int x, int y, VIDEO_FRAME_INFO_S *frame, float r, float g, float b);

【Description】

Draw specified text

【Parameter】

Data Type

Parameter

Description

Input

char*

name

Drawn text

Input

int

x

The x coordinate is drawn

Input

int

y

The y coordinate is drawn

Input/Output

IDEO_FRAME_INFO_S*

rame

Input/output image

Input

float

r

Draw the color r channel value

Input

float

g

Draw the color g channel value

Input

float

b

Draw the color b channel value

4.3.3.6. CVI_TDL_Service_Incar_ObjectDrawRect

【Syntax】

CVI_S32 CVI_TDL_Service_Incar_ObjectDrawRect(cvitdl_service_handle_t handle, const cvtdl_dms_od_t *meta, VIDEO_FRAME_INFO_S *frame, const bool drawText, cvtdl_service_brush_t brush);

【Description】

Draw specified text

【Parameter】

Data Type

Parameter

Description

Input

cvitdl_service_handle_t

handle

handle

Input

const cvtdl_dms_od_t*

meta

Object detection result

Input/Output

IDEO_FRAME_INFO_S*

rame

Input/output image

Input

const bool

drawText

Whether to draw class text

Input

cvtdl_service_brush_t

brush

color