4. API Reference¶
4.1. Handle¶
【Syntax】
typedef void *cvitdl_handle_t;
typedef void *cvitdl_service_handle_t;
【Description】
In TDL SDK, each module has its own handle, but when creating the cvitdl_service_handle_t module, the cvitdl_handle_t is required as an input.
4.2. CVI_TDL_Core¶
4.2.1. Common¶
4.2.1.1. CVI_TDL_CreateHandle¶
【Syntax】
CVI_S32 CVI_TDL_CreateHandle(cvitdl_handle_t *handle);
【Description】
Create handle to use TDL SDK. The TDL SDK automatically creates a VPSS Group.
【Parameter】
Data Type |
Parameter |
Description |
|
---|---|---|---|
Input/Output |
cvitdl_handle_t* |
handle |
Input handle pointer |
4.2.1.2. CVI_TDL_CreateHandle2¶
【Syntax】
CVI_S32 CVI_TDL_CreateHandle2(cvitdl_handle_t *handle, const VPSS_GRP vpssGroupId, const CVI_U8 vpssDev);
【Description】
Create a handle to use the TDL SDK and use the specified VPSS Group ID and Dev ID to create a VPSS.
【Parameter】
Data Type |
Parameter |
Description |
|
---|---|---|---|
Output |
cvitdl_handle_t* |
handle |
Input handle pointer |
Input |
VPSS_GRP |
vpssGroupId |
Group ID used by VPSS |
Input |
CVI_U8 |
vpssDev |
VPSS Device id |
4.2.1.3. CVI_TDL_DestroyHandle¶
【Syntax】
CVI_S32 CVI_TDL_DestroyHandle(cvitdl_handle_t handle);
【Description】
Destroy the created handle cvitdl_handle_t. Destroy all open models at the same time
【Parameter】
Data Type |
Parameter |
Description |
|
---|---|---|---|
Input |
cvitdl_handle_t |
handle |
Input handle pointer |
4.2.1.4. CVI_TDL_GetModelPath¶
const char *CVI_TDL_GetModelPath(cvitdl_handle_t handle, CVI_TDL_SUPPORTED_MODEL_E model);
【Description】
Gets the model path of the model that has been set up internally to support. Release the filepath variable after use.
【Parameter】
Data Type |
Parameter |
Description |
|
---|---|---|---|
Input |
cvitdl_handle_t |
handle |
Input handle pointer |
Input |
CVI_TDL_SUPPORTED_MODEL_E |
model |
Model ID |
【Output】
Data Type |
Description |
|
---|---|---|
Output |
char* |
Model path pointer |
4.2.1.5. CVI_TDL_OpenModel¶
【Syntax】
CVI_S32 CVI_TDL_OpenModel(cvitdl_handle_t handle, CVI_TDL_SUPPORTED_MODEL_E model, const char *filepath);
【Description】
Enable and initialize the model.
【Parameter】
Data Type |
Parameter |
Description |
|
---|---|---|---|
Input |
cvitdl_handle_t |
handle |
Input handle pointer |
Input |
CVI_TDL_SUPPORTED_MODEL_E |
model |
Model index |
Input |
const char* |
filepath |
cvimodel model path |
4.2.1.6. CVI_TDL_SetSkipVpssPreprocess¶
【Syntax】
CVI_S32 CVI_TDL_SetSkipVpssPreprocess(cvitdl_handle_t handle, CVI_TDL_SUPPORTED_MODEL_E model, bool skip);
【Description】
Disable preprocessing for a specified model.
By default, TDL SDK uses the internally created VPSS to preprocess the model (skip = false).
When skip is set to true, TDL SDK will not preprocess the model.
The model input must be preprocessed externally before being input to the model.
This is usually used when directly binding VPSS to VI and using only a single model.
CVI_TDL_GetVpssChnConfig can be used to obtain the VPSS preprocessing parameters for the model.
【Parameter】
Data Type |
Parameter |
Description |
|
---|---|---|---|
Input |
cvitdl_handle_t |
handle |
Handel |
Input |
CVI_TDL_SUPPORTED_MODEL_E |
model |
Model ID |
Input |
bool |
skip |
Whether to skip pre-processing |
4.2.1.7. CVI_TDL_GetSkipVpssPreprocess¶
【Syntax】
CVI_S32 CVI_TDL_GetSkipVpssPreprocess(cvitdl_handle_t handle, CVI_TDL_SUPPORTED_MODEL_E model, bool *skip);
【Description】
Inquire whether the model will be preprocessed within TDL SDK.
【Parameter】
Data Type |
Parameter |
Description |
|
---|---|---|---|
Input |
cvitdl_handle_t |
handle |
Handel |
Input |
CVI_TDL_SUPPORTED_MODEL_E |
model |
Model ID |
Output |
bool* |
skip |
Inquire whether the model will be preprocessed within TDL SDK. |
4.2.1.8. CVI_TDL_SetVpssThread¶
【Syntax】
CVI_S32 CVI_TDL_SetVpssThread(cvitdl_handle_t handle, CVI_TDL_SUPPORTED_MODEL_E model, const uint32_t thread);
【Description】
Set the thread ID used by a specific model. In TDL SDK, a VPSS thread represents a set of VPSS group settings. Thread 0 is used by default for the VPSS group used by the model. When multiple threads use the same TDL SDK handle for model inference, this API must be used to specify different VPSS threads to avoid race conditions.
【Parameter】
Data Type |
Parameter |
Description |
|
---|---|---|---|
Input |
cvitdl_handle_t |
handle |
Handel |
Input |
CVI_TDL_SUPPORTED_MODEL_E |
model |
Model ID |
Input |
uint32_t |
thread |
Thread ID |
4.2.1.9. CVI_TDL_SetVpssThread2¶
【Syntax】
CVI_S32 CVI_TDL_SetVpssThread2(cvitdl_handle_t handle, CVI_TDL_SUPPORTED_MODEL_E model, const uint32_t thread, const VPSS_GRP vpssGroupId, const CVI_U8 dev);
【Description】
Same as CVI_TDL_SetVpssThread. You can specify the Vpss Group ID.
【Parameter】
Data Type |
Parameter |
Description |
|
---|---|---|---|
Input |
cvitdl_handle_t |
handle |
Handel |
Input |
CVI_TDL_SUPPORTED_MODEL_E |
model |
Model ID |
Input |
uint32_t |
thread |
Thread ID |
Input |
VPSS_GRP |
vpssGroupId |
VPSS Group id |
Input |
const CVI_U8 |
dev |
VPSS Device id |
4.2.1.10. CVI_TDL_GetVpssThread¶
【Syntax】
CVI_S32 CVI_TDL_GetVpssThread(cvitdl_handle_t handle, CVI_TDL_SUPPORTED_MODEL_E model, uint32_t *thread);
【Description】
Gets the thread id used by the model.
【Parameter】
Data Type |
Parameter |
Description |
|
---|---|---|---|
Input |
cvitdl_handle_t |
handle |
Handel |
Input |
CVI_TDL_SUPPORTED_MODEL_E |
model |
Model ID |
Output |
uint32_t* |
thread |
VPSS thread ID |
4.2.1.11. CVI_TDL_GetVpssGrpIds¶
【Syntax】
CVI_S32 CVI_TDL_GetVpssGrpIds(cvitdl_handle_t handle, VPSS_GRP **groups, uint32_t *num);
【Description】
Get all the VPSS group IDs used in the handle. After use, the groups must be released manually.
【Parameter】
Data Type |
Parameter |
Description |
|
---|---|---|---|
Input |
cvitdl_handle_t |
handle |
Handel |
Output |
VPSS_GRP ** |
groups |
Reference to a null pointer |
Output |
uint32_t* |
num |
Length of groups |
4.2.1.12. CVI_TDL_SetVpssTimeout¶
【Syntax】
CVI_S32 CVI_TDL_SetVpssTimeout(cvitdl_handle_t handle, uint32_t timeout);
【Description】
Set the TDL SDK to wait for VPSS hardware timeout, with a default setting of 100ms. This setting applies to all VPSS threads within TDL SDK.
【Parameter】
Data Type |
Parameter |
Description |
|
---|---|---|---|
Input |
cvitdl_handle_t |
handle |
Handel |
Input |
uint32_t |
timeout |
timeout |
4.2.1.13. CVI_TDL_SetVBPool¶
【Syntax】
CVI_S32 CVI_TDL_SetVBPool(cvitdl_handle_t handle, uint32_t thread, VB_POOL pool_id);
【Description】
Specify a VBPool for the internal VPSS in TDL SDK. After being specified, the internal VPSS in TDL SDK will directly obtain memory from this pool. If this API is not used to specify a pool, the system will allocate one automatically.
【Parameter】
Data Type |
Parameter |
Description |
|
---|---|---|---|
Input |
cvitdl_handle_t |
handle |
Handle |
Input |
uint32_t |
thread |
VPSS thread ID |
Input |
VB_POOL |
pool_id |
VB Pool Id. If INVALID_POOLID is set, the Pool is not specified and is automatically allocated by the system. |
4.2.1.14. CVI_TDL_GetVBPool¶
【Syntax】
CVI_S32 CVI_TDL_GetVBPool(cvitdl_handle_t handle, uint32_t thread, VB_POOL *pool_id);
【Description】
Get the VBPool ID used by the specified VPSS. If CVI_TDL_SetVBPool is not used to specify a pool, INVALID_POOLID will be returned.
【Parameter】
Data Type |
Parameter |
Description |
|
---|---|---|---|
Input |
cvitdl_handle_t |
handle |
Handle |
Input |
uint32_t |
thread |
VPSS thread ID |
Output |
VB_POOL* |
pool_id |
The current VB Pool Id is used. |
4.2.1.15. CVI_TDL_CloseAllModel¶
【Syntax】
CVI_S32 CVI_TDL_CloseAllModel(cvitdl_handle_t handle);
【Description】
Unload all models that have been loaded in the handle.
【Parameter】
Data Type |
Parameter |
Description |
|
---|---|---|---|
Input |
cvitdl_handle_t |
handle |
Handle |
4.2.1.16. CVI_TDL_CloseModel¶
【Syntax】
CVI_S32 CVI_TDL_CloseModel(cvitdl_handle_t handle, CVI_TDL_SUPPORTED_MODEL_E model);
【Description】
Dismount the specific model that has been loaded in the handle.
【Parameter】
Data Type |
Parameter |
Description |
|
---|---|---|---|
Input |
cvitdl_handle_t |
handle |
Handle |
Input |
CVI_TDL_SUPPORTED_MODEL_E |
model |
Model index |
4.2.1.17. CVI_TDL_Dequantize¶
【Syntax】
CVI_S32 CVI_TDL_Dequantize(const int8_t *quantizedData, float *data, const uint32_t bufferSize, const float dequantizeThreshold);
【Description】
Dequantize int8 values to float.
【Parameter】
Data Type |
Parameter |
Description |
|
---|---|---|---|
Input |
const int8_t* |
quantizedData |
Int8 data |
Output |
float* |
data |
Float output data |
Input |
const uint32_t |
bufferSize |
Int8 data quantity |
Input |
const float |
dequantizeThreshold |
Quantization threshold |
4.2.1.18. CVI_TDL_ObjectNMS¶
【Syntax】
CVI_S32 CVI_TDL_ObjectNMS(const cvtdl_object_t *obj, cvtdl_object_t *objNMS, const float threshold, const char method);
【Description】
Run the Non-Maximum Suppression algorithm for bboxes in cvtdl_object_t.
【Parameter】
Data Type |
Parameter |
Description |
|
---|---|---|---|
Input |
const cvtdl_object_t* |
obj |
Object Meta of the NMS |
Output |
cvtdl_object_t* |
objNMS |
Result after NMS |
Input |
const float |
threshold |
IOU threshold |
Input |
const char |
method |
‘u’: Intersection over Union ‘m’: Intersection over min area |
4.2.1.19. CVI_TDL_FaceNMS¶
【Syntax】
CVI_S32 CVI_TDL_ObjectNMS(const cvtdl_face_t *face, cvtdl_face_t *faceNMS, const float threshold, const char method);
【Description】
Run the Non-Maximum Suppression algorithm for bboxes in cvtdl_face_t.
【Parameter】
Data Type |
Parameter |
Description |
|
---|---|---|---|
Input |
const cvtdl_face_t* |
face |
face meta of the NMS |
Output |
cvtdl_face_t* |
faceNMS |
Result after NMS |
Input |
const float |
threshold |
IOU threshold |
Input |
const char |
method |
‘u’: Intersection over Union ‘m’: Intersection over min area |
4.2.1.20. CVI_TDL_FaceAlignment¶
【Syntax】
CVI_S32 CVI_TDL_FaceAlignment(VIDEO_FRAME_INFO_S *inFrame, const uint32_t metaWidth, const uint32_t metaHeight, const cvtdl_face_info_t *info, VIDEO_FRAME_INFO_S *outFrame, const bool enableGDC);
【Description】
InsightFace Alignment parameter was used to perform Face Alignment for inFrame image Face.
【Parameter】
Data Type |
Parameter |
Description |
|
---|---|---|---|
Input |
VIDEO_FRAME_INFO_S* |
inFrame |
Input image |
Input |
const uint32_t metaWidth |
metaWidth |
The width of the frame in Info |
Input |
const uint32_t metaHeight |
metaHeight |
The height of the frame in Info |
Input |
const cvtdl_face_info_t* |
info |
Face info |
Output |
VIDEO_FRAME_INFO_S* |
outFrame |
The face image after face alignment. |
Input |
const bool |
enableGDC |
Whether to use GDC hardware |
4.2.1.21. CVI_TDL_CropImage¶
【Syntax】
CVI_S32 CVI_TDL_CropImage(VIDEO_FRAME_INFO_S *srcFrame, cvtdl_image_t *dst, cvtdl_bbox_t *bbox, bool cvtRGB888);
【Description】
Retrieves the Bbox-specified region image from the srcFrame image.
【Parameter】
Data Type |
Parameter |
Description |
|
---|---|---|---|
Input |
VIDEO_FRAME_INFO_S* |
srcFrame |
Input image, currently only supports RGB packed format. |
Output |
cvtdl_image_t* |
dst |
Output image. |
Input |
cvtdl_bbox_t* |
bbox |
Bounding box |
Input |
bool |
cvtRGB888 |
Whether to convert to RGB888 format for output. |
4.2.1.22. CVI_TDL_CropImage_Face¶
【Syntax】
CVI_S32 CVI_TDL_CropImage_Face(VIDEO_FRAME_INFO_S *srcFrame, cvtdl_image_t *dst, cvtdl_face_info_t *face_info, bool align, bool cvtRGB888);
【Description】
Extract the image within the specified face bbox from the srcFrame image.
【Parameter】
Data Type |
Parameter |
Description |
|
---|---|---|---|
Input |
VIDEO_FRAME_INFO_S* |
srcFrame |
Input image, currently only supports RGB packed format. |
Output |
cvtdl_image_t* |
dst |
Output image. |
Input |
cvtdl_face_info_t* |
face_info |
Specified face info. |
Input |
bool |
align |
Whether to perform face alignment. Using InsightFace alignment parameters. |
Input |
bool |
cvtRGB888 |
Whether to convert the output format to RGB888. |
4.2.1.23. CVI_TDL_SoftMax¶
【Syntax】
CVI_S32 CVI_TDL_SoftMax(const float *inputBuffer, float *outputBuffer, const uint32_t bufferSize);
【Description】
Compute Softmax for the inputBuffer.
【Parameter】
Data Type |
Parameter |
Description |
|
---|---|---|---|
Input |
const float* |
inputBuffer |
Buffer to be softmaxed |
Output |
const float* |
outputBuffer |
resulting softmax buffer |
Input |
const uint32_t |
bufferSize |
buffer size |
4.2.1.24. CVI_TDL_GetVpssChnConfig¶
【Syntax】
CVI_S32 CVI_TDL_GetVpssChnConfig(cvitdl_handle_t handle, CVI_TDL_SUPPORTED_MODEL_E model, const CVI_U32 frameWidth, const CVI_U32 frameHeight, const CVI_U32 idx, cvtdl_vpssconfig_t *chnConfig);
【Description】
Get the VPSS parameters used in model pre-processing.
【Parameter】
Data Type |
Parameter |
Description |
|
---|---|---|---|
Input |
cvitdl_handle_t |
handle |
Handle |
Input |
CVI_TDL_SUPPORTED_MODEL_E |
model |
Model ID |
Input |
CVI_U32 |
frameWidth |
Input image width |
Input |
CVI_U32 |
fra meHeight |
Input image height |
Input |
CVI_U32 |
idx |
The input index of the model |
Output |
cvtdl_vpssconfig_t* |
chnConfig |
The specified parameter value to be returned. |
4.2.1.25. CVI_TDL_Free¶
CVI_TDL_Free(X)
【Description】
Free the data structure generated by the model results. Some data structures may contain sub-items allocated by malloc, so they need to be released.
【Parameter】
The following are the supported input variables:
4.2.1.26. CVI_TDL_CopyInfo¶
CVI_TDL_CopyInfo(IN, OUT)
【Description】
Generic API for copying CVI_TDL structures. This API will allocate the pointer memory for the copied structure and perform a complete copy.
【Parameter】
Data Type |
Parameter |
Description |
|
---|---|---|---|
Input |
Support type: cvtdl_face_info_t cvtdl_object_info_t cvtdl_image_t |
IN |
Replication source |
Output |
Support type: cvtdl_face_info_t cvtdl_object_info_t cvtdl_image_t |
OUT |
Replication purpose |
4.2.1.27. CVI_TDL_RescaleMetaCenter¶
【Description】
Rescale the coordinates in the structure to the size of the input image with padding on top, bottom, left, and right.
【Parameter】
Here are the supported input variables:
4.2.1.28. CVI_TDL_RescaleMetaRB¶
【Description】
In the padding image, the coordinates in the structure are restored to the same size as the input image.
【Parameter】
Here are the supported input variables:
4.2.1.29. getFeatureTypeSize¶
int getFeatureTypeSize(feature_type_e type);
【Description】
Gets the unit size of the eigenvalue.
【Parameter】
Data Type |
Parameter |
Description |
|
---|---|---|---|
Input |
feature_type_e |
type |
Unit |
【Output】
Data Type |
Parameter |
Description |
|
---|---|---|---|
Output |
int |
X |
The unit size is byte |
4.2.1.30. CVI_TDL_SetModelThreshold¶
【Syntax】
CVI_S32 CVI_TDL_SetModelThreshold(cvitdl_handle_t handle, CVI_TDL_SUPPORTED_MODEL_E model, float threshold);
【Description】
Set the threshold value for the model, currently only supported for models of the detection type.
【Parameter】
Data Type |
Parameter |
Description |
|
---|---|---|---|
Input |
cvitdl_handle_t |
handle |
Handle |
Input |
CVI_TDL_SUPPORTED_MODEL_E |
model |
Model index |
Input |
float |
threshold |
Threshold (0.0~1.0) |
4.2.1.31. CVI_TDL_GetModelThreshold¶
【Syntax】
CVI_S32 CVI_TDL_GetModelThreshold(cvitdl_handle_t handle, CVI_TDL_SUPPORTED_MODEL_E model, float *threshold);
【Description】
Take out the model threshold, and only the Detection type model is supported at present.
【Parameter】
Data Type |
Parameter |
Description |
|
---|---|---|---|
Input |
cvitdl_handle_t |
handle |
Handle |
Input |
CVI_TDL_SUPPORTED_MODEL_E |
model |
Model index |
Output |
float* |
threshold |
Threshold |
4.2.2. Object Detection¶
4.2.2.1. CVI_TDL_Detection¶
【Syntax】
CVI_S32 CVI_TDL_Detection(const cvitdl_handle_t handle, VIDEO_FRAME_INFO_S *frame, CVI_TDL_SUPPORTED_MODEL_E model_id, cvtdl_object_t *obj)
【Description】
Use the specified model ID to call the corresponding target detection algorithm.
【Parameter】
Data Type |
Parameter |
Description |
|
---|---|---|---|
Input |
cvitdl_handle_t |
handle |
Handle |
Input |
VIDEO_FRAME_INFO_S* |
frame |
Input image |
Input |
CVI_TDL_SUPPORTED_MODEL_E |
model_id |
model id |
Output |
cvtdl_object_t* |
obj |
Detected objects |
【Optional model ID list】
model_id |
description |
---|---|
CVI_TDL_SUPPORTED_MODEL_YOLOV3 |
YoloV3 |
CVI_TDL_SUPPORTED_MODEL_YOLOV5 |
YoloV5 |
CVI_TDL_SUPPORTED_MODEL_YOLOV6 |
YoloV6 |
CVI_TDL_SUPPORTED_MODEL_YOLOV7 |
YoloV7 |
CVI_TDL_SUPPORTED_MODEL_YOLOV8 _DETECTION |
YoloV8 |
CVI_TDL_SUPPORTED_MODEL_YOLOX |
Yolo X |
CVI_TDL_SUPPORTED_MODEL_PPYOLOE |
PP-YoloE |
CVI_TDL_SUPPORTED_MODEL_YOLOV10 _DETECTION |
YoloV10 |
CVI_TDL_SUPPORTED_MODEL_YOLOV8 _HARDHAT |
YOLOV8 Ciffic detection |
CVI_TDL_SUPPORTED_MODEL_HAND _DETECTION |
hand detection |
CVI_TDL_SUPPORTED_MODEL_PERSON _PETS_DETECTION |
Person, cat, dog target detection |
CVI_TDL_SUPPORTED_MODEL_PERSON _VEHICLE_DETECTION |
Person and car target detection |
CVI_TDL_SUPPORTED_MODEL_HAND _FACE_PERSON_DETECTION |
Person, face, hand target detection |
CVI_TDL_SUPPORTED_MODEL_HEAD _PERSON_DETECTION |
Person, face, head target detection |
CVI_TDL_SUPPORTED_MODEL_MOBILEDETV2 _COCO80 |
Mobiledet model trained on the COCO80 dataset |
CVI_TDL_SUPPORTED_MODEL_MOBILEDETV2 _VEHICLE |
MOBILEDET model of vehicle detection |
CVI_TDL_SUPPORTED_MODEL_MOBILEDETV2 _PEDESTRIAN |
Pedestrian detection MobileDET model |
4.2.3. Face detection¶
4.2.3.1. CVI_TDL_FaceDetection¶
【Syntax】
CVI_S32 CVI_TDL_FaceDetection(const cvitdl_handle_t handle, VIDEO_FRAME_INFO_S *frame, CVI_TDL_SUPPORTED_MODEL_E model_id, cvtdl_face_t *face_meta)
【Description】
Use the specified model to detect the face.
【Parameter】
Data Type |
Parameter |
Description |
|
---|---|---|---|
Input |
cvitdl_handle_t |
handle |
Handle |
Input |
VIDEO_FRAME_INFO_S* |
frame |
Input image |
Input |
CVI_TDL_SUPPORTED_MODEL_E |
model_id |
model id |
Output |
cvtdl_face_t* |
faces |
The faces we detected |
【Optional model ID list】
model_id |
description |
---|---|
CVI_TDL_SUPPORTED_MODEL_RETINAFACE |
Face detection (RetinaFace), Caffe reasoning framework |
CVI_TDL_SUPPORTED_MODEL_SCRFDFACE |
ScrFDFace |
CVI_TDL_SUPPORTED_MODEL_RETINAFACE _IR |
Infrared face detection (Reti-naFace),PYTORCH reasoning framework |
CVI_TDL_SUPPORTED_MODEL _FACEMASKDETECTION |
RetinafaceYolox,Detect the face of the mask.Face score stored in “faces->info[i].bbox.score”,Wearing mask face score stored in “faces->info[i].mask_score”。 |
4.2.3.2. CVI_TDL_FaceLandmarkerDet2¶
【Syntax】
CVI_S32 CVI_TDL_FaceLandmarkerDet2(cvitdl_handle_t handle, VIDEO_FRAME_INFO_S *frame, cvtdl_face_t *faces);
【Description】
Evaluate the face punctuation of the person in the Faces structure.
【Parameter】
Data Type |
Parameter |
Description |
|
---|---|---|---|
Input |
cvitdl_handle_t |
handle |
Handle |
Input |
VIDEO_FRAME_INFO_S* |
frame |
The faces we detected |
Output |
cvtdl_face_t* |
faces |
faces punctuation |
4.2.3.3. CVI_TDL_FaceQuality¶
【Syntax】
CVI_S32 CVI_TDL_FaceQuality(cvitdl_handle_t handle, VIDEO_FRAME_INFO_S *frame, cvtdl_face_t *faces, bool *skip);
【Description】
Evaluate the quality of the faces in the given “faces” structure and detect their angles. The quality is affected by the clarity of the face and whether it is occluded. The quality score of a face is stored in “faces->info[i].face_quality”, while the angle is stored in “faces->info[i].head_pose”.
【Parameter】
Data Type |
Parameter |
Description |
|
---|---|---|---|
Input |
cvitdl_handle_t |
handle |
handle |
Input |
VIDEO_FRAME_INFO_S* |
frame |
Input image |
Input |
cvtdl_face_t* |
face |
The faces we detected |
Input |
bool* |
skip |
Bool array: Specifies the face quality required for the face. NULL indicates that all faces do this. |
4.2.3.4. CVI_TDL_MaskClassification¶
【Syntax】
CVI_S32 CVI_TDL_MaskClassification(cvitdl_handle_t handle, VIDEO_FRAME_INFO_S *frame, cvtdl_face_t *face);
【Description】
Check if all faces in the input faces are masked faces. A face detection must be performed before calling this interface. The score of masked face is stored in faces->info[i].mask_score.
【Parameter】
Data Type |
Parameter |
Description |
|
---|---|---|---|
Input |
cvitdl_handle_t |
handle |
handle |
Input |
VIDEO_FRAME_INFO_S* |
frame |
Input image |
Input |
cvtdl_face_t* |
faces |
The faces we detected |
4.2.4. Face Recognition.¶
4.2.4.1. CVI_TDL_FaceRecognition¶
【Syntax】
CVI_S32 CVI_TDL_FaceRecognition(cvitdl_handle_t handle, VIDEO_FRAME_INFO_S *frame, cvtdl_face_t *faces);
【Description】
Extract facial features. This interface will extract features for all faces in the face variable and store them in faces->info[i].feature.
【Parameter】
Data Type |
Parameter |
Description |
|
---|---|---|---|
Input |
cvitdl_handle_t |
handle |
handle |
Input |
VIDEO_FRAME_INFO_S* |
frame |
Input image |
Input/Output |
cvtdl_face_t* |
faces |
The faces we detected |
4.2.4.2. CVI_TDL_FaceAttribute_cls¶
【Syntax】
CVI_S32 CVI_TDL_FaceAttribute_cls(cvitdl_handle_t handle, VIDEO_FRAME_INFO_S *frame, cvtdl_face_t *faces);
【Description】
Face attribute classification. This interface will classify the attributes of the face in the picture.
Face attributes include: gender, age, whether to wear glasses and whether to wear a mask, the results are placed infaces->info[i].gender_score, faces->info[i].age, faces->info[i].glass, faces->info[i].mask_score。
【Parameter】
Data Type |
Parameter |
Description |
|
---|---|---|---|
Input |
cvitdl_handle_t |
handle |
handle |
Input |
VIDEO_FRAME_INFO_S* |
frame |
input image |
Input/Output |
cvtdl_face_t* |
faces |
face attribute |
4.2.5. Gesture Recognition¶
4.2.5.1. CVI_TDL_HandClassification¶
【Syntax】
CVI_S32 CVI_TDL_HandClassification(const cvitdl_handle_t handle, VIDEO_FRAME_INFO_S *frame, cvtdl_object_t *meta);
【Description】
This gesture classification algorithm is used for recognizing gestures on the specified ‘frame’. The results will be stored in ‘meta->info[i].name’ and ‘meta->info[i].bbox.score’.
【Parameter】
Data Type |
Parameter |
Description |
|
---|---|---|---|
Input |
cvitdl_handle_t |
handle |
The handle for the TDL model. |
Input |
VIDEO_FRAME_INFO_S* |
frame |
The input image. |
Input/Output |
cvtdl_object_t* |
meta |
Detected hand bounding boxes and gesture classifications. |
4.2.5.2. CVI_TDL_HandKeypoint¶
【Syntax】
CVI_S32 CVI_TDL_HandKeypoint(const cvitdl_handle_t handle, VIDEO_FRAME_INFO_S *frame, cvtdl_handpose21_meta_ts *meta);
【Description】
This function is used for outputting hand key points and storing them in ‘meta->info[i]’.
【Parameter】
Data Type |
Parameter |
Description |
|
---|---|---|---|
Input |
cvitdl_handle_t |
handle |
The handle for the TDL model. |
Input |
VIDEO_FRAME_INFO_S* |
frame |
Input image. |
Input/Output |
cvtdl_handpose21_meta_ts* |
meta |
Detected hand bounding boxes and 21 hand key points. |
4.2.5.3. CVI_TDL_HandKeypointClassification¶
【Syntax】
CVI_S32 CVI_TDL_HandKeypointClassification(const cvitdl_handle_t handle, VIDEO_FRAME_INFO_S *frame, cvtdl_handpose21_meta_t *meta);
【Description】
This function is used for extracting hand keypoint information. The keypoint information will be stored in ‘meta->info[i]’.
【Parameter】
Data Type |
Parameter |
Description |
|
---|---|---|---|
Input |
cvitdl_handle_t |
handle |
The handle for the TDL model. |
Input |
VIDEO_FRAME_INFO_S* |
frame |
Input 21 key points of the hand. The x and y coordinates are sequentially stored in ‘frame->stVFrame.pu8VirAddr[0]’, ‘frame->stVFrame.u32Height=1’, ‘frame->stVFrame.u32Width=42*sizeof(float)’. |
Output |
cvtdl_handpose21_meta_t* |
meta |
The output of hand gestures, including the gesture label in ‘meta->label’ and the gesture score in ‘meta->score’. |
4.2.6. Object Tracking¶
4.2.6.1. CVI_TDL_DeepSORT_Init¶
【Syntax】
CVI_S32 CVI_TDL_DeepSORT_Init(const cvitdl_handle_t handle, bool use_specific_counter);
【Description】
Initialize the DeepSORT algorithm.
【Parameter】
Data Type |
Parameter |
Description |
|
---|---|---|---|
Input |
cvitdl_handle_t |
handle |
handle |
Input |
bool |
use_specific_counter |
Whether each object class is assigned its own id |
4.2.6.2. CVI_TDL_DeepSORT_GetDefaultConfig¶
【Syntax】
CVI_S32 CVI_TDL_DeepSORT_GetDefaultConfig(cvtdl_deepsort_config_t *ds_conf);
【Description】
Get the default parameters for DeepSORT.
【Parameter】
Data Type |
Parameter |
Description |
|
---|---|---|---|
Input |
cvtdl_deepsort_config_t* |
ds_conf |
DeepSORT parameter |
4.2.6.3. CVI_TDL_DeepSORT_SetConfig¶
【Syntax】
CVI_S32 CVI_TDL_DeepSORT_SetConfig(const cvitdl_handle_t handle , cvtdl_deepsort_config_t *ds_conf, int cvi_tdl_obj_type, bool show_config);
【Description】
Setting DeepSORT parameters.
【Parameter】
Data Type |
Parameter |
Description |
|
---|---|---|---|
Input |
cvitdl_handle_t |
handle |
Handle |
Input |
cvtdl_deepsort_config_t* |
ds_conf |
DeepSORT parameter |
Input |
int |
cvi_tdl_obj_type |
“-1” indicates default settings. Non-negative values indicate setting parameters for cvi_tdl_obj_type. |
Input |
bool |
show_config |
Display Settings. |
4.2.6.4. CVI_TDL_DeepSORT_GetConfig¶
【Syntax】
CVI_S32 CVI_TDL_DeepSORT_GetConfig(const cvitdl_handle_t handle , cvtdl_deepsort_config_t *ds_conf, int cvi_tdl_obj_type);
【Description】
To inquire about the DeepSORT parameters that have been set, please call the corresponding function or method provided by the DeepSORT implementation being used in your system. The specific method or function name may vary depending on the DeepSORT implementation you are using.
【Parameter】
Data Type |
Parameter |
Description |
|
---|---|---|---|
Input |
cvitdl_handle_t |
handle |
TDL SDK handle |
Output |
cvtdl_deepsort_config_t* |
ds_conf |
DeepSORT parameter |
Input |
int |
cvi_tdl_obj_type |
-1 indicates that the default parameter is obtained. Non-1 values represent parameters set against the category of cvi_tdl_obj_type |
4.2.6.5. CVI_TDL_DeepSORT_CleanCounter¶
【Syntax】
CVI_S32 CVI_TDL_DeepSORT_CleanCounter(const cvitdl_handle_t handle);
【Description】
Reset the ID counter recorded in DeepSORT.
【Parameter】
Data Type |
Parameter |
Description |
|
---|---|---|---|
Input |
cvitdl_handle_t |
handle |
Handle |
4.2.6.6. CVI_TDL_DeepSORT_Obj¶
【Syntax】
CVI_S32 CVI_TDL_DeepSORT_Obj(const cvitdl_handle_t handle, cvtdl_object_t *obj, cvtdl_tracker_t *tracker, bool use_reid);
【Description】
Track objects and update the tracker status. This interface will assign a unique ID to each Object, which can be obtained from obj->info[i].unique_id. The tracker_t will record the tracking status of each object and the current predicted Bounding Box. If you want to use object appearance features for tracking, set use_reid to true, and use CVI_TDL_OSNet for feature extraction before tracking. Currently, feature extraction only supports human-shaped objects.
【Parameter】
Data Type |
Parameter |
Description |
|
---|---|---|---|
Input |
cvitdl_handle_t |
handle |
handle |
Input |
cvtdl_object_t* |
obj |
The object to be tracked |
Output |
cvtdl_tracker_t* |
tracker |
The trace state of the object |
Input |
bool |
use_reid |
Whether object appearance characteristics are used for tracing |
4.2.6.7. CVI_TDL_DeepSORT_Face¶
【Syntax】
CVI_S32 CVI_TDL_DeepSORT_Face(const cvitdl_handle_t handle, cvtdl_face_t *face, cvtdl_tracker_t *tracker, bool use_reid);
【Description】
Track faces and update the Tracker state. This function assigns a unique ID to each face, which can be retrieved from face->info[i].unique_id. The tracker_t will record the tracking status and the predicted bounding box of each face. If you want to use face features for tracking, set use_reid to true and call CVI_TDL_FaceRecognition to compute face features before tracking.
【Parameter】
Data Type |
Parameter |
Description |
|
---|---|---|---|
Input |
cvitdl_handle_t |
handle |
handle |
Input |
cvtdl_face_t* |
face |
The face you want to track |
Output |
cvtdl_tracker_t* |
tracker |
The tracking state of a face |
Input |
bool |
use_reid |
Whether to use appearance characteristics for tracing. Currently, only false can be set |
4.2.7. Motion Detection¶
4.2.7.1. CVI_TDL_Set_MotionDetection_Background¶
【Syntax】
CVI_S32 CVI_TDL_Set_MotionDetection_Background(const cvitdl_handle_t handle, VIDEO_FRAME_INFO_S *frame);
【Description】
Set the background for Motion Detection. The first time this function is called, it will initialize the Motion Detection module. Subsequent calls will only update the background. The Motion Detection module in TDL SDK uses frame difference method.
【Parameter】
Data Type |
Parameter |
Description |
|
---|---|---|---|
Input |
cvitdl_handle_t |
handle |
Handle |
Input |
VIDEO_FRAME_INFO_S* |
frame |
Background |
4.2.7.2. CVI_TDL_MotionDetection¶
【Syntax】
CVI_S32 CVI_TDL_MotionDetection(const cvitdl_handle_t handle, VIDEO_FRAME_INFO_S *frame, cvtdl_object_t *objects, uint8_t threshold, double min_area);
【Description】
Detect objects using frame difference method. The detection results will be stored in “objects”.
【Parameter】
Data Type |
Parameter |
Description |
|
---|---|---|---|
Input |
cvitdl_handle_t |
handle |
handle |
Input |
VIDEO_FRAME_INFO_S* |
frame |
image |
Output |
cvtdl_object_t* |
object |
Motion detection result |
Input |
uint8_t |
threshold |
The frame difference threshold must be 0-255 |
Input |
double |
min_area |
Minimum object area (Pixels). Filters out objects smaller than this value. |
4.2.7.3. CVI_TDL_Set_MotionDetection_ROI¶
【Syntax】
CVI_S32 CVI_TDL_Set_MotionDetection_ROI(const cvitdl_handle_t handle, MDROI_t *roi_s);
【Description】
This function is used for object detection using frame difference method. The detection results will be stored in ‘objects’.
【Parameter】
Data Type |
Parameter |
Description |
|
---|---|---|---|
Input |
cvitdl_handle_t |
handle |
The handle for the TDL model. |
Input |
MDROI_t* |
roi_s |
Set the region of interest for motion detection. |
4.2.8. License Plate Recognition¶
4.2.8.1. CVI_TDL_LicensePlateDetection¶
【Syntax】
CVI_S32 CVI_TDL_LicensePlateDetection(cvitdl_handle_t handle, VIDEO_FRAME_INFO_S *frame, cvtdl_object_t *vehicle_meta);
【Description】
License plate detection. Before calling this API, vehicle detection must be performed first. This algorithm will perform license plate detection on the already detected objects. The location of the license plate will be stored in obj->info[i].vehicle_properity->license_pts.
【Parameter】
Data Type |
Parameter |
Description |
|
---|---|---|---|
Input |
cvitdl_handle_t |
handle |
handle |
Input |
VIDEO_FRAME_INFO_S* |
frame |
image |
Input |
cvtdl_object_t* |
obj |
Object (vehicle) detection result |
4.2.8.2. CVI_TDL_LicensePlateRecognition_TW¶
CVI_S32 CVI_TDL_LicensePlateRecognition_TW(const cvitdl_handle_t handle, VIDEO_FRAME_INFO_S *frame, cvtdl_object_t *obj);
【Description】
Perform license plate recognition (Taiwan) on all vehicles in the input obj. Prior to calling this API, CVI_TDL_LicensePlateDetection must be called once for license plate detection. The license plate number is stored in obj->info[i].vehicle_properity->license_char.
【Parameter】
Data Type |
Parameter |
Description |
|
---|---|---|---|
Input |
cvitdl_handle_t |
handle |
handle |
Input |
VIDEO_FRAME_INFO_S* |
frame |
The input image |
Input |
cvtdl_object_t* |
obj |
License plate detection results |
4.2.8.3. CVI_TDL_LicensePlateRecognition_CN¶
CVI_S32 CVI_TDL_LicensePlateRecognition_CN(const cvitdl_handle_t handle, VIDEO_FRAME_INFO_S *frame, cvtdl_object_t *obj);
【Description】
Perform license plate recognition (Mainland China) for all vehicles in the input obj. CVI_TDL_LicensePlateDetection must be called before calling this API for license plate detection. The license plate number is stored in obj->info[i].vehicle_properity->license_char.
【Parameter】
Data Type |
Parameter |
Description |
|
---|---|---|---|
Input |
cvitdl_handle_t |
handle |
handle |
Input |
VIDEO_FRAME_INFO_S* |
frame |
image |
Input/Output |
cvtdl_object_t* |
obj |
icense plate detection results |
4.2.9. Liveness Detection¶
4.2.9.1. CVI_TDL_IrLiveness¶
【Syntax】
CVI_S32 CVI_TDL_IrLiveness(const cvitdl_handle_t handle, VIDEO_FRAME_INFO_S *irFrame, cvtdl_face_t *ir_faces);
【Description】
This function performs IR (infrared) monocular liveness detection.
It determines whether the detected faces in ‘ir_faces’ are real or not.
The liveness scores for each face will be stored in ‘ir_faces->info[i].liveness_score’.
【Parameter】
Data Type |
Parameter |
Description |
|
---|---|---|---|
Input |
cvitdl_handle_t |
handle |
Handle |
Input |
VIDEO_FRAME_INFO_S* |
irFrame |
The ir image |
Input/Output |
cvtdl_face_t* |
ir_faces |
Detected IR face/ liveness score |
4.2.10. Pose Estimation¶
4.2.10.1. CVI_TDL_PoseDetection¶
【Syntax】
CVI_S32 CVI_TDL_PoseDetection(const cvitdl_handle_t handle, VIDEO_FRAME_INFO_S *frame, CVI_TDL_SUPPORTED_MODEL_E model_id, cvtdl_object_t *obj_meta)
【Description】
Use the model ID to call the attitude detection algorithm to detect 17 key points of the human body.
【Parameter】
Data Type |
Parameter |
Description |
|
---|---|---|---|
Input |
cvitdl_handle_t |
handle |
handle |
Input |
VIDEO_FRAME_INFO_S* |
frame |
input image |
Input |
CVI_TDL_SUPPORTED_MODEL_E |
model_id |
model id |
Output |
cvtdl_object_t* |
obj_meta |
The detected objects and key points |
【Optional model ID list】
id |
name |
description |
---|---|---|
34 |
CVI_TDL_SUPPORTED_MODEL_YOLOV8POSE |
Optional model ID list |
35 |
CVI_TDL_SUPPORTED_MODEL_SIMCC_POSE |
simcc model |
4.2.11. Semantic Segmentation¶
4.2.11.1. CVI_TDL_YoloV8_Seg¶
【Syntax】
CVI_S32 CVI_TDL_YoloV8_Seg(const cvitdl_handle_t handle, VIDEO_FRAME_INFO_S *frame, cvtdl_object_t *obj_meta);
【Description】
Use the YOLOV8 model for example segmentation.
【Parameter】
Data Type |
Parameter |
Description |
|
---|---|---|---|
Input |
cvitdl_handle_t |
handle |
handle |
Input |
VIDEO_FRAME_INFO_S* |
frame |
input image |
Output |
cvtdl_object_t* |
obj_meta |
The detected object and boundary point |
4.2.11.2. CVI_TDL_Topformer_Seg¶
【Syntax】
CVI_S32 CVI_TDL_Topformer_Seg(const cvitdl_handle_t handle, VIDEO_FRAME_INFO_S *frame, cvtdl_seg_t *seg_ann);
【Description】
Use the TopFormer model for semantic segmentation.
【Parameter】
Data Type |
Parameter |
Description |
|
---|---|---|---|
Input |
cvitdl_handle_t |
handle |
handle |
Input |
VIDEO_FRAME_INFO_S* |
frame |
input image |
Output |
cvtdl_seg_t * |
seg_ann |
the id of input image |
4.2.12. Fall Detection¶
4.2.12.1. CVI_TDL_Fall_Monitor¶
【Syntax】
CVI_S32 CVI_TDL_Fall_Monitor(cvitdl_handle_t handle, cvtdl_object_t *obj);
【Description】
To judge the fall state, you need to call the attitude detection model before use.
【Parameter】
Data Type |
Parameter |
Description |
|
---|---|---|---|
Input |
cvitdl_handle_t |
handle |
handle |
Output |
cvtdl_object_t* |
obj |
the result of fall detection |
4.2.13. Driver Fatigue Detection¶
4.2.13.1. CVI_TDL_FaceLandmarker¶
【Syntax】
CVI_S32 CVI_TDL_FaceLandmarker(cvitdl_handle_t handle, VIDEO_FRAME_INFO_S *frame, cvtdl_face_t *faces);
【Description】
Before using this API, the face detection API should be called to obtain the detection result of 106 facial landmarks, which should be placed in face->dms[i].landmarks_106 and update the 5 facial landmarks in face->dms[i].landmarks_5.
【Parameter】
Data Type |
Parameter |
Description |
|
---|---|---|---|
Input |
cvitdl_handle_t |
handle |
handle |
Input |
VIDEO_FRAME_INFO_S* |
frame |
Input image |
Input |
cvtdl_face_t* |
face |
Human face |
4.2.13.2. CVI_TDL_EyeClassification¶
【Syntax】
CVI_S32 CVI_TDL_EyeClassification (cvitdl_handle_t handle, VIDEO_FRAME_INFO_S *frame, cvtdl_face_t *faces);
【Description】
The closed state of the eyes is determined based on the results of face detection and facial landmark detection. The scores of the right and left eyes are stored in face->dms[i].reye_score and face->dms[i].leye_score, respectively.
【Parameter】
Data Type |
Parameter |
Description |
|
---|---|---|---|
Input |
cvitdl_handle_t |
handle |
handle |
Input |
VIDEO_FRAME_INFO_S* |
frame |
Input image |
Input |
cvtdl_face_t* |
face |
Human face |
4.2.13.3. CVI_TDL_YawnClassification¶
【Syntax】
CVI_S32 CVI_TDL_YawnClassification (cvitdl_handle_t handle, VIDEO_FRAME_INFO_S *frame, cvtdl_face_t *faces);
【Description】
Detect yawning based on the results of face detection and facial landmark detection. CVI_FaceRecognition must be called first to obtain the results of face detection and facial landmark detection. The yawning score will be placed in face->dms[i].yawn_score, ranging from 0.0 to 1.0.
【Parameter】
Data Type |
Parameter |
Description |
|
---|---|---|---|
Input |
cvitdl_handle_t |
handle |
handle |
Input |
VIDEO_FRAME_INFO_S* |
frame |
Input image |
Input |
cvtdl_face_t* |
face |
Human face |
4.2.14. Sound Classification¶
4.2.14.1. CVI_TDL_SoundClassification¶
【Syntax】
CVI_S32 CVI_TDL_SoundClassification(cvitdl_handle_t handle, VIDEO_FRAME_INFO_S *frame, int *index);
【Description】
Classify the audio in the frame into different categories and output the scores of each category in sorted order.
【Parameter】
Data Type |
Parameter |
Description |
|
---|---|---|---|
Input |
cvitdl_handle_t |
handle |
handle |
Input |
VIDEO_FRAME_INFO_S* |
frame |
Input image |
Input |
int* |
index |
Scores in each category |
4.2.14.2. CVI_TDL_Get_SoundClassification_ClassesNum¶
【Syntax】
CVI_S32 CVI_TDL_Get_SoundClassification_ClassesNum(cvitdl_handle_t handle);
【Description】
Get the number of audio categories.
【Parameter】
Data Type |
Parameter |
Description |
|
---|---|---|---|
Input |
cvitdl_handle_t |
handle |
Handle |
【Output】
Data Type |
Description |
|
---|---|---|
Output |
int |
The number of category |
4.2.14.3. CVI_TDL_Set_SoundClassification_Threshold¶
【Syntax】
CVI_S32 CVI_TDL_Set_SoundClassification_Threshold(cvitdl_handle_t handle, const float th);
【Description】
Set the audio category threshold.
【Parameter】
Data Type |
Parameter |
Description |
|
---|---|---|---|
Input |
cvitdl_handle_t |
handle |
Handle |
Input |
const float |
th |
Similarity threshold, the similarity higher than this threshold will be taken out. |
4.3. CVI_TDL_Service¶
4.3.1. Common¶
4.3.1.1. CVI_TDL_Service_CreateHandle¶
【Syntax】
CVI_S32 CVI_TDL_Service_CreateHandle(cvitdl_service_handle_t *handle, cvtdl_handle tdl_handle);
【Description】
Create a Service handle
【Parameter】
Data Type |
Parameter |
Description |
|
---|---|---|---|
Input |
cvitdl_service_handle_t* |
handle |
handle |
Input |
cvitdl_handle_t |
tdl_handle |
cvi_tdl_core handle |
4.3.1.2. CVI_TDL_Service_DestroyHandle¶
【Syntax】
CVI_S32 CVI_TDL_Service_DestroyHandle(cvitdl_service_handle_t *handle);
【Description】
Destroy the Service handle
【Parameter】
Data Type |
Parameter |
Description |
|
---|---|---|---|
Input |
cvitdl_service_handle_t* |
handle |
Handle |
4.3.1.3. CVI_TDL_Service_Polygon_SetTarget¶
【Syntax】
CVI_S32 CVI_TDL_Service_Polygon_SetTarget(cvitdl_service_handle_t handle, const cvtdl_pts_t *pts);
【Description】
Set up a region of interest for intrusion detection with pts as the coordinates of the convex polygon. The order of the points should be clockwise or counterclockwise. Call CVI_TDL_Service_Polygon_Intersect to check if a bounding box intersects with the defined region.
【Parameter】
Data Type |
Parameter |
Description |
|
---|---|---|---|
Input |
cvitdl_service_handle_t |
handle |
handle |
Input |
cvtdl_pts_t* |
pts |
Convex polygon point |
4.3.1.4. CVI_TDL_Service_Polygon_Intersect¶
【Syntax】
CVI_S32 CVI_TDL_Service_Polygon_Intersect(cvitdl_service_handle_t handle, const cvtdl_bbox_t *bbox, bool *has_intersect);
【Description】
Detect if a given bounding box is within the targeted intrusion area previously set by calling CVI_TDL_Service_Polygon_SetTarget.
【Parameter】
Data Type |
Parameter |
Description |
|
---|---|---|---|
Input |
cvitdl_service_handle_t |
handle |
handle |
Input |
cvtdl_bbox_t* |
bbox |
Bounding box |
Output |
bool* |
ha s_intersect |
Intrusion or not |
4.3.1.5. CVI_TDL_Service_RegisterFeatureArray¶
【Syntax】
CVI_S32 CVI_TDL_Service_RegisterFeatureArray(cvitdl_service_handle_t handle, const cvtdl_service_feature_array_t featureArray, const cvtdl_service_feature_matching_e method);
【Description】
Register a feature database by precomputing and loading the features in the featureArray into memory.
【Parameter】
Data Type |
Parameter |
Description |
|
---|---|---|---|
Input |
cvitdl_service_handle_t |
handle |
handle |
Input |
const cvtdl_service_feature_array_t |
featureArray |
Feature array structure |
Input |
const cvtdl_service_feature_matching_e |
method |
Currently, only COS_SIMILARITY is supported |
4.3.1.6. CVI_TDL_Service_CalculateSimilarity¶
【Syntax】
CVI_S32 CVI_TDL_Service_CalculateSimilarity(cvitdl_service_handle_t handle, const cvtdl_feature_t *feature_rhs, const cvtdl_feature_t *feature_lhs, float *score);
【Description】
Compute the cosine similarity between two feature vectors using the RISC-V. The cosine similarity is calculated using the following formula:
The Cosine Similarity between two features of length n is calculated using the following formula. Currently, only INT8 features are supported.
【Parameter】
Data Type |
Parameter |
Description |
|
---|---|---|---|
Input |
cvitdl_service_handle_t |
handle |
handle |
Input |
const cvtdl_feature_t* |
feature_rhs |
The first feature |
Input |
const cvtdl_feature_t* |
feature_lhs |
Second feature |
Output |
float* |
score |
Similarity |
4.3.1.7. CVI_TDL_Service_ObjectInfoMatching¶
【Syntax】
CVI_S32 CVI_TDL_Service_ObjectInfoMatching(cvitdl_service_handle_t handle, const cvtdl_object_info_t *object_info, const uint32_t topk, float threshold, uint32_t *indices, float *sims, uint32_t *size);
【Description】
Compute the cosine similarity between the object feature in object_info and the pre-calculated object features in the registered object feature library. Retrieve the top-K similarities that are greater than threshold. The calculation formula is as follows:
Calculate the Cosine Similarity between the object feature in object_info and the registered object feature in the feature library.
The top-K similarities greater than the given threshold will be returned.
If the number of features in the library is less than 1000, RISC-V will be used for calculation, otherwise TPU will be launched for computation.
Feature registration can be done through CVI_TDL_Service_RegisterFeatureArray.
Note that the feature length n must be consistent and INT8 features are currently supported.
【Parameter】
Data Type |
Parameter |
Description |
|
---|---|---|---|
Input |
cvitdl_service_handle_t |
handle |
handle |
Input |
const cvtdl_object_info_t* |
object_info |
Object Info |
Input |
const uint32_t |
topk |
Take topk similarity |
Output |
float |
threshold |
Similarity threshold, above which the similarity will be taken out |
Output |
uint32_t* |
indices |
The Index of the similarity that satisfies the condition in the library |
Output |
float* |
sims |
The similarity of the condition |
Output |
uint32_t* |
size |
The number of similarities that are finally extracted |
4.3.1.8. CVI_TDL_Service_FaceInfoMatching¶
【Syntax】
CVI_S32 CVI_TDL_Service_FaceInfoMatching(cvitdl_service_handle_t handle, const cvtdl_face_info_t *face_info, const uint32_t topk, float threshold, uint32_t *indices, float *sims, uint32_t *size);
【Description】
Calculate the Cosine Similarity between the face features in face_info and the registered face feature library, and output the top-K similarities that are greater than the given threshold.
The calculation formula is as follows:
Where n is the feature length.
If the number of features in the library is less than 1000, the calculation will be performed on the RISC-V.
Otherwise, the TPU will be used for calculation.
Feature registration requires calling CVI_TDL_Service_RegisterFeatureArray.
Currently, only INT8 features are supported.
【Parameter】
Data Type |
Parameter |
Description |
|
---|---|---|---|
Input |
cvitdl_service_handle_t |
handle |
handle |
Input |
const cvtdl_face_info_t* |
face_info |
Face info |
Input |
const uint32_t |
topk |
Take topk similarity |
Output |
float |
threshold |
Similarity threshold, above which the similarity will be taken out |
Output |
uint32_t* |
indices |
The Index of the similarity that satisfies the condition in the library |
Output |
float* |
sims |
The similarity of the condition |
Output |
uint32_t* |
size |
The number of similarities that are finally extracted |
4.3.1.9. CVI_TDL_Service_RawMatching¶
【Syntax】
CVI_S32 CVI_TDL_Service_RawMatching(cvitdl_service_handle_t handle, const void *feature, const feature_type_e type, const uint32_t topk, float threshold, uint32_t *indices, float *scores, uint32_t *size);
【Description】
Compute the Cosine Similarity between a given feature and the registered feature library.
Retrieve the top-K similarities greater than the specified threshold.
The computation is done using the following formula:
where n is the length of the feature.
If the number of registered feature arrays is less than 1000, the calculation will be performed on RISC-V.
Otherwise, TPU will be launched for calculation.
The feature arrays need to be registered using CVI_TDL_Service_RegisterFeatureArray.
Unlike CVI_TDL_Service_FaceInfoMatching and CVI_TDL_Service_ObjectInfoMatching, this API directly uses the feature array for comparison and does not require cvtdl_face_info_t or cvtdl_object_info_t to be passed in.
This API limits the feature type to be the same as the feature type in the feature arrays.
Currently, only INT8 features are supported.
【Parameter】
Data Type |
Parameter |
Description |
|
---|---|---|---|
Input |
cvitdl_service_handle_t |
handle |
handle |
Input |
const void* |
feature |
Feature array |
Input |
const feature_type_e |
type |
Feature type, currently only TYPE_INT8 is supported |
Input |
const uint32_t |
topk |
Take topk similarity |
Output |
float |
threshold |
Similarity threshold, above which the similarity will be taken out |
Output |
uint32_t* |
indices |
The Index of the similarity that satisfies the condition in the library |
Output |
float* |
scores |
The similarity of the condition |
Output |
uint32_t* |
size |
The number of similarities that are finally extracted |
4.3.1.10. CVI_TDL_Service_FaceAngle¶
【Syntax】
CVI_S32 CVI_TDL_Service_FaceAngle(const cvtdl_pts_t *pts, cvtdl_head_pose_t *hp);
【Description】
Estimate the pose of a single face.
【Parameter】
Data Type |
Parameter |
Description |
|
---|---|---|---|
Input |
cvtdl_pts_t* |
pts |
Face landmark |
Output |
cvtdl_head_pose_t* |
hp |
Face posture |
4.3.1.11. CVI_TDL_Service_FaceAngleForAll¶
【Syntax】
CVI_S32 CVI_TDL_Service_FaceAngleForAll(const cvtdl_face_t *meta);
【Description】
Calculate pose for multiple faces
【Parameter】
Data Type |
Parameter |
Description |
|
---|---|---|---|
Input |
cvtdl_face_t* |
meta |
Face data |
4.3.2. Image Scaling¶
4.3.2.1. CVI_TDL_Service_FaceDigitalZoom¶
【Syntax】
CVI_S32 CVI_TDL_Service_FaceDigitalZoom(
cvitdl_service_handle_t handle,
const VIDEO_FRAME_INFO_S *inFrame,
const cvtdl_face_t *meta,
const float face_skip_ratio,
const float trans_ratio,
const float padding_ratio,
VIDEO_FRAME_INFO_S *outFrame);
【Description】
Resizing (zooming in) the detected face.
【Parameter】
Data Type |
Parameter |
Description |
|
---|---|---|---|
Input |
cvitdl_service_handle_t |
handle |
handle |
Input |
VIDEO_FRAME_INFO_S* |
inFrame |
Input image |
Input |
cvtdl_face_t* |
meta |
Face data |
Input |
float |
face_skip_ratio |
Neglect ratio |
Input |
float |
trans_ratio |
Amplification ratio |
Input |
float |
padding_ratio |
Expand bounding box ratio |
Output |
VIDEO_FRAME_INFO_S* |
outFrame |
Output image |
4.3.2.2. CVI_TDL_Service_ObjectDigitalZoom¶
【Syntax】
CVI_S32 CVI_TDL_Service_ObjectDigitalZoom(cvitdl_service_handle_t handle,
const VIDEO_FRAME_INFO_S *inFrame, const cvtdl_object_t *meta, const float obj_skip_ratio, const float trans_ratio, const float padding_ratio,
VIDEO_FRAME_INFO_S *outFrame);
【Description】
The object detected in the detection result is zoomed in.
【Parameter】
Data Type |
Parameter |
Description |
|
---|---|---|---|
Input |
cvitdl_service_handle_t |
handle |
handle |
Input |
const VIDEO_FRAME_INFO_S* |
inFrame |
Input image |
Input |
const cvtdl_object_t* |
meta |
Object data |
Input |
const float |
obj_skip_ratio |
Neglect ratio |
Input |
const float |
trans_ratio |
Amplification ratio |
Input |
const float |
padding_ratio |
Expand bounding box ratio |
Output |
VIDEO_FRAME_INFO_S* |
outFrame |
Output image |
4.3.2.3. CVI_TDL_Service_ObjectDitgitalZoomExt¶
【Syntax】
CVI_S32 CVI_TDL_Service_ObjectDigitalZoomExt(cvitdl_service_handle_t handle, const VIDEO_FRAME_INFO_S *inFrame, const cvtdl_object_t *meta,
const float obj_skip_ratio, const float trans_ratio, const float pad_ratio_left, const float pad_ratio_right, const float pad_ratio_top,
const float pad_ratio_bottom, VIDEO_FRAME_INFO_S *outFrame);
【Description】
Zoom in on the object in the detection results
【Parameter】
Data Type |
Parameter |
Description |
|
---|---|---|---|
Input |
cvitdl_service_handle_t |
handle |
handle |
Input |
VIDEO_FRAME_INFO_S* |
inFrame |
Input image |
Input |
const cvtdl_object_t* |
meta |
Object data |
Input |
const float |
obj_skip_ratio |
Neglect ratio |
Input |
const float |
trans_ratio |
Amplification ratio |
Input |
const float |
pad_ratio_left |
Expansion rate (left) |
Input |
const float |
pad_ratio_right |
Expansion rate (right) |
Input |
const float |
pad_ratio_top |
Expansion rate (top) |
Input |
const float |
pad_ratio_bottom |
Expansion rate (lower) |
Output |
VIDEO_FRAME_INFO_S* |
outFrame |
Output image |
4.3.3. Image drawing¶
4.3.3.1. CVI_TDL_Service_FaceDrawPts¶
【Syntax】
CVI_S32 CVI_TDL_Service_FaceDrawPts(cvtdl_pts_t *pts, VIDEO_FRAME_INFO_S *frame);
【Description】
Drawing face landmarks.
【Parameter】
Data Type |
Parameter |
Description |
|
---|---|---|---|
Input |
cvtdl_pts_t* |
pts |
Face landmark |
Input |
VIDEO_FRAME_INFO_S* |
hp |
Input/output image |
4.3.3.2. CVI_TDL_Service_FaceDrawRect¶
【Syntax】
CVI_S32 CVI_TDL_Service_FaceDrawRect(cvitdl_service_handle_t handle, const cvtdl_face_t *meta, VIDEO_FRAME_INFO_S *frame, const bool drawText, cvtdl_service_brush_t brush);
【Description】
Drawing a bounding box around a detected face.
【Parameter】
Data Type |
Parameter |
Description |
|
---|---|---|---|
Input |
cvitdl_service_handle_t |
handle |
handle |
Input |
cvtdl_face_t* |
meta |
Face data |
Input |
VIDEO_FRAME_INFO_S* |
frame |
Input/output image |
Input |
bool |
drawText |
Whether to draw face names |
Input |
cvtdl_service_brush_t |
brush |
color |
4.3.3.3. CVI_TDL_Service_ObjectDrawPose¶
【Syntax】
CVI_S32 CVI_TDL_Service_ObjectDrawPose(const cvtdl_object_t *meta, VIDEO_FRAME_INFO_S *frame);
【Description】
Draw 17 skeletal points detected by pose estimation.
【Parameter】
Data Type |
Parameter |
Description |
|
---|---|---|---|
Input |
cvtdl_object_t* |
meta |
Skeletal points detection result |
Input |
VIDEO_FRAME_INFO_S* |
frame |
Input image |
4.3.3.4. CVI_TDL_Service_ObjectDrawRect¶
【Syntax】
CVI_S32 CVI_TDL_Service_ObjectDrawRect(cvitdl_service_handle_t handle, const cvtdl_object_t *meta, VIDEO_FRAME_INFO_S *frame, const bool drawText);
【Description】
Draw bounding box of detected objects
【Parameter】
Data Type |
Parameter |
Description |
|
---|---|---|---|
Input |
cvitdl_service_handle_t |
handle |
handle |
Input |
cvtdl_object_t* |
meta |
Object detection result |
Input |
VIDEO_FRAME_INFO_S* |
frame |
Input/output image |
Input |
const bool |
drawText |
Whether to draw class text |
4.3.3.5. CVI_TDL_Service_ObjectWriteText¶
【Syntax】
CVI_S32 CVI_TDL_Service_ObjectWriteText(char *name, int x, int y, VIDEO_FRAME_INFO_S *frame, float r, float g, float b);
【Description】
Draw specified text
【Parameter】
Data Type |
Parameter |
Description |
|
---|---|---|---|
Input |
char* |
name |
Drawn text |
Input |
int |
x |
The x coordinate is drawn |
Input |
int |
y |
The y coordinate is drawn |
Input/Output |
IDEO_FRAME_INFO_S* |
rame |
Input/output image |
Input |
float |
r |
Draw the color r channel value |
Input |
float |
g |
Draw the color g channel value |
Input |
float |
b |
Draw the color b channel value |
4.3.3.6. CVI_TDL_Service_Incar_ObjectDrawRect¶
【Syntax】
CVI_S32 CVI_TDL_Service_Incar_ObjectDrawRect(cvitdl_service_handle_t handle, const cvtdl_dms_od_t *meta, VIDEO_FRAME_INFO_S *frame, const bool drawText, cvtdl_service_brush_t brush);
【Description】
Draw specified text
【Parameter】
Data Type |
Parameter |
Description |
|
---|---|---|---|
Input |
cvitdl_service_handle_t |
handle |
handle |
Input |
const cvtdl_dms_od_t* |
meta |
Object detection result |
Input/Output |
IDEO_FRAME_INFO_S* |
rame |
Input/output image |
Input |
const bool |
drawText |
Whether to draw class text |
Input |
cvtdl_service_brush_t |
brush |
color |