5. Application（APP）¶

5.1. Objective¶

Cvitek TDL Application (APP) is a solution designed for different client applications based on TDL SDK.

The APP integrates TDL SDK and provides customers with more convenient operation APIs.

The APP code is open source and can be used as a reference for client development.

【compile】

Download TDL SDK and its dependent SDK: MW, TPU, IVE.
Move to the module/app directory of the TDL SDK
Execute the following commands:

make MW_PATH=<MW_SDK> TPU_PATH=<TPU_SDK> IVE_PATH=<IVE_SDK>

make install

make clean

The compiled lib will be placed in the tmp install directory of the TDL SDK

5.2. API¶

5.2.1. Handle¶

【Syntax】

typedef struct {

cvitdl_handle_t tdl_handle;

IVE_HANDLE ive_handle;

face_capture_t *face_cpt_info;

} cvitdl_app_context_t;

typedef cvitdl_app_context *cvitdl_app_handle_t;

【Description】

The cvitdl_app_handle_t is a pointer type of cvitdl_app_context, which includes the TDL handle, IVE handle, and other data structures for the application.

5.2.1.1. CVI_TDL_APP_CreateHandle¶

【Syntax】

CVI_S32 CVI_TDL_APP_CreateHandle(cvitdl_app_handle_t *handle, cvitdl_handle_t tdl_handle);

【Description】

Create the metrics required for using the APP, which require input of TDL handle and IVE handle.

【Parameter】

	Data Type	Parameter	Description
Output	cvitdl_app_handle_t*	handle	Handle pointer input
Input	cvitdl_handle_t	a i_handle	TDL handle

5.2.1.2. CVI_TDL_APP_DestroyHandle¶

【Syntax】

CVI_S32 CVI_TDL_APP_DestroyHandle(cvitdl_app_handle_t handle);

【Description】

Destroy the created handle cvitdl_app_handle_t. Only the data used by individual applications will be destroyed, and it does not affect the tdl handle and ive handle.

【Parameter】

	Data Type	Parameter	Description
Input	cvitdl_app_handle_t	handle	Handle pointer input

5.2.2. Face Capture¶

Face Capture is a function that combines face detection, multi-object tracking, and face quality assessment to capture (or extract) facial images of different individuals in an image or video. The capture conditions can be adjusted using a configuration file, such as capture time, face quality assessment algorithm, face angle threshold, etc.

【Configuration File】

Parameter	Default Value	Description
miss_time_limit:	40	Face loss time limit. When the APP cannot trace a face continuously, it will determine that the face has left. [Unit: frame]
threshold_size_min	32	Minimum/maximum acceptable face size, if either side of the face bbox is less than/greater than this threshold, quality will be forced to 0.
threshold_size_max	512
quality_assessment_method	0	If face evaluation does not use FQNet, enable the built-in quality detection algorithm 0: based on face size and Angle 1: Based on eye distance
threshold_quality	0.1	Face quality threshold. If the quality of the new face is greater than this threshold and higher than the quality of the currently captured face, the face data in the staging area will be captured and updated.
threshold_quality_high	0.95	Face quality threshold (high). If the quality of a face in the temporary area is higher than this threshold, the face is judged to be of high quality and will not be updated later. (level 2,3 only)
threshold_yaw	0.25	Face Angle threshold, if the Angle is greater than this threshold, quality will be forced to set to 0. (One unit is 90 degrees)
threshold_pitch	0.25
threshold_roll	0.25
fast_mode_interval	10	FAST mode capture interval. [Unit: frame]
fast_mode_capture_num	3	Number of captures in FAST mode.
cycle_mode_interval	20	CYCLE mode Snapshot interval. [Unit: frame]
auto_mode_time_limit	0	Duration of the last output in AUTO mode. [Unit: frame]
auto_mode_fast_cap	1	AUTO Mode Whether to output one quick snapshot.
capture_aligned_face	0	Whether the captured/captured face is corrected.

【Face Quality Assessment Algorithm】

#

Algorithm

Computation

0

Based on face size and Angle

Face Area Score

Definition of standard face size A_base = 112 * 112

Calculate the detected face area A_face = length * width

Calculate MIN(1.0, A_face/A_base) as a fraction

Face Pose Score

Calculate face Angle yaw, pitch and roll respectively and take their absolute values

Calculate 1 - (yaw + pitch + roll) / 3 as a score

Face Quality = Face Area Score * Face Pose Score

1

Eye distance based

1 Define standard pupil distance D = 80

2 Calculate the distance d between the eyes

3 Calculate MIN(1.0, d/D) as a fraction

5.2.2.1. CVI_TDL_APP_FaceCapture_Init¶

【Syntax】

CVI_S32 CVI_TDL_APP_FaceCapture_Init(const cvitdl_app_handle_t handle, uint32_t buffer_size);

【Description】

Initialize the data structure for face capture.

【Parameter】

	Data Type	Parameter	Description
Input	cvitdl_app_handle_t	handle	Handle pointer input
Input	uint32_t	buffer_size	Face staging area size

5.2.2.2. CVI_TDL_APP_FaceCapture_QuickSetUp¶

【Syntax】

CVI_S32 CVI_TDL_APP_FaceCapture_QuickSetUp(const cvitdl_app_handle_t handle, int fd_model_id, int fr_model_id, const char *fd_model_path, const char *fr_model_path, const char *fq_model_path, const char *fl_model_path);

【Description】

Quickly set up face capture.

【Parameter】

	Data Type	Parameter	Description
Input	cvitdl_app_handle_t	handle	Handle pointer input
Input	int	fd_model_id	Face detection Model ID
Input	int	fr_model_id	Face recognition model ID
Input	const char*	fd_model_path	Face detection model Path
Input	const char*	fr_model_path	Face recognition model path
Input	const char*	fq_model_path	Face quality detection model path
Input	const char*	fl_model_path	Face key point detection model path

5.2.2.3. CVI_TDL_APP_FaceCapture_FusePedSetup¶

【Syntax】

CVI_S32 CVI_TDL_APP_FaceCapture_FusePedSetup(const cvitdl_app_handle_t handle, int ped_model_id, const char *ped_model_path);

【Description】

Quickly set up face capture.

【Parameter】

	Data Type	Parameter	Description
Input	cvitdl_app_handle_t	handle	Handle pointer input
Input	int	fd_model_id	Pedestrian detection model ID
Input	const char*	fl_model_path	Pedestrian detection model path

5.2.2.4. CVI_TDL_APP_FaceCapture_GetDefaultConfig¶

【Syntax】

CVI_S32 CVI_TDL_APP_FaceCapture_GetDefaultConfig(face_capture_config_t *cfg);

【Description】

Get the preset parameters for face capture.

【Parameter】

	Data Type	Parameter	Description
Output	face_capture_config_t*	cfg	Face capture parameters

5.2.2.5. CVI_TDL_APP_FaceCapture_SetConfig¶

【Syntax】

CVI_S32 CVI_TDL_APP_FaceCapture_SetConfig(const cvitdl_app_handle_t handle, face_capture_config_t *cfg);

【Description】

Setting face capture parameters.

【Parameter】

	Data Type	Parameter	Description
Input	cvitdl_app_handle_t	handle	Handle pointer input
Input	face_capture_config_t*	cfg	Face capture parameters.

5.2.2.6. CVI_TDL_APP_FaceCapture_FDFR¶

【Syntax】

CVI_S32 CVI_TDL_APP_FaceCapture_FDFR(const cvitdl_app_handle_t handle, VIDEO_FRAME_INFO_S *frame, cvtdl_face_t *p_face);

【Description】

Set face capture parameters.

【Parameter】

	Data Type	Parameter	Description
Input	cvitdl_app_handle_t	handle	Handle pointer input
Input	VIDEO_FRAME_INFO_S*	frame	Input image
Input	cvtdl_face_t*	p_face	Face capture output result

5.2.2.7. CVI_TDL_APP_FaceCapture_SetMode¶

【Syntax】

CVI_S32 CVI_TDL_APP_FaceCapture_SetMode(const cvitdl_app_handle_t handle, capture_mode_e mode);

【Description】

Set face capture mode.

【Parameter】

	Data Type	Parameter	Description
Input	cvitdl_app_handle_t	handle	Handle pointer input
Input	capture_mode_e	mode	Face capture mode

5.2.2.8. CVI_TDL_APP_FaceCapture_Run¶

【Syntax】

CVI_S32 CVI_TDL_APP_FaceCapture_Run(const cvitdl_app_handle_t handle, VIDEO_FRAME_INFO_S *frame);

【Description】

Perform face capture.

【Parameter】

	Data Type	Parameter	Description
Input	cvitdl_app_handle_t	handle	Handle pointer input
Input	VIDEO_FRAME_INFO_S*	frame	Input image

5.2.2.9. CVI_TDL_APP_FaceCapture_CleanAll¶

【Syntax】

CVI_S32 CVI_TDL_APP_FaceCapture_CleanAll(const cvitdl_app_handle_t handle);

【Description】

Clear all the data in the temporary storage area for face capture.

【Parameter】

	Data Type	Parameter	Description
Input	cvitdl_app_handle_t	handle	Handle pointer input

5.2.3. Humanoid Capture¶

Human body capture is a function that combines human body detection, multi-object tracking, and face quality detection to capture (or extract) photos of different people’s faces in an image.

The capture conditions can be adjusted using a configuration file, such as the capture time, face quality detection algorithm, face angle threshold, etc.

【Configuration File】

Parameter	Default Value	Description
miss_time_limit:	40	Face loss time limit. When the APP cannot trace a face continuously, it will determine that the face has left. [Unit: frame]
threshold_size_min	32	Minimum/maximum acceptable face size, if either side of the face bbox is less than/greater than this threshold, quality will be forced to 0.
threshold_size_max	512
quality_assessment_method	0	If face evaluation does not use FQNet, enable the built-in quality detection algorithm 0: based on face size and Angle 1: Based on eye distance
threshold_quality	0.1	Face quality threshold. If the quality of the new face is greater than this threshold and higher than the quality of the currently captured face, the face data in the staging area will be captured and updated.
threshold_quality_high	0.95	Face quality threshold (high). If the quality of a face in the temporary area is higher than this threshold, the face is judged to be of high quality and will not be updated later. (level 2,3 only)
threshold_yaw	0.25	Face Angle threshold, if the Angle is greater than this threshold, quality will be forced to set to 0. (One unit is 90 degrees)
threshold_pitch	0.25
threshold_roll	0.25
fast_mode_interval	10	FAST mode capture interval. [Unit: frame]
fast_mode_capture_num	3	Number of captures in FAST mode.
cycle_mode_interval	20	CYCLE mode Snapshot interval. [Unit: frame]
auto_mode_time_limit	0	Duration of the last output in AUTO mode. [Unit: frame]
auto_mode_fast_cap	1	AUTO Mode Whether to output one quick snapshot.
capture_aligned_face	0	Whether the captured/captured face is corrected.

【Face Quality Assessment Algorithm】

#

Algorithm

Computation

0

Based on face size and Angle

Face Area Score

Definition of standard face size A_base = 112 * 112

Calculate the detected face area A_face = length * width

Calculate MIN(1.0, A_face/A_base) as a fraction

Face Pose Score

Calculate face Angle yaw, pitch and roll respectively and take their absolute values

Calculate 1 - (yaw + pitch + roll) / 3 as a score

Face Quality = Face Area Score * Face Pose Score

1

Eye distance based

1 Define standard pupil distance D = 80

2 Calculate the distance d between the eyes

3 Calculate MIN(1.0, d/D) as a fraction

5.2.3.1. CVI_TDL_APP_PersonCapture_Init¶

【Syntax】

CVI_TDL_APP_PersonCapture_Init(const cvitdl_app_handle_t handle, uint32_t buffer_size);

【Description】

Initialize the data structure for human body capture.

【Parameter】

	Data Type	Parameter	Description
Input	cvitdl_app_handle_t	handle	Handle pointer input
Input	uint32_t	buffer_size	Face staging area size

5.2.3.2. CVI_TDL_APP_PersonCapture_QuickSetUp¶

【Syntax】

CVI_S32 CVI_TDL_APP_PersonCapture_QuickSetUp(const cvitdl_app_handle_t handle,

const char *od_model_name,

const char *od_model_path,

const char *reid_model_path);

【Description】

Quickly set up human body capture.

【Parameter】

	Data Type	Parameter	Description
Input	cvitdl_app_handle_t	handle	Handle pointer input
Input	const char*	od_model_name	Model name for human detection
Input	const char*	od_model_path	Path of human detection model
Input	const char*	reid_model_path	Path of ReID model

5.2.3.3. CVI_TDL_APP_FaceCapture_GetDefaultConfig¶

【Syntax】

CVI_S32 CVI_TDL_APP_PersonCapture_GetDefaultConfig(person_capture_config_t *cfg);

【Description】

Get the preset parameters for human body capture.

【Parameter】

	Data Type	Parameter	Description
Output	person_capture_config_t*	cfg	Human body capture parameters

5.2.3.4. CVI_TDL_APP_PersonCapture_SetConfig¶

【Syntax】

CVI_S32 CVI_TDL_APP_PersonCapture_SetConfig(const cvitdl_app_handle_t handle, person_capture_config_t *cfg);

【Description】

Setting the parameters for the person capture.

【Parameter】

	Data Type	Parameter	Description
Input	cvitdl_app_handle_t	handle	Handle pointer input
Input	person_capture_config_t*	cfg	The parameters for human pose capture

5.2.3.5. CVI_TDL_APP_PersonCapture_SetMode¶

【Syntax】

CVI_S32 CVI_TDL_APP_PersonCapture_SetMode(const cvitdl_app_handle_t handle, capture_mode_e mode);

【Description】

Setting the mode of person capture

【Parameter】

	Data Type	Parameter	Description
Input	cvitdl_app_handle_t	handle	Handle pointer input
Input	capture_mode_e	mode	Person capture mode

5.2.3.6. CVI_TDL_APP_PersonCapture_Run¶

【Syntax】

CVI_S32 CVI_TDL_APP_PersonCapture_Run(const cvitdl_app_handle_t handle, VIDEO_FRAME_INFO_S *frame);

【Description】

Perform human body capture

【Parameter】

	Data Type	Parameter	Description
Input	cvitdl_app_handle_t	handle	Handle pointer input
Input	VIDEO_FRAME_INFO_S*	frame	Input image

5.2.3.7. CVI_TDL_APP_ConsumerCounting_Run¶

【Syntax】

CVI_S32 CVI_TDL_APP_ConsumerCounting_Run(const cvitdl_app_handle_t handle, VIDEO_FRAME_INFO_S *frame);

【Description】

Execute human type capture.

【Parameter】

	Data Type	Parameter	Description
Input	cvitdl_app_handle_t	handle	Handle pointer input
Input	VIDEO_FRAME_INFO_S*	frame	Input image

5.2.3.8. CVI_TDL_APP_PersonCapture_CleanAll¶

【Syntax】

CVI_S32 CVI_TDL_APP_PersonCapture_ClanAll(const cvitdl_app_handle_t handle);

【Description】

Clear all data in the temporary storage area for human form capture.

【Parameter】

	Data Type	Parameter	Description
Input	cvitdl_app_handle_t	handle	Handle pointer input