5. Deployment of Yolov7 Model for General Use¶

5.1. Introduction¶

This document introduces the operation process of deploying the YOLOV7 architecture model on the CV181x development board. The main steps include:

Convert YOLOV7 model Pytorch version to ONNX model
Convert onnx model to cvi model format
Finally, write a calling interface to obtain the inference results

5.2. Convert pt Model to onnx¶

Download official [yolov7](https://github.com/WongKinYiu/yolov7)Warehouse code

Git clone https://github.com/WongKinYiu/yolov7.git

Create a new folder called weights in the directory where you downloaded the code above, and then move the model that needs to be exported to onnx to yolov7/weights

cd yolov7&mkdir weights

cp path/to/nnx/ Weights/

Create a new file onnx in the yolov7/directory_Export.py, add the following code

import torch
import torch.nn as nn
import onnx
import models
from models.common import Conv
from utils.activations import Hardswish, SiLU
import types

pt_path = "path/to/yolov7-tiny.pt"
save_path = pt_path.replace(".pt", ".onnx")

ckpt = torch.load(pt_path, map_location="cpu")
model = ckpt["model"].float().fuse().eval()

# Compatibility updates
for m in model.modules():
    if type(m) in [nn.Hardswish, nn.LeakyReLU, nn.ReLU, nn.ReLU6, nn.SiLU]:
        m.inplace = True  # pytorch 1.7.0 compatibility
    elif type(m) is nn.Upsample:
        m.recompute_scale_factor = None  # torch 1.11.0 compatibility
    elif type(m) is Conv:
        m._non_persistent_buffers_set = set()

# Update model
    for k, m in model.named_modules():
        m._non_persistent_buffers_set = set()  # pytorch 1.6.0 compatibility
        if isinstance(m, models.common.Conv):  # assign export-friendly activations
            if isinstance(m.act, nn.Hardswish):
                m.act = Hardswish()
            elif isinstance(m.act, nn.SiLU):
                m.act = SiLU()

def forward(self, x):
        # x = x.copy()  # for profiling
        z = []  # inference output

        for i in range(self.nl):
            x[i] = self.m[i](x[i])  # conv

            bs, _, ny, nx = x[i].shape  # x(bs,255,20,20) to x(bs,3,20,20,85)
            x[i] = x[i].view(bs, self.na, self.no, ny, nx).permute(0, 1, 3, 4, 2).contiguous()

            xywh, conf, score = x[i].split((4, 1, self.nc), 4)

            z.append(xywh[0])
            z.append(conf[0])
            z.append(score[0])

        return z

model.model[-1].forward = types.MethodType(forward, model.model[-1])
img = torch.zeros(1, 3, 640, 640)
torch.onnx.export(model, img, save_path, verbose=False,
                  opset_version=12, input_names=['images'])

5.3. Onnx Model Conversion cvi model¶

The cvi model conversion operation can refer to the onnx model conversion cvi model section in the Yolo v5 porting chapter.

5.4. TDL SDK Interface Description¶

The detection and decoding process of the YOLOv7 model is basically similar to that of the YOLOv5 model, so the interface of YOLOv5 can be directly used

> ** Pay attention to modifying anchors to anchors of yolov7 ** > > > anchors: > - [12,16, 19,36, 40,28] # P3/8 > - [36,75, 76,55, 72,146] # P4/16 > - [142,110, 192,243, 459,401] # P5/32 >

5.5. Test Result¶

Tested the indicators of various versions of the YOLOV7-TINY model, with the test data being COCO2017, where the threshold was set to:

Conf_Threshold: 0.001
Nms_Threshold: 0.65

All resolutions are 640 x 640

The official export method of the YOLOV7-TINY model onnx performance:

platform	Inference time (ms)	bandwidth (MB)	ION(MB)	MAP 0.5	MAP 0.5-0.95
pytorch	N/A	N/A	N/A	56.7	38.7
cv181x	75.4	85.31	17.54	Quantification failure	Quantification failure
cv182x	56.6	85.31	17.54	Quantification failure	Quantification failure
cv183x	21.85	71.46	16.15	Quantification failure	Quantification failure

TDL of yolov7-tiny model SDK export method onnx performance：

platform	Inference time (ms)	bandwidth (MB)	ION(MB)	MAP 0.5	MAP 0.5-0.95
onnx	N/A	N/A	N/A	53.7094	36.438
cv181x	70.41	70.66	15.43	53.3681	32.6277
cv182x	52.01	70.66	15.43	53.3681	32.6277
cv183x	18.95	55.86	14.05	53.3681	32.6277