【目标检测】YOLOv7算法实现(一)：模型搭建

本文介绍: 本系列文章记录本人硕士阶段YOLO系列目标检测算法自学及其代码实现的过程。其中算法具体实现借鉴于ultralytics YOLO源码，删减了源码中部分内容，满足个人科研需求。本系列文章在YOLOv5算法实现的基础上，进一步完成YOLOv7算法的实现。

γ(b−mean)+β

BN层->卷积层：构造一个参数为0的卷积层(1×1)，实现卷积层+BN层融合

卷积层(3×3)+卷积层(1×1) →卷积层(3×3)：将卷积层(1×1)填充为3×3，再将卷积层权重和偏置相加实现融合

2.5.2 模块实现

class RepConv(nn.Module):
    def __init__(self, c1, c2, k=3, s=1, p=None, g=1, act=True, deploy=False):
        '''
        重参数卷积
        训练时:
            deploy = False
            rbr_dense(3x3卷积) + rbr_1x1(1x1卷积) + rbr_identity(c2==c1时)相加
            rbr_reparam = None
        推理时:
            deploy = True
            rbr_param = Conv2d
            rbr_dense, rbr_1x1, rbr_identity = None, None, None
        '''
        super().__init__()

        self.deploy = deploy
        self.groups = g
        self.in_channels = c1
        self.out_channels = c2

        assert k == 3
        assert autopad(k, p) == 1

        padding_11 = autopad(k, p) - k // 2

        self.act = nn.SiLU() if act is True else (act if isinstance(act, nn.Module) else nn.Identity())

        # 推理阶段, 仅有一个3x3卷积
        if self.deploy:
            self.rbr_reparam = nn.Conv2d(c1, c2, k, s, autopad(k, p), groups=g, bias=True)
        else:
            # 输入输出通道数相同时, identity层(BN层)
            self.rbr_identity = (nn.BatchNorm2d(num_features=c1) if c2 == c1 and s == 1 else None)
            # 3×3卷积 + BN层
            self.rbr_dense = nn.Sequential(
                nn.Conv2d(c1, c2, k, s, autopad(k, p), groups=g, bias=False),
                nn.BatchNorm2d(num_features=c2),
            )
            # 1×1卷积 + BN层
            self.rbr_1x1 = nn.Sequential(
                nn.Conv2d(c1, c2, 1, s, padding_11, groups=g, bias=False),
                nn.BatchNorm2d(num_features=c2),
            )

    def forward(self, x):
        # 推理阶段
        if hasattr(self, 'rbr_reparam'):
            return self.act(self.rbr_reparam(x))

        # 训练阶段
        if self.rbr_identity is None:
            id_out = 0
        else:
            id_out = self.rbr_identity(x)

        return self.act(self.rbr_dense(x) + self.rbr_1x1(x) + id_out)

    #融合卷积层和BN层: Conv2D+BN=Conv2D
    def fuse_conv_bn(self, conv, bn):
        std = (bn.running_var + bn.eps).sqrt()
        bias = bn.bias - bn.running_mean * bn.weight / std

        t = (bn.weight / std).reshape(-1, 1, 1, 1)
        weights = conv.weight * t

        bn = nn.Identity()
        conv = nn.Conv2d(in_channels=conv.in_channels,
                         out_channels=conv.out_channels,
                         kernel_size=conv.kernel_size,
                         stride=conv.stride,
                         padding=conv.padding,
                         dilation=conv.dilation,
                         groups=conv.groups,
                         bias=True,
                         padding_mode=conv.padding_mode)

        conv.weight = torch.nn.Parameter(weights)
        conv.bias = torch.nn.Parameter(bias)
        return conv
    # 重参数操作(在推理阶段执行)
    def fuse_repvgg_block(self):
        if self.deploy:
            return
        print(f"RepConv.fuse_repvgg_block")

        # 融合3x3的卷积层和BN层为一个3x3卷积(有偏置)
        self.rbr_dense = self.fuse_conv_bn(self.rbr_dense[0], self.rbr_dense[1])
        # 融合1x1的卷积层和BN层为一个1x1卷积(有偏置)
        self.rbr_1x1 = self.fuse_conv_bn(self.rbr_1x1[0], self.rbr_1x1[1])
        rbr_1x1_bias = self.rbr_1x1.bias
        # 填充卷积核大小与3x3卷积大小相同
        weight_1x1_expanded = torch.nn.functional.pad(self.rbr_1x1.weight, [1, 1, 1, 1])
        # 融合identity的BN层为一个1x1卷积(无偏置)
        if isinstance(self.rbr_identity, nn.BatchNorm2d) or isinstance(self.rbr_identity, nn.modules.batchnorm.SyncBatchNorm):
            identity_conv_1x1 = nn.Conv2d(
                    in_channels=self.in_channels,
                    out_channels=self.out_channels,
                    kernel_size=1,
                    stride=1,
                    padding=0,
                    groups=self.groups,
                    bias=False)
            identity_conv_1x1.weight.data = identity_conv_1x1.weight.data.to(self.rbr_1x1.weight.data.device)
            identity_conv_1x1.weight.data = identity_conv_1x1.weight.data.squeeze().squeeze()
            identity_conv_1x1.weight.data.fill_(0.0)
            identity_conv_1x1.weight.data.fill_diagonal_(1.0)
            identity_conv_1x1.weight.data = identity_conv_1x1.weight.data.unsqueeze(2).unsqueeze(3)
            # 融合该1x1卷积和Identity的BN层
            identity_conv_1x1 = self.fuse_conv_bn(identity_conv_1x1, self.rbr_identity)
            bias_identity_expanded = identity_conv_1x1.bias
            weight_identity_expanded = torch.nn.functional.pad(identity_conv_1x1.weight, [1, 1, 1, 1])
        else:
            bias_identity_expanded = torch.nn.Parameter(torch.zeros_like(rbr_1x1_bias))
            weight_identity_expanded = torch.nn.Parameter(torch.zeros_like(weight_1x1_expanded))

        # 融合3x3卷积和扩充的1x1卷积的权重和偏置
        self.rbr_dense.weight = torch.nn.Parameter(
            self.rbr_dense.weight + weight_1x1_expanded + weight_identity_expanded)
        self.rbr_dense.bias = torch.nn.Parameter(self.rbr_dense.bias + rbr_1x1_bias + bias_identity_expanded)

        self.rbr_reparam = self.rbr_dense
        self.deploy = True

        if self.rbr_identity is not None:
            del self.rbr_identity
            self.rbr_identity = None

        if self.rbr_1x1 is not None:
            del self.rbr_1x1
            self.rbr_1x1 = None

        if self.rbr_dense is not None:
            del self.rbr_dense
            self.rbr_dense = None

2.6 Detect模块

Detect模块的具体实现过程可见文章YOLOv5算法实现(二)：模型搭建

3 模型配置文件构建(model.yaml)

基于图1所示的模型结构和模型模块所需的参数，构建模型配置文件。其中结构解析包含四个参数[from，number，module，args]：

from：当前层的输入来自于哪一层
number：当前层数量
module：当前层所有模块(在common.py中实现，需与类名对应)
args：第一个参数为当前层输出通道数，其余参数为模块特有参数；当前层的输入通道数由“from”参数指向的层决定，在结构解析时加入该参数。

# Parameters
nc: 80  # number of classes 类别数
depth_multiple: 1.0  # model depth multiple 模型深度(模块个数系数)
width_multiple: 1.0  # layer channel multiple 模型宽度(模块通道数系数)
anchors: 
  - [10,13, 16,30, 33,23]  # P3/8 (stride=8 feature_map所用Anchor,小目标检测)
  - [30,61, 62,45, 59,119]  # P4/16 (stride=16 feature_map所用Anchor)
  - [116,90, 156,198, 373,326]  # P5/32 (stride=32 feature_map所用Anchor,大目标检测)

backbone:
  # [from, number, module, args]
  [[-1, 1, Conv, [32, 3, 1]],  # 0
   [-1, 1, Conv, [64, 3, 2]],  # 1-P1/2
   [-1, 1, Conv, [64, 3, 1]],  # 2

   [-1, 1, Conv, [128, 3, 2]], # 3-P2/4
   [-1, 1, ELAN_B, [256]],       # 4

   [-1, 1, MP, [256]],         # 5-P3/8
   [-1, 1, ELAN_B, [512]],       # 6

   [ -1, 1, MP, [512]],        # 7-P4/16
   [ -1, 1, ELAN_B, [1024]],     # 8

   [ -1, 1, MP, [1024]],        # 9-P5/32
   [ -1, 1, ELAN_B, [1024, 0.25]],# 10
  ]

head:
  [[-1, 1, SPPCSPC, [512]],     # 11

   [-1, 1, Conv, [256, 1, 1]],
   [-1, 1, nn.Upsample, [None, 2, 'nearest']],
   [8, 1, Conv, [256, 1, 1]],  # route backbone P4
   [[-1, -2], 1, Concat, [1]],

   [-1, 1, ELAN_H, [256]],  # 16

   [-1, 1, Conv, [128, 1, 1]],
   [-1, 1, nn.Upsample, [None, 2, 'nearest']],
   [6, 1, Conv, [128, 1, 1]], # route backbone P3
   [[-1, -2], 1, Concat, [1]],

   [-1, 1, ELAN_H, [128]],  # 21 (P3/8-samll)

   [-1, 1, MP, [256]],
   [[-1, 16], 1, Concat, [1]],

   [-1, 1, ELAN_H, [256]],  # 24 (P4/16-medium)

   [-1, 1, MP, [512]],
   [[-1, 11], 1, Concat, [1]],

   [-1, 1, ELAN_H, [512]],  # 27 (P5/32-large)

   [21, 1, RepConv, [256, 3, 1]], # 28
   [24, 1, RepConv, [512, 3, 1]], # 29
   [27, 1, RepConv, [1024, 3, 1]], # 30

   [[28, 29, 30], 1, Detect, [nc, anchors]],  # Detect(P3, P4, P5)
  ]

4 模型搭建(yolo.py)

模型搭建的具体实现方法可见文章YOLOv5算法实现(二)：模型搭建，在YOLOv7中，在模型类中额外添加一个如下函数实现RepConv模块的融合即可。

    def fuse(self):
        print('Fusing layers...')
        for m in self.model.modules():
            if isinstance(m, RepConv):
                m.fuse_repvgg_block()
        return self