pytorch 反向传播计算

python pytorch

本文用一个不含激活函数和偏置的简单网络，演示一次完整的前向传播、反向传播和参数更新。

1. 前向传播

网络结构如下：

输入、参数和真实值分别为：

\[ \begin{aligned} x_1 &= 0.5, \quad x_2 = 1.0, \quad y = 0.8 \\ w_1 &= 1.0, \quad w_2 = 0.5, \quad w_3 = 0.5 \\ w_4 &= 0.7, \quad w_5 = 1.0, \quad w_6 = 2.0 \end{aligned} \]

先计算两个中间节点 \(h_1\)、\(h_2\)，再得到预测值 \(\hat{y}\)：

\[ \begin{aligned} h_1 &= x_1w_1 + x_2w_2 = 0.5\times 1.0 + 1.0\times 0.5 = 1.0 \\ h_2 &= x_1w_3 + x_2w_4 = 0.5\times 0.5 + 1.0\times 0.7 = 0.95 \\ \hat{y} &= h_1w_5 + h_2w_6 = 1.0\times 1.0 + 0.95\times 2.0 = 2.9 \end{aligned} \]

2. 反向传播

使用平方误差作为损失函数：

\[ L = \frac{1}{2}(\hat{y}-y)^2 = \frac{1}{2}(2.9-0.8)^2 = 2.205 \]

反向传播从 \(L\) 对预测值的梯度开始：

\[ \frac{\partial L}{\partial \hat{y}} = \hat{y}-y = 2.9-0.8 = 2.1 \]

2.1 w₅、w₆ 的梯度

由 \(\hat{y}=h_1w_5+h_2w_6\) 可得：

\[ \begin{aligned} \frac{\partial \hat{y}}{\partial w_5} &= h_1 = 1.0 \\ \frac{\partial \hat{y}}{\partial w_6} &= h_2 = 0.95 \end{aligned} \]

根据链式法则：

\[ \begin{aligned} \frac{\partial L}{\partial w_5} &= \frac{\partial L}{\partial \hat{y}} \frac{\partial \hat{y}}{\partial w_5} = 2.1\times 1.0 = 2.1 \\ \frac{\partial L}{\partial w_6} &= \frac{\partial L}{\partial \hat{y}} \frac{\partial \hat{y}}{\partial w_6} = 2.1\times 0.95 = 1.995 \end{aligned} \]

2.2 w₁、w₂、w₃、w₄ 的梯度

\(w_1\)、\(w_2\) 通过 \(h_1\) 影响预测值，\(w_3\)、\(w_4\) 通过 \(h_2\) 影响预测值。各部分的局部梯度为：

\[ \begin{aligned} \frac{\partial \hat{y}}{\partial h_1} &= w_5 = 1.0, & \frac{\partial \hat{y}}{\partial h_2} &= w_6 = 2.0 \\ \frac{\partial h_1}{\partial w_1} &= x_1 = 0.5, & \frac{\partial h_1}{\partial w_2} &= x_2 = 1.0 \\ \frac{\partial h_2}{\partial w_3} &= x_1 = 0.5, & \frac{\partial h_2}{\partial w_4} &= x_2 = 1.0 \end{aligned} \]

\[ \begin{aligned} \frac{\partial L}{\partial w_1} &= \frac{\partial L}{\partial \hat{y}} \frac{\partial \hat{y}}{\partial h_1} \frac{\partial h_1}{\partial w_1} = 2.1\times 1.0\times 0.5 = 1.05 \\ \frac{\partial L}{\partial w_2} &= \frac{\partial L}{\partial \hat{y}} \frac{\partial \hat{y}}{\partial h_1} \frac{\partial h_1}{\partial w_2} = 2.1\times 1.0\times 1.0 = 2.1 \\ \frac{\partial L}{\partial w_3} &= \frac{\partial L}{\partial \hat{y}} \frac{\partial \hat{y}}{\partial h_2} \frac{\partial h_2}{\partial w_3} = 2.1\times 2.0\times 0.5 = 2.1 \\ \frac{\partial L}{\partial w_4} &= \frac{\partial L}{\partial \hat{y}} \frac{\partial \hat{y}}{\partial h_2} \frac{\partial h_2}{\partial w_4} = 2.1\times 2.0\times 1.0 = 4.2 \end{aligned} \]

3. pytorch 实践

下面直接使用 nn.Parameter 表示六个标量参数，使代码与前面的公式一一对应。

import torch
import torch.nn as nn

class Net(nn.Module):
    def __init__(self):
        super().__init__()
        self.w1 = nn.Parameter(torch.tensor(1.0))
        self.w2 = nn.Parameter(torch.tensor(0.5))
        self.w3 = nn.Parameter(torch.tensor(0.5))
        self.w4 = nn.Parameter(torch.tensor(0.7))
        self.w5 = nn.Parameter(torch.tensor(1.0))
        self.w6 = nn.Parameter(torch.tensor(2.0))

    def forward(self, x1, x2):
        h1 = x1 * self.w1 + x2 * self.w2
        h2 = x1 * self.w3 + x2 * self.w4
        y_hat = h1 * self.w5 + h2 * self.w6
        return y_hat

model = Net()
x1 = torch.tensor(0.5)
x2 = torch.tensor(1.0)
y = torch.tensor(0.8)

optimizer = torch.optim.SGD(model.parameters(), lr=0.01)
optimizer.zero_grad()

# 前向传播
y_hat = model(x1, x2)
loss = 0.5 * (y_hat - y) ** 2
print(f"y_hat = {y_hat.item():.3f}")
print(f"loss = {loss.item():.3f}")

# 反向传播
loss.backward()
for name, parameter in model.named_parameters():
    print(f"grad({name}) = {parameter.grad.item():.3f}")

# 更新参数
optimizer.step()
for name, parameter in model.named_parameters():
    print(f"{name} = {parameter.item():.4f}")

前向传播结果为：

y_hat = 2.900
loss = 2.205

反向传播得到的梯度与手算结果相同：

grad(w1) = 1.050
grad(w2) = 2.100
grad(w3) = 2.100
grad(w4) = 4.200
grad(w5) = 2.100
grad(w6) = 1.995

学习率为 \(0.01\)，SGD 按照 \(w\leftarrow w-0.01\nabla w\) 更新参数：

w1 = 0.9895
w2 = 0.4790
w3 = 0.4790
w4 = 0.6580
w5 = 0.9790
w6 = 1.9800

本文由 Yonghui Wang 创作，采用 知识共享署名4.0 国际许可协议进行许可
本站文章除注明转载/出处外，均为本站原创或翻译，转载前请务必署名
最后编辑时间为: Sep 28, 2026 08:33 pm