Image Classification with ResNet

A realistic classifier combines updates, normalization, and aggregation

Arc 5: CNN fundamentals → empirical evidence → ResNet classification → U-Net segmentation.

Update: refine a representation with residual blocks.
Normalize: control feature scales within the network.
Assemble: read a complete ResNet-18.

The simple comparison is complete. Now recognize how these components fit into a widely used architecture.

A residual block learns a change to the representation

An identity shortcut carries the input around a learned correction branch to an addition node.

\[\boldsymbol{x}_{\mathrm{next}}=\boldsymbol{x}+f(\boldsymbol{x})\]

Even a linear residual block starts from a different reference point

Replace the representation

\[\boldsymbol{x}_{\mathrm{next}}=W\boldsymbol{x}\]

Preserve the input: learn \(W=I\).

Near \(W=0\), the output is near zero.

Update the representation

\[\boldsymbol{x}_{\mathrm{next}}=(I+W)\boldsymbol{x}\]

Preserve the input: learn \(W=0\).

Near \(W=0\), the update is small.

Stacking linear residual blocks still gives a linear map. The change is how we parameterize learning, not the linear model’s expressivity.

The shortcut supplies a direct gradient contribution

For the additive block \(\boldsymbol{y}=\boldsymbol{x}+f(\boldsymbol{x})\):

\[\frac{\partial\boldsymbol{y}}{\partial\boldsymbol{x}} =I+\frac{\partial f(\boldsymbol{x})}{\partial\boldsymbol{x}}.\]

\[\frac{\partial L}{\partial\boldsymbol{x}} =\frac{\partial L}{\partial\boldsymbol{y}} +\left(\frac{\partial f(\boldsymbol{x})}{\partial\boldsymbol{x}}\right)^{\!T}\frac{\partial L}{\partial\boldsymbol{y}}.\]

The gradient has a path that does not multiply through the learned branch.

This helps explain deep residual networks; it does not guarantee stable gradients or better generalization.

Batch normalization uses one set of statistics per channel

BatchNorm pools one channel over batch and spatial dimensions, normalizes values, and applies a learned channel scale and offset.

\[y_{bchw}=\gamma_c\frac{x_{bchw}-\mu_c}{\sqrt{v_c+\epsilon}}+\beta_c\]

Normalization layers differ in which values share statistics

Layer	Values grouped for each normalization
BatchNorm	One channel across examples and spatial positions
InstanceNorm	One channel’s spatial positions within one example
GroupNorm	A group of channels and spatial positions within one example
LayerNorm	Chosen trailing feature dimensions within one example

ResNet-18 uses BatchNorm. The U-Net example uses InstanceNorm.

The other three need no cross-example statistics. LayerNorm will return when we discuss transformers.

Pooling either reduces a map or summarizes each channel

Local max pooling turns a four-by-four map into two-by-two local maxima. Global average pooling gives one average for each channel.

ResNet uses both: local pooling near the input; global averaging before the classification head.

ResNet-18 uses smaller spatial maps and more channels

Annotated ResNet-18 architecture. RGB 224-square input passes through convolution, normalization and pooling, four residual stages with 64 to 512 channels, global average pooling and a linear classifier. An inset shows the basic block and shape-changing shortcut.

Original illustration of torchvision ResNet-18; dimensions omit the batch axis.