Convolutional Neural Networks (CNN) Interview Preparation Guide

The output spatial dimension (Height or Width) is calculated using the formula: $O = \lfloor \frac{W - K + 2P}{S} \rfloor + 1$, where $W$ represents the input spatial dimension, $K$ is the kernel size, $P$ is the padding amount, and $S$ is the stride length. Understanding this formula is essential for interview questions regarding tensor shape alignment and preventing runtime shape mismatch exceptions between convolutional blocks and fully connected classification heads.

During the backward pass, max pooling routes incoming gradient signals exclusively to the specific spatial index within the pooling window that held the maximum activation value during the forward pass, setting gradients for all other indices to zero. In contrast, average pooling distributes incoming gradients equally across all elements within the pooling window. This makes max pooling act as a selective spatial feature selector, while average pooling smooths spatial gradients across the entire neighborhood.

Residual skip connections introduce an identity shortcut that bypasses one or more convolutional layers, allowing the input tensor to be added directly to the block output ($F(x) + x$). During backpropagation, the derivative of the loss with respect to the input contains an additive identity term ($1 + \frac{\partial F}{\partial x}$). This ensures that even if the convolutional weights' gradients vanish toward zero, the additive '1' allows gradient signals to propagate unimpeded directly back to the earliest layers of the network.

A standard convolution applies a multi-channel kernel across all input channels simultaneously, coupling spatial and channel-wise feature extraction. A depthwise separable convolution factorizes this into two separate steps: a depthwise convolution that applies a single 2D spatial filter independently to each input channel, followed by a pointwise 1x1 convolution that linearly combines channels. This factorization reduces both parameter counts and FLOPs by roughly a factor of the kernel area (e.g., 8x to 9x reduction for 3x3 kernels), making it a staple for mobile and edge vision architectures.

In PyTorch, certain layers like Batch Normalization and Dropout behave fundamentally differently during training versus evaluation. Batch normalization normalizes activations using batch mean and variance during training while updating running exponential moving averages, but uses those stored running statistics during inference. If `model.eval()` is omitted, batch normalization continues calculating statistics over small inference batches or test samples, leading to unstable activations, severe degradation in model accuracy, and unpredictable predictions.

Atrous convolution inserts configurable spaces—known as dilation rates—between the kernel elements. For example, a dilation rate of 2 expands a 3x3 kernel to effectively cover a 5x5 spatial area by spacing out kernel weights with zeros. This allows the convolutional layer to capture broader contextual semantic information over a wider receptive field without adding extra trainable parameters or increasing spatial resolution downsampling.

FP32 precision maintains maximum numerical representation range and model accuracy, but consumes higher memory bandwidth and VRAM while executing slower on hardware tensor cores. INT8 post-training quantization compresses model weights and activations into 8-bit integers, drastically accelerating inference throughput and reducing memory footprints by 4x. However, if early layers have extreme activation outliers, INT8 quantization can introduce precision clipping errors that cause noticeable drops in top-1 classification accuracy.

Transposed convolutions perform upsampling by inserting zero-padding between input elements before applying standard kernel convolutions. Because the stride is smaller than the kernel size, overlapping receptive fields cause uneven contribution weights across adjacent pixels in the output feature map. Specifically, pixels touched by multiple kernel overlaps receive higher values than adjacent pixels, creating a visible periodic grid or checkerboard pattern. This is typically mitigated by replacing transposed convolutions with explicit nearest-neighbor or bilinear upsampling followed by a standard convolution.

Global Average Pooling takes each 2D spatial feature map (of dimensions Height x Width) from the final convolutional layer and computes its arithmetic mean across all spatial locations, reducing it to a single scalar value per channel. This converts an entire spatial feature map into a 1D vector whose length equals the number of channels. Structurally, it eliminates thousands or millions of fully connected parameters that are highly prone to overfitting, while also making the CNN invariant to input image spatial dimensions.

Winograd minimal filtering is an algorithmic optimization technique that transforms input tensor tiles and 3x3 filter kernels into a transformed mathematical domain where convolutions can be computed using fewer multiplications at the cost of additional minor additions. Because floating-point multiplications are significantly more computationally expensive on hardware than additions, Winograd transforms reduce the theoretical multiplication count for 3x3 convolutions (such as F(2x2, 3x3)) by 4x, drastically accelerating GPU tensor core throughput.

While weight magnitude (such as L1 norm) often correlates with feature importance, pruning channels with low weights assumes independent feature contributions. In reality, deep CNNs rely heavily on distributed representations where seemingly inactive or low-magnitude channels carry critical high-frequency gradient information or act as corrective offsets for other channels. Pruning them abruptly can disrupt interdependent feature pathways, leading to catastrophic accuracy drops that require fine-tuning or quantization-aware recovery.

Data parallelism replicates the entire CNN model across multiple GPUs, splits incoming mini-batches evenly across devices, computes forward and backward passes independently, and synchronizes gradients using an AllReduce operation before optimizer steps. Tensor parallelism, by contrast, splits individual layers (such as large convolutional filter banks or linear weights) across multiple GPUs, requiring inter-device communication during both the forward and backward passes to compute localized tensor slices collaboratively.