Answer : B
Explanation: Understanding these terms is fundamental: Epoch: one complete pass through the entire training dataset. Training for 100 epochs means the model sees each training example 100 times. More epochs → more learning (but risk of overfitting). Batch (Mini-batch): a subset of the training data processed together in one forward and backward pass. Batch size is typically 32, 64, 128, or 256. Larger batches → more stable gradients, require more memory. Iteration: one forward pass + one backward pass + one weight update using one batch. Relationship: if you have 1000 training samples and a batch size of 100, one epoch = 10 iterations. Total iterations = epochs × (dataset_size / batch_size). Training terminology in practice: “training for 50 epochs with batch size 32 on 50,000 images” = 50 × (50,000/32) = 78,125 total weight updates. These concepts appear in every neural network implementation and interview.

Answer : B
Explanation: Parameters: internal model variables learned from data through gradient descent. Include: weights (W) and biases (b) in every layer. A ResNet-50 has ~25 million parameters. GPT-3 has 175 billion parameters. Not set manually — they emerge from training. Hyperparameters: external configuration settings set before training by the practitioner. Architectural: number of layers, neurons per layer, type of activation function, dropout rate, filter size (CNN). Training: learning rate, batch size, number of epochs, optimizer type (Adam, SGD, RMSprop), momentum, weight decay, learning rate schedule. Hyperparameter Tuning methods: Manual search (experienced practitioners), Grid Search (try all combinations — slow), Random Search (sample randomly — often better than grid), Bayesian Optimization (intelligent search using surrogate model), Neural Architecture Search (AutoML — uses ML to find best architecture). Proper hyperparameter tuning is critical — it can be the difference between 60% and 95% accuracy on the same dataset.

Answer : B
Explanation: Regularization reduces overfitting by constraining model complexity: L1 Regularization (Lasso): adds λΣ|wᵢ| to the loss. Encourages sparsity — drives some weights exactly to zero. Acts as automatic feature selection. L2 Regularization (Ridge/Weight Decay): adds λΣwᵢ² to the loss. Shrinks all weights toward zero but rarely to exactly zero. Most common in deep learning — called “weight decay” in optimizer settings. Dropout: randomly zeros neurons during training (covered in Q48). Elastic Net: combines L1 and L2. Early Stopping: stop training when validation loss starts increasing — most practical form of regularization. Data Augmentation: creates more training data artificially. Batch Normalization: mild regularization effect. Max-Norm Constraints: limit maximum norm of weight vectors. The λ hyperparameter controls regularization strength — too high → underfitting, too low → overfitting. In PyTorch, L2 regularization is implemented as weight_decay parameter in the optimizer.

Answer : B
Explanation: The Universal Approximation Theorem (Cybenko, 1989; Hornik, 1991) establishes the theoretical power of neural networks. It states that a single hidden layer neural network with a finite number of neurons can approximate any continuous function f: Rⁿ → Rᵐ (on a bounded domain) to arbitrary precision — given enough hidden neurons. Important nuances: It guarantees existence of such a network, but doesn’t tell you how to find the weights (training is still required). It doesn’t guarantee that learning algorithms will find the approximation. It doesn’t say how many neurons are needed (could be astronomically many for complex functions). Depth matters: the theorem applies to single hidden layer networks, but deep networks are exponentially more efficient — they can approximate the same functions with far fewer neurons. Why it matters: It provides the theoretical justification for using neural networks for any supervised learning task — they are universal function approximators.

Answer : B
Explanation: The loss landscape of a neural network is a high-dimensional surface. Local Minimum: a point where loss is lower than all immediately surrounding points, but not the global lowest value — gradient = 0 but there are lower valleys elsewhere. Global Minimum: the absolute lowest loss value across all possible weight configurations. Saddle Points: gradient = 0, but it is a minimum in some directions and a maximum in others — more common than local minima in high-dimensional spaces. Key insight from research: in deep learning, most local minima are approximately as good as the global minimum — the “loss surface” in high dimensions has few poor local minima. The main obstacles are saddle points (where gradient descent can get stuck) and plateaus (flat regions with very small gradients). Solutions: Momentum (helps escape saddle points), Stochastic gradient (noise helps escape local optima), Learning rate schedules, Multiple random restarts. Modern deep learning has largely moved past worrying about local vs. global minima — the real challenge is generalization.

Answer : B
Explanation: Adam (Adaptive Moment Estimation), introduced by Kingma and Ba (2014), is the most widely used optimizer in deep learning. Algorithm: mₜ = β₁mₜ₋₁ + (1-β₁)gₜ (first moment — mean of gradients, like momentum). vₜ = β₂vₜ₋₁ + (1-β₂)gₜ² (second moment — mean of squared gradients, like RMSprop). Bias correction: m̂ₜ = mₜ/(1-β₁ᵗ), v̂ₜ = vₜ/(1-β₂ᵗ). Update: w = w – α × m̂ₜ/(√v̂ₜ + ε). Default parameters: β₁=0.9, β₂=0.999, ε=1e-8, α=0.001. Benefits: Adapts learning rate per parameter (parameters with rare updates get larger steps), handles sparse gradients well, bias correction for initial steps, generally converges faster than SGD. Variants: AdaW (Adam with decoupled weight decay — better generalization for Transformers), AMSGrad, Nadam. Adam is the default choice for most deep learning tasks — when in doubt, start with Adam (lr=0.001).

Answer : B
Explanation: Both pooling operations reduce spatial dimensions of feature maps, but capture different information: Max Pooling: takes the maximum value in each pooling window. Selects the strongest activation (most prominent feature detected). Provides translation invariance — small shifts in features don’t change the output. Discards spatial location of features within the window. Most commonly used in CNNs — better for detecting presence of features (e.g., is there an edge here?). Average Pooling: takes the mean value. Provides a smoother summary of features. Less translation invariant — small shifts affect the average. Used in Global Average Pooling (GAP) — averages each entire feature map to a single value, often used before final classification layer (replaces large FC layers, fewer parameters, less overfitting). In modern architectures: convolutional layers with stride > 1 are increasingly used instead of pooling (the stride effectively downsamples while maintaining gradient flow).

Answer : B
Explanation: Understanding the neuron’s computation is fundamental: Step 1 — Compute Net Input (pre-activation): z = w₁x₁ + w₂x₂ + … + wₙxₙ + b = Σwᵢxᵢ + b. This is a linear combination of all inputs weighted by their respective weights, plus a bias term. The bias allows the activation threshold to be shifted. Step 2 — Apply Activation Function: a = f(z), where f is the chosen non-linear function (ReLU, sigmoid, tanh, softmax). Without the activation function (f = identity), the neuron is purely linear — stacking many such neurons produces another linear function (no expressive power). With non-linear f, the network can approximate complex functions. This two-step process (linear combination → non-linear activation) is repeated for every neuron in every layer during the forward pass. During backpropagation, gradients are computed with respect to both the activation and the net input (using the chain rule and the derivative of the activation function).

Answer : B
Explanation: The Receptive Field of a neuron is the region of the input that can affect its output. In the first convolutional layer with a 3×3 filter: each neuron has a 3×3 receptive field — it “sees” only a 3×3 patch of the original image. After a second 3×3 convolutional layer: each neuron’s effective receptive field is 5×5 of the original image (because it aggregates 3×3 patches from the previous layer’s 3×3 patches). As depth increases, each neuron’s effective receptive field grows, allowing the network to capture larger and larger patterns. Deep CNNs eventually have receptive fields spanning the entire image. Techniques to increase receptive field efficiently: Strided convolutions (skip positions), Dilated/Atrous convolutions (add gaps between filter elements — same parameters, much larger receptive field), Pooling (reduces spatial size, increases receptive field). Larger receptive fields enable detection of global patterns; smaller receptive fields capture local textures and edges.

Answer : B
Explanation: Stride: the step size of the filter as it slides across the input. Stride = 1: filter moves one pixel at a time (default). Output size: (input_size – filter_size + 1). Stride = 2: filter jumps 2 pixels. Output size roughly halved. Larger stride → smaller output, larger effective receptive field, but may miss fine details. Used to replace pooling in modern architectures (strided convolutions). Padding: adds zeros around the input borders before convolution. ‘Valid’ padding (no padding): output shrinks after each convolution. ‘Same’ padding (zero-padding): output same size as input. Padding = (filter_size – 1)/2 for a filter of size 3×3 with stride 1 → add 1 pixel of zeros on each side. Output size formula: output = (input + 2×padding – filter_size)/stride + 1. Example: 28×28 input, 3×3 filter, stride=1, padding=1 → (28 + 2-3)/1 + 1 = 28×28 output (same padding preserves dimension). Understanding stride and padding is essential for computing CNN output dimensions.