Neural Networks MCQ Questions And Answers

Neural Networks MCQ Questions And Answers

31. What is a perceptron in neural networks?

  1. A type of recurrent neural network used for sequential data processing
  2. The simplest form of a neural network — a single-layer binary classifier that takes weighted inputs, applies a step activation function, and outputs a binary decision (0 or 1)
  3. A multi-layered deep neural network used for complex image recognition tasks
  4. A clustering algorithm that groups similar data points into perceptron classes

Answer : B
Explanation: The Perceptron, introduced by Frank Rosenblatt in 1957, is the foundational building block of neural networks. It computes: output = 1 if (Σ wᵢxᵢ + bias) ≥ threshold, else 0. It can only learn linearly separable problems — famously unable to solve the XOR problem (proved by Minsky and Papert in 1969, temporarily halting NN research). A Single-Layer Perceptron has one input layer and one output layer. A Multilayer Perceptron (MLP) adds hidden layers, enabling it to learn non-linear decision boundaries. The Perceptron Learning Rule updates weights only when a prediction is wrong — a precursor to modern gradient descent. Understanding the perceptron is the starting point for all neural network interview questions.

32. What is the vanishing gradient problem in neural networks?

  1. A problem where neural network gradients become too large, causing unstable training
  2. A problem where gradients become extremely small as they are backpropagated through many layers — making earlier layers learn very slowly or stop learning entirely, commonly occurring with sigmoid and tanh activation functions in deep networks
  3. A memory problem where stored gradient values disappear from GPU memory during training
  4. A problem where the gradient descent optimizer cannot find the minimum loss value

Answer : B
Explanation: The Vanishing Gradient Problem occurs because backpropagation multiplies gradients layer by layer through the chain rule. When activation functions like sigmoid or tanh are used, their derivatives are always less than 1 (sigmoid derivative max = 0.25). Multiplying many small numbers together produces a vanishingly small gradient in early layers — those layers receive almost no learning signal. Solutions: ReLU activation (derivative = 1 for positive inputs, doesn’t saturate). Batch Normalization (normalizes activations, keeps gradients healthy). Residual Connections/Skip Connections (ResNet — adds shortcuts that allow gradients to flow directly). LSTM/GRU gates (designed specifically to handle long-range dependencies). Weight initialization techniques (He initialization for ReLU, Xavier/Glorot for tanh). This is one of the most frequently tested neural network concepts.

33. What is the exploding gradient problem in neural networks?

  1. The opposite of vanishing gradient — gradients grow exponentially large during backpropagation, causing weight updates to become extremely large and the model to diverge or produce NaN values
  2. A problem where the neural network’s output activations become too large during forward pass
  3. A GPU memory overflow that occurs when batch size is too large during training
  4. A data preprocessing problem where input features have very large numerical ranges

Answer : A
Explanation: The Exploding Gradient Problem occurs when gradients accumulate multiplicatively through layers, growing exponentially large — causing weight updates that are far too large and pushing the model into an unstable, divergent state. Symptoms: loss suddenly becomes NaN or infinity, model accuracy collapses during training. More common in RNNs (which unroll through many time steps) than in feedforward networks. Solutions: Gradient Clipping — cap gradients at a maximum value (e.g., if ||gradient|| > threshold, rescale it). Proper weight initialization (prevent initial weights from being too large). LSTM/GRU (gates control information flow, preventing runaway gradients). Batch normalization. Lower learning rate. Gradient clipping is the most direct solution and is implemented in all major deep learning frameworks (torch.nn.utils.clip_grad_norm_ in PyTorch).

34. What is gradient descent in neural network training?

  1. An algorithm that randomly searches the parameter space for the minimum loss value
  2. An optimization algorithm that iteratively updates model weights in the direction of the negative gradient of the loss function — moving toward the minimum loss step by step using the learning rate
  3. A data preprocessing technique that scales input features to the range [0, 1]
  4. A search algorithm used to find the best neural network architecture for a given task

Answer : B
Explanation: Gradient Descent is the core optimization algorithm for training neural networks. Weight update rule: w = w – α × ∂L/∂w, where α is the learning rate and ∂L/∂w is the gradient of the loss with respect to the weight. Types: Batch GD — uses the entire dataset per update (slow, stable). Stochastic GD (SGD) — uses one sample per update (fast, noisy). Mini-batch GD — uses small batches (most commonly used, balances speed and stability). The learning rate α is critical: too large → overshoots minimum, diverges; too small → extremely slow convergence. Modern optimizers improve upon basic GD: Momentum (accelerates in the right direction), Adam (adaptive learning rates per parameter), RMSprop (adapts learning rate based on recent gradient magnitudes). Adam is the default choice for most deep learning tasks.

35. What is the role of the learning rate in neural network training?

  1. The learning rate determines how many layers the neural network learns during training
  2. The learning rate is a hyperparameter that controls the size of weight updates during gradient descent — too high causes divergence, too low causes extremely slow convergence or getting stuck in local minima
  3. The learning rate controls how quickly new data samples are loaded during mini-batch training
  4. The learning rate determines the percentage of neurons that are active during each forward pass

Answer : B
Explanation: The Learning Rate (α) is perhaps the most important hyperparameter in neural network training. It determines the step size taken in the direction of the negative gradient. Too high: weight updates overshoot the minimum — loss oscillates or diverges. Too low: training takes extremely long; may get stuck in local minima or plateaus. Finding the right learning rate: Learning Rate Range Test (train for a few iterations with increasing LR, pick where loss decreases fastest). Learning Rate Scheduling: start high, decrease over time (step decay, cosine annealing, warm restarts). Cyclical Learning Rates: oscillate between min and max LR. Adaptive optimizers (Adam, AdaGrad, RMSprop) automatically adjust the effective learning rate per parameter. Typical values: 0.1 for SGD, 0.001 for Adam. The learning rate is the first hyperparameter to tune when a model fails to converge.

36. What is the difference between a shallow neural network and a deep neural network?

  1. Shallow networks train faster but always produce less accurate results than deep networks
  2. A shallow neural network has one or very few hidden layers; a deep neural network has many hidden layers (typically 3+) enabling it to learn hierarchical, abstract feature representations from raw data automatically
  3. Deep neural networks require less training data than shallow networks for the same task
  4. Shallow and deep neural networks are identical — the distinction is only semantic

Answer : B
Explanation: The depth of a neural network refers to the number of hidden layers. Shallow network (1-2 hidden layers): learns simple feature representations, limited in what functions it can approximate, trains quickly, needs less data. Deep network (3+ hidden layers): learns hierarchical features — early layers detect edges, middle layers detect shapes, deep layers detect complex objects. The Universal Approximation Theorem states that even a single hidden layer with enough neurons can approximate any function — but depth provides efficiency: tasks requiring exponentially more neurons in a shallow network can be solved with far fewer neurons when depth is added. Deep networks power modern AI breakthroughs: ResNet has 152 layers, GPT-4 has 96 transformer layers. The term “Deep Learning” specifically refers to using deep (multi-layer) neural networks.

37. What is the sigmoid activation function and what are its limitations?

  1. An activation function that outputs values between -1 and 1, used primarily in output layers of regression models
  2. An S-shaped activation function that maps any input to a value between 0 and 1 — used for binary classification output layers, but suffers from vanishing gradients and non-zero-centered outputs in hidden layers
  3. An activation function that outputs exactly 0 or 1 with no gradient for backpropagation
  4. A linear activation function that doubles the input value and clips it between 0 and 1

Answer : B
Explanation: The Sigmoid function σ(x) = 1/(1+e⁻ˣ) produces an S-shaped curve, outputting values between 0 and 1. Key limitations: Vanishing Gradient — sigmoid derivative (σ(x)(1-σ(x))) has a maximum of 0.25. For inputs far from 0, the derivative ≈ 0, causing vanishing gradients in deep networks. Not zero-centered — outputs are always positive (0 to 1). This means gradients are always the same sign for weights in a layer, causing zig-zag gradient updates (less efficient). Computationally expensive — computing e^x is slow compared to ReLU. Where sigmoid is still appropriate: binary classification output layer (output = probability of class 1), and LSTM/GRU gates (where values must be between 0 and 1). For hidden layers, ReLU and its variants are strongly preferred over sigmoid in modern deep learning.

38. What is the ReLU (Rectified Linear Unit) activation function and why is it preferred in deep networks?

  1. An activation function that outputs random values during training to improve regularization
  2. An activation function defined as f(x) = max(0, x) — outputs the input directly if positive, zero otherwise — preferred because it is computationally efficient, does not saturate for positive values, and dramatically reduces the vanishing gradient problem
  3. A function that recursively applies linear transformations to capture non-linear patterns
  4. An activation function that reinforces learning by increasing output values exponentially

Answer : B
Explanation: ReLU (Rectified Linear Unit): f(x) = max(0, x). Why preferred: Gradient = 1 for x > 0 (no vanishing gradient for positive activations). Computationally trivial (just a threshold at 0). Creates sparse activation (typically 50% of neurons output 0) — more efficient representation. Key weakness: Dying ReLU — neurons that always receive negative input permanently output 0 and have zero gradient — they “die” and stop learning. Solutions: Leaky ReLU: f(x) = x if x > 0, else 0.01x (small gradient for negative inputs prevents dying). Parametric ReLU (PReLU): learnable slope for negative inputs. ELU (Exponential Linear Unit): smooth, non-zero mean output. GELU (Gaussian Error Linear Unit): used in Transformers (BERT, GPT). Swish (x × sigmoid(x)): used in EfficientNet. ReLU and its variants are the most important activation functions in modern deep learning.

39. What is the tanh (hyperbolic tangent) activation function?

  1. A trigonometric function that maps inputs to values between 0 and π
  2. An S-shaped activation function similar to sigmoid but outputs values between -1 and 1, making it zero-centered — preferred over sigmoid for hidden layers but still suffers from vanishing gradients
  3. A tangential activation function that computes the angle of gradient descent
  4. An activation function used exclusively in the output layer of regression neural networks

Answer : B
Explanation: The tanh function: f(x) = (e^x – e^(-x))/(e^x + e^(-x)), outputs values between -1 and 1. Advantages over sigmoid: Zero-centered output (outputs range from -1 to +1), which means gradients can be positive or negative — enabling more efficient gradient updates than sigmoid’s always-positive outputs. Stronger gradients than sigmoid (tanh derivative maximum = 1, vs sigmoid maximum = 0.25). Limitations: Still suffers from vanishing gradients for very large or small inputs (derivatives ≈ 0 at extremes). Computationally more expensive than ReLU. Where tanh is used: Hidden layers (preferable to sigmoid but worse than ReLU for most tasks), LSTM and GRU gates (tanh is used for cell state and output), RNNs, and natural language processing models. Tanh is a standard choice in RNN architectures where zero-centered outputs are beneficial.

40. What is the Softmax activation function and when is it used?

  1. A smooth version of the ReLU function used in hidden layers of classification networks
  2. An activation function applied to the output layer of multi-class classification models that converts raw scores (logits) into a probability distribution where all outputs sum to 1
  3. A function that softly maximizes the highest activation while zeroing all others
  4. An activation function that normalizes hidden layer activations for batch normalization

Answer : B
Explanation: Softmax: σ(zᵢ) = e^zᵢ / Σⱼe^zⱼ — exponentiates each logit and normalizes by their sum, producing probabilities between 0 and 1 that sum to exactly 1. Example: logits [2.0, 1.0, 0.1] → Softmax → [0.659, 0.242, 0.099] — the model is 65.9% confident it is class 0. Properties: Emphasizes differences between outputs (the largest logit gets disproportionately high probability), differentiable (compatible with backpropagation), and produces interpretable probability outputs. Used with Cross-Entropy Loss during training. For binary classification: Sigmoid on a single output is equivalent to Softmax on two outputs. For multi-class: always use Softmax on final layer. Alternative: when using PyTorch’s CrossEntropyLoss, Softmax is applied internally — don’t add it manually. Softmax is one of the most important and commonly tested neural network concepts in interviews.

Pages: 1 2 3 4 5 6 7 8 9 10