Deep Learning MCQ Questions And Answers

Deep Learning MCQ Questions And Answers

31. What is the difference between deep learning and machine learning?

  1. Deep learning and machine learning are identical — the terms are used interchangeably
  2. Machine learning uses algorithms that learn from structured data and often require manual feature engineering; deep learning is a subset of machine learning that uses multi-layered neural networks to automatically learn hierarchical feature representations from raw data — excelling especially with unstructured data like images, audio, and text
  3. Machine learning is more powerful than deep learning for all types of data and tasks
  4. Deep learning requires less data than traditional machine learning algorithms to achieve good results

Answer : B
Explanation: Machine Learning: algorithms like decision trees, SVMs, random forests learn from structured/tabular data. Require manual feature engineering — humans design the input features. Work well with small-to-medium datasets. Interpretable (you can understand why a prediction was made). Examples: predicting house prices (regression), spam detection (classification). Deep Learning: uses neural networks with many hidden layers. Automatically learns features at multiple levels of abstraction from raw data — no manual feature engineering needed. Requires large amounts of data and compute (GPUs). Excels at unstructured data: images, speech, text, video. Less interpretable (black box). Key advantage: given enough data, deep learning can discover features humans never thought to engineer, often surpassing human performance on perception tasks.

32. What is backpropagation in deep learning?

  1. Backpropagation is the process of feeding data backward through the network from output to input
  2. Backpropagation is an algorithm that calculates gradients of the loss function with respect to each weight using the chain rule of calculus — allowing the optimizer to update weights in the direction that reduces the loss
  3. Backpropagation is a technique for validating a model by testing it on backward-ordered data
  4. Backpropagation is only used during the inference phase when making predictions on new data

Answer : B
Explanation: Backpropagation (backprop) is the foundation of all neural network training. Process: Forward pass: input data flows through the network, producing a prediction. Loss computation: the loss function measures how wrong the prediction is. Backward pass: gradients of the loss are computed layer by layer from output back to input using the chain rule. ∂L/∂w = ∂L/∂a × ∂a/∂z × ∂z/∂w (for each weight w). Weight update: each weight updated: w = w – α × ∂L/∂w (gradient descent). Repeat for many batches and epochs. Key insight: the chain rule allows efficient computation of all gradients in one backward pass — without it, training deep networks would be computationally infeasible. David Rumelhart, Geoffrey Hinton, and Ronald Williams popularized backprop for neural networks in 1986. Modern frameworks (PyTorch, TensorFlow) implement automatic differentiation (autograd) that performs backpropagation automatically.

33. What is the vanishing gradient problem in deep learning and how is it solved?

  1. Vanishing gradient means the model’s loss function disappears, making training impossible
  2. The vanishing gradient problem occurs when gradients become extremely small as they are backpropagated through many layers — causing early layers to learn very slowly or not at all; solved using ReLU activation, batch normalization, residual connections, and proper weight initialization
  3. Vanishing gradient is a hardware problem caused by insufficient GPU memory during training
  4. The vanishing gradient only occurs in shallow networks and is not an issue in deep networks

Answer : B
Explanation: In deep networks with many layers, gradients are multiplied together during backpropagation. When using activation functions like sigmoid or tanh (derivatives < 1), this multiplication shrinks gradients exponentially toward zero in early layers — those layers stop learning. Solutions: ReLU activation: derivative = 1 for positive inputs — doesn't shrink gradients. Batch Normalization: normalizes activations, keeps gradients healthy. Residual Connections (ResNet): skip connections allow gradients to flow directly without vanishing. LSTM/GRU gates: designed specifically for long-range gradient flow in RNNs. Weight Initialization: He initialization (ReLU), Xavier/Glorot (tanh, sigmoid) — keeps activations in healthy range from the start. Gradient Clipping: prevents both vanishing and exploding gradients. The vanishing gradient problem was a key barrier to training deep networks in the 1990s-2000s. ResNets (He et al., 2015) solved it for very deep networks by introducing skip connections.

34. What is the role of activation functions in deep learning?

  1. Activation functions activate (turn on) the entire neural network at the start of training
  2. Activation functions introduce non-linearity into the network — without them, stacking multiple layers would be mathematically equivalent to a single linear transformation, making the network unable to learn complex patterns
  3. Activation functions are used only in the output layer to produce the final prediction probability
  4. Activation functions reduce the learning rate automatically as training progresses

Answer : B
Explanation: Without activation functions, a neural network is just a sequence of matrix multiplications — no matter how many layers, the result is still a linear function of the input. Linear functions cannot model complex relationships like image recognition. Activation functions introduce non-linearity, allowing deep networks to approximate any function. Key activation functions: ReLU: f(x) = max(0,x) — default for hidden layers. Fast, no vanishing gradient for positive inputs. Sigmoid: f(x) = 1/(1+e^-x) — output 0-1 — used for binary classification output. Tanh: f(x) = (e^x – e^-x)/(e^x + e^-x) — output -1 to 1 — zero-centered. Softmax: converts logits to probabilities summing to 1 — used for multi-class output. Leaky ReLU: allows small gradients for negative inputs — prevents dying ReLU. GELU: used in Transformers (BERT, GPT). Swish: x × sigmoid(x) — used in EfficientNet. Choosing the right activation function is crucial for training stability and performance.

35. What is batch normalization in deep learning and why is it used?

  1. Batch normalization normalizes the size of training batches to ensure consistent batch sizes
  2. Batch normalization normalizes the activations of each layer within a mini-batch to have approximately zero mean and unit variance — accelerating training, enabling higher learning rates, reducing sensitivity to initialization, and acting as a regularizer
  3. Batch normalization is a data preprocessing technique applied before training begins
  4. Batch normalization is only useful in convolutional neural networks and not in other architectures

Answer : B
Explanation: Batch Normalization (BN), introduced by Ioffe and Szegedy (2015), is one of the most impactful techniques in deep learning. How it works: for each mini-batch, compute mean (μ) and variance (σ²) of activations. Normalize: x̂ = (x – μ)/√(σ²+ε). Scale and shift with learnable parameters γ and β: y = γx̂ + β. Benefits: Reduces internal covariate shift — keeps activations in a stable range throughout training. Enables higher learning rates — training converges faster. Reduces sensitivity to weight initialization. Mild regularization effect — reduces need for dropout in many architectures. Placement: typically after the linear/conv layer and before the activation function (or sometimes after). Alternatives: Layer Normalization (used in Transformers) — normalizes across features for one sample. Instance Normalization (used in style transfer). Group Normalization (stable for small batch sizes). BN is now standard in almost all CNN architectures.

36. What is the difference between CNN and RNN architectures?

  1. CNNs process text data; RNNs process image data — they are designed for opposite data types
  2. CNNs use convolutional filters to process spatial data like images — capturing local patterns through weight sharing; RNNs process sequential data by maintaining hidden states that carry information across time steps — capturing temporal dependencies
  3. CNNs require more training data than RNNs for all tasks regardless of input type
  4. RNNs are a subset of CNNs and share the same fundamental architectural principles

Answer : B
Explanation: CNN (Convolutional Neural Network): designed for spatial data with grid-like structure (images, video). Key feature: convolutional layers use shared filters that scan the input to detect local features (edges, textures, shapes). Pooling layers reduce spatial dimensions. Applications: image classification, object detection, face recognition, medical image analysis. Architecture: Conv → Pool → Conv → Pool → Flatten → FC → Output. RNN (Recurrent Neural Network): designed for sequential data where order matters (text, speech, time series, video). Key feature: hidden state carries information from previous time steps — the network has “memory.” Same weights used at each time step (parameter sharing over time). Applications: language modeling, machine translation, speech recognition, sentiment analysis, time series forecasting. Main RNN variants: LSTM and GRU (better long-range memory). Modern trend: Transformers have largely replaced RNNs for NLP due to parallelism, but RNNs still have applications in time series and streaming data.

37. What is the Transformer architecture in deep learning?

  1. The Transformer is a type of CNN that transforms feature maps between different spatial resolutions
  2. The Transformer is a deep learning architecture based entirely on self-attention mechanisms — processing entire sequences in parallel rather than sequentially, enabling better long-range dependency modeling and massive parallelization for training on large datasets
  3. A Transformer is a hardware component that transforms electrical power for GPU training clusters
  4. Transformers are only applicable to computer vision tasks and cannot be used for NLP

Answer : B
Explanation: The Transformer, introduced in “Attention Is All You Need” (Vaswani et al., Google, 2017), revolutionized deep learning. Key innovations: Self-attention: every token attends to every other token, capturing long-range dependencies regardless of distance. Multi-head attention: multiple attention operations in parallel, each learning different relationship types. Positional encoding: since there’s no recurrence, position information added explicitly. Encoder-Decoder structure: original Transformer for translation. BERT uses encoder only. GPT uses decoder only. Advantages over RNNs: Parallelism: entire sequence processed simultaneously — much faster training. Better long-range dependencies: attention provides direct connections between any two tokens. Scalability: scales extremely well with more data and compute. Impact: foundation of all modern large language models: BERT, GPT-2/3/4, T5, LLaMA, Gemini, Claude. Also applied to vision: Vision Transformer (ViT), DINO. Audio: Whisper, AudioLM. Multi-modal: CLIP, Flamingo, GPT-4V. The Transformer is arguably the most important deep learning architecture of the 2020s.

38. What is LSTM (Long Short-Term Memory) and what problem does it solve?

  1. LSTM is a type of memory chip used in GPUs to speed up deep learning computations
  2. LSTM is a specialized recurrent neural network architecture with gates (input, forget, output) and a cell state — designed to solve the vanishing gradient problem in standard RNNs by selectively retaining and forgetting information across long sequences
  3. LSTM is a loss function used to measure long-term prediction accuracy in time series models
  4. LSTM is a technique for compressing long training sequences to improve computational efficiency

Answer : B
Explanation: LSTM (Long Short-Term Memory), developed by Hochreiter and Schmidhuber (1997), addresses the limitation of standard RNNs which fail to learn long-range dependencies due to vanishing gradients. Key components: Cell state (Ct): the “memory highway” — gradients can flow unchanged over many time steps. Forget gate: decides what to erase from cell state. σ(Wf·[ht-1, xt] + bf) — output 0-1, where 0 = forget, 1 = keep. Input gate: decides what new information to add. Output gate: decides what part of the cell state to expose as the hidden state ht. Why it works: gating mechanism creates paths where gradients can flow without vanishing. Applications: machine translation, speech recognition, text generation, music composition, time series forecasting, video captioning. Limitation vs Transformers: LSTMs process sequentially — cannot parallelize. Transformers have largely replaced LSTMs for NLP tasks, but LSTMs remain relevant for streaming/online learning scenarios.

39. What is the difference between GRU and LSTM?

  1. GRU is a more complex version of LSTM with additional gates for better performance
  2. GRU (Gated Recurrent Unit) is a simplified variant of LSTM with two gates (reset and update) instead of three — achieving comparable performance with fewer parameters, faster training, and better performance on smaller datasets
  3. GRU uses convolutional layers while LSTM uses recurrent layers for sequence processing
  4. GRU and LSTM are identical architectures with different naming conventions

Answer : B
Explanation: GRU (Gated Recurrent Unit), introduced by Cho et al. (2014), is a streamlined alternative to LSTM. GRU gates: Update gate: combines LSTM’s forget and input gates — decides how much past information to keep vs. new information. Reset gate: controls how much past hidden state to use when computing the new candidate hidden state. Key differences from LSTM: No separate cell state — uses only hidden state. Fewer parameters → faster training, less data needed to generalize. Often matches LSTM performance on many tasks. When to use GRU vs LSTM: GRU: shorter sequences, smaller datasets, faster inference needed. LSTM: long-range dependencies, complex tasks, larger datasets. In practice: performance is often similar — experiment with both. Both have been largely superseded by Transformers for NLP tasks, but GRU is still widely used for time series forecasting, IoT sensor data, and applications requiring low latency.

40. What is the attention mechanism in deep learning?

  1. Attention is a technique that increases computational power directed at difficult training examples
  2. The attention mechanism allows a model to dynamically focus on the most relevant parts of the input when producing each output — computing a weighted sum of input representations where weights reflect relevance, enabling the model to handle long-range dependencies efficiently
  3. Attention is a type of pooling layer that selectively pools the most attended (activated) features
  4. The attention mechanism is only used in the final output layer of deep learning models

Answer : B
Explanation: Attention, introduced by Bahdanau et al. (2014) for neural machine translation, allows models to selectively focus on relevant input parts when generating each output token. How it works: for each output position, compute attention scores between that position and all input positions. Softmax normalizes scores into attention weights (summing to 1). Output = weighted sum of input representations using attention weights. Self-Attention (Transformers): each position attends to every other position in the same sequence. Computed with Query (Q), Key (K), Value (V) matrices: Attention(Q,K,V) = Softmax(QKᵀ/√dk)V. Multi-Head Attention: run attention h times in parallel with different projections — captures different relationship types. Types: Local attention (attend to nearby tokens — efficient). Cross-attention (attend to encoder outputs in decoder). Sparse attention (attend to subset of tokens — efficient for long sequences). Applications: machine translation (align source and target words), summarization, question answering, image captioning. Attention is the single most important building block of modern AI systems.

Pages: 1 2 3 4 5 6 7 8 9 10