Deep Learning MCQ Questions And Answers

Deep Learning MCQ Questions And Answers

81. What is deep learning’s role in natural language generation (NLG)?

  1. NLG in deep learning is only used for converting structured data tables into simple one-sentence summaries
  2. Deep learning has transformed natural language generation — enabling systems to produce coherent, contextually appropriate text through autoregressive language models that generate text one token at a time by predicting the most likely next word given all previous words
  3. Natural language generation using deep learning can only produce text in formats that exist in training data
  4. Deep learning NLG requires a human-approved outline before it can generate any text passage

Answer : B
Explanation: Natural Language Generation (NLG) covers any task where the model produces text output. Core mechanism: autoregressive generation: given tokens [x1, x2, …, xt], predict xt+1. Repeat: append xt+1 to the sequence and predict xt+2. Continue until end-of-sequence token or max length. Sampling strategies: Greedy: always pick the highest probability token. Beam search: explore multiple candidate sequences simultaneously. Temperature sampling: T < 1 → sharper distributions (more deterministic), T > 1 → softer (more creative). Top-k sampling: sample from the k highest probability tokens. Top-p (nucleus) sampling: sample from the smallest set of tokens whose cumulative probability ≥ p. NLG tasks: Machine translation, Text summarization (abstractive), Story generation, Dialogue systems / chatbots, Question answering (generative), Code generation (GitHub Copilot), Poetry and creative writing, Report generation from data, Email/document drafting. Modern systems: GPT-4: most capable general-purpose text generator. Claude: strong reasoning + long context. Gemini: multimodal generation. LLaMA, Mistral: open-source alternatives. Evaluation challenges: automatic metrics (BLEU, ROUGE, BERTScore) often don’t correlate with human judgment. Human evaluation remains gold standard for creative and conversational tasks.

82. What is mixed precision training in deep learning?

  1. Mixed precision training uses different activation functions in different layers of the same network
  2. Mixed precision training uses FP16 (half-precision) for most computations while maintaining FP32 (full-precision) for loss scaling and critical operations — reducing memory usage by ~50% and increasing training speed by 2-3× on modern GPUs without significant accuracy loss
  3. Mixed precision combines multiple different optimizers during training for better convergence
  4. Mixed precision training alternates between supervised and unsupervised learning within the same epoch

Answer : B
Explanation: Mixed Precision Training is standard practice for modern large-scale deep learning. Background: FP32 (single precision): 4 bytes per value, high precision. FP16 (half precision): 2 bytes per value, less precise but faster. BF16 (bfloat16): same exponent range as FP32, less mantissa — better for training than FP16. How it works: Forward pass: compute in FP16 for speed. Backward pass: compute in FP16 for speed. Loss scaling: multiply loss by a large scale factor before backward pass (prevents underflow in FP16 gradients). Weight updates: maintain master copy in FP32 (prevents precision loss in optimizer states). Benefits: ~2× memory reduction: fits larger batches or larger models in GPU memory. 2-3× speedup: modern GPU Tensor Cores optimized for FP16/BF16 operations. Better hardware utilization. NVIDIA A100: up to 312 TFLOPS (teraflops) for BF16 vs 77 TFLOPS for FP32. Required infrastructure: NVIDIA GPUs with Tensor Cores (Volta, Turing, Ampere, Hopper). Implementation: PyTorch: torch.cuda.amp.autocast() + GradScaler. TF: tf.keras.mixed_precision.set_global_policy(‘mixed_float16’). All modern large model training (GPT-4, LLaMA, etc.) uses mixed precision.

83. What is the concept of encoder-decoder architecture in deep learning?

  1. Encoder-decoder refers to encoding data before training and decoding results after training ends
  2. The encoder-decoder architecture maps an input sequence or image to a compact latent representation (encoder) and then generates an output sequence or image from that representation (decoder) — used for machine translation, image segmentation, image generation, and sequence-to-sequence tasks
  3. The encoder handles training while the decoder handles inference in all deep learning models
  4. Encoder-decoder is only applicable to text-to-text tasks and cannot be used for image processing

Answer : B
Explanation: Encoder-Decoder is one of the most versatile patterns in deep learning. Encoder: compresses the input into a compact, information-rich representation (latent/context vector). For images: CNN extracts feature maps of decreasing spatial resolution. For text: Transformer encoder processes sequence into contextual embeddings. For any input: captures essential information while abstracting details. Decoder: generates output from the encoded representation. For images: upsamples/convolves feature maps back to original or desired resolution. For text: Transformer decoder generates tokens autoregressively. Applications: Machine Translation: English encoder → German decoder (Transformer seq2seq). Image Segmentation: U-Net — CNN encoder (downsampling path) + CNN decoder (upsampling path) + skip connections. Image-to-Image Translation: Pix2Pix (edge map → photo), CycleGAN (horses ↔ zebras). Image Generation: VAE encoder → latent space → decoder generates images. Video Prediction: encode past frames, decode future frames. Speech Recognition: encode audio → decode text. Multimodal: image encoder → text decoder (image captioning). Skip connections (U-Net): directly connect encoder and decoder layers at same resolution — passes low-level spatial details to decoder. Key insight: the encoder-decoder design elegantly separates representation learning (encoder) from generation (decoder).

84. What is hyperparameter tuning in deep learning?

  1. Hyperparameter tuning is the process of adjusting model weights during the training process
  2. Hyperparameter tuning is the process of finding optimal configuration values (learning rate, batch size, architecture choices, regularization strength) that are set before training — using methods like grid search, random search, Bayesian optimization, or neural architecture search
  3. Hyperparameter tuning is a post-deployment activity to improve model performance on production data
  4. Hyperparameters are only the number of layers and neurons — all other settings are fixed by the framework

Answer : B
Explanation: Hyperparameters are set before training; parameters (weights/biases) are learned during training. Common hyperparameters: Training: learning rate, batch size, number of epochs, optimizer type, learning rate schedule. Architecture: number of layers, units per layer, kernel size, number of filters, dropout rate, activation function. Regularization: L1/L2 weight decay coefficient, dropout probability, data augmentation type/strength. Search methods: Manual search: expert intuition. Try common defaults: Adam lr=0.001, batch=32. Grid Search: try all combinations of candidate values. Exhaustive but exponentially expensive. Random Search: sample randomly from hyperparameter distributions. More efficient than grid search for high-dimensional spaces. Bayesian Optimization: build a surrogate model of performance vs. hyperparameters. Use acquisition function to select the most promising next point. Most sample-efficient for expensive models. Optuna, Hyperopt libraries. Population-Based Training (PBT): train multiple models in parallel. Periodically copy best model’s weights and hyperparameters, mutate slightly. Hyperband: early stopping of bad configurations, allocate more resources to promising ones. Tools: Optuna (Bayesian + other), Ray Tune (distributed), W&B Sweeps, Google Vizier, AWS SageMaker AutoPilot. Rule of thumb: learning rate is the most critical hyperparameter — tune it first.

85. What is the concept of gradient descent variants in deep learning?

  1. Gradient descent variants are different types of loss functions used for different deep learning tasks
  2. Gradient descent variants differ in how many training samples are used per weight update — Batch GD uses all data, Stochastic GD (SGD) uses one sample, Mini-batch GD uses small subsets — each offering different trade-offs between stability, speed, and convergence quality
  3. All gradient descent variants are identical in performance but differ in code implementation style
  4. Gradient descent variants only differ in whether they update weights during forward or backward passes

Answer : B
Explanation: Gradient Descent computes the direction to update weights using gradients. Variants differ in batch size: Batch Gradient Descent (BGD): uses entire dataset for each update. Accurate gradient estimate but extremely slow for large datasets. Must load all data into memory. Stochastic Gradient Descent (SGD): updates weights after each single training example. Very noisy — gradient estimate has high variance. Fast updates but zigzag path. Noisy updates can escape local minima. Mini-batch Gradient Descent (most common): process small batches (32-256 examples). Balances: more accurate than SGD, faster than BGD. GPU-optimized: matrix operations on batches are highly parallelized. Effectively called “SGD” in most frameworks. Optimizer extensions (all variants): Momentum: v = βv – α∇L, w = w + v. Accelerates in consistent directions, dampens oscillations. Nesterov Momentum: “look-ahead” variant — slightly better convergence. Adam: adaptive learning rates per parameter. Combines momentum + RMSprop. Default for most DL tasks. AdamW: Adam + decoupled weight decay — better for Transformers. RMSprop: adaptive per-parameter LR, good for RNNs. Adagrad: accumulates squared gradients — good for sparse features, diminishing LR issue. SGD with momentum + careful LR schedule can match or beat Adam for CNNs.

86. What is the role of GPUs in deep learning training?

  1. GPUs in deep learning are only used for displaying training progress graphs on monitors
  2. GPUs (Graphics Processing Units) accelerate deep learning training by performing thousands of floating-point operations in parallel — their many-core architecture is ideal for the matrix multiplications that dominate neural network computation, providing 10-100× speedup over CPUs
  3. GPUs slow down deep learning training because they have lower clock speeds than CPUs
  4. GPUs are only needed for convolutional neural networks and not for other deep learning models

Answer : B
Explanation: GPUs are why modern deep learning is possible — training on CPUs would take years for what GPUs do in hours. Why GPUs excel at deep learning: Neural network forward/backward passes = matrix multiplications. GPUs have thousands of smaller cores optimized for parallel arithmetic. NVIDIA A100: 6912 CUDA cores + 432 Tensor Cores. CPU (e.g., i9): 16-32 cores, optimized for complex sequential tasks. A matrix multiply that takes 1 second on CPU takes ~10 milliseconds on GPU (100× speedup). NVIDIA GPU generations: Tesla (V100, 2017), Ampere (A100, 2020), Hopper (H100, 2022), Blackwell (B100/B200, 2024). H100: 3958 TFLOPS for FP16 tensor operations. Memory: GPU VRAM is crucial — V100: 32GB, A100: 80GB, H100: 80GB. Large models require model parallelism across multiple GPUs. Frameworks: PyTorch: tensor.cuda(), model.cuda() — moves data/model to GPU. TensorFlow: automatic GPU detection. CUDA: NVIDIA’s parallel computing platform. cuDNN: optimized deep learning primitives. Alternatives: Google TPUs (Tensor Processing Units): custom ASIC designed for matrix math. TPU v4 pods: used to train PaLM, Gemini. AMD GPUs: ROCm platform. Apple Silicon (M1/M2/M3): unified memory, good for smaller models. Cloud GPUs: AWS (A100, H100), Google (TPU), Microsoft Azure.

87. What is the concept of neural network interpretability and explainability?

  1. Interpretability means converting neural network weights into human-readable code explanations
  2. Neural network interpretability refers to methods for understanding why a deep learning model made a specific prediction — including techniques like GRAD-CAM (visualize important image regions), LIME, SHAP (feature importance), attention visualization, and probing classifiers
  3. Interpretability is only important for medical models and irrelevant for other deep learning applications
  4. Interpretable models are always less accurate than non-interpretable (black box) models

Answer : B
Explanation: Neural Network Interpretability (XAI — Explainable AI) addresses the “black box” problem — understanding model decisions. Why it matters: Trust: doctors need to know why an AI diagnosed a disease. Debugging: find what the model learned wrong. Bias detection: identify unfair predictions (e.g., biased face recognition). Regulatory compliance: GDPR requires “right to explanation.” Model improvement. Key techniques: Gradient-based: Saliency Maps: compute ∂output/∂input — which input pixels most affect the output. GRAD-CAM (Gradient-weighted Class Activation Mapping): compute gradient of class score w.r.t. last convolutional feature map → heatmap of important image regions. Integrated Gradients: attribute prediction to input features. Perturbation-based: LIME (Local Interpretable Model-Agnostic Explanations): train a local linear model around a prediction. SHAP (SHapley Additive exPlanations): game-theory-based feature attribution — consistent, theoretically grounded. Attention visualization: visualize attention weights in Transformers — which tokens the model attends to. Probing classifiers: train simple classifiers on intermediate representations to understand what information is encoded. Circuit analysis (mechanistic interpretability): analyze specific network circuits that implement particular capabilities. Anthropic, DeepMind, OpenAI actively research mechanistic interpretability to understand large language models.

88. What is federated learning in the context of deep learning?

  1. Federated learning is a technique where multiple countries federate their data for joint AI training
  2. Federated learning is a distributed training approach where the model is trained across many devices (phones, hospitals) by sending model updates (gradients) to a central server rather than raw data — preserving privacy by keeping sensitive data on local devices
  3. Federated learning means federally regulated AI training that complies with government AI standards
  4. Federated learning combines multiple different models into one federated (unified) super-model

Answer : B
Explanation: Federated Learning (Google, 2017) enables training on distributed, privacy-sensitive data without centralizing it. Process: Central server initializes global model. Model sent to each participating device (phone, hospital, bank). Each device trains locally on its own private data. Only gradient updates (not raw data) sent back to server. Server aggregates updates (FedAvg: weighted average of gradients). Updated global model sent back to devices. Repeat for many rounds. Why privacy matters: medical records (HIPAA), financial data (GDPR), personal messages, keyboard inputs — cannot legally or ethically be centralized. Applications: Google Keyboard (Gboard): next-word prediction improved using your typing without sending messages to Google. Healthcare: training disease detection models across hospitals without sharing patient records. Banking: fraud detection across multiple banks without sharing transaction data. Mobile devices: personalization without uploading user data. Challenges: Non-IID data: data distribution varies across devices. Communication efficiency: reduce bandwidth with gradient compression. Convergence: harder to achieve than centralized training. System heterogeneity: devices have different compute and battery. Differential Privacy: combined with federated learning for mathematical privacy guarantees. Secure Aggregation: cryptographic technique ensuring server can’t see individual updates.

89. What is multi-task learning in deep learning?

  1. Multi-task learning is a technique for assigning different deep learning tasks to multiple GPUs simultaneously
  2. Multi-task learning is a training approach where a single model is trained on multiple related tasks simultaneously — sharing representations across tasks to improve generalization, especially for tasks with limited data
  3. Multi-task learning trains one model per task and then combines them using an ensemble method
  4. Multi-task learning is identical to transfer learning — both involve applying models to new tasks

Answer : B
Explanation: Multi-task Learning (MTL) trains one model to handle multiple tasks simultaneously by sharing representations. Architecture: Shared Encoder: learns common representations useful for all tasks. Task-specific Heads: separate output layers for each task. Loss: weighted sum of individual task losses. L = λ₁L₁ + λ₂L₂ + … + λnLn. Why it works: Auxiliary signal: additional supervision from related tasks improves representations. Regularization: shared representations must generalize across tasks → less overfitting. Inductive bias: related tasks provide complementary signals. Examples: NLP multi-task: T5 and GPT train on many tasks simultaneously (translation, summarization, classification, QA). BERT-style: pre-training on MLM + NSP. Computer Vision: object detection + depth estimation + semantic segmentation in a single network (autonomous driving). Medical: diagnose multiple diseases from the same scan simultaneously. MTL vs Transfer Learning: Transfer Learning: sequential — pre-train on task A, then fine-tune on task B. Multi-task: simultaneous — train on A + B at the same time. MTL challenges: negative transfer — if tasks are unrelated, they can hurt each other. Task weighting: determine optimal λᵢ for each task. Task interference: gradient conflicts between tasks. Gradient surgery, uncertainty weighting, PCGrad address these issues.

90. What is the difference between model training and model evaluation in deep learning?

  1. Training is done by data scientists; evaluation is done by software engineers for deployment
  2. Training is the process of optimizing model weights by minimizing loss on the training set; evaluation measures the model’s generalization performance on held-out data — using a validation set during training (for hyperparameter tuning) and a test set for final unbiased performance estimation
  3. Training uses GPUs; evaluation must be done on CPUs for fair performance comparison
  4. Model evaluation is only done once after training completes and cannot be repeated

Answer : B
Explanation: Training and Evaluation are complementary parts of the deep learning pipeline. Model Training: forward pass: compute predictions. Loss computation: measure prediction error. Backward pass: compute gradients. Weight update: gradient descent step. Repeat for many batches and epochs. Dropout active (training mode). BatchNorm uses mini-batch statistics. Model Evaluation: fixed weights — no gradient computation (torch.no_grad() in PyTorch). Dropout inactive — all neurons active. BatchNorm uses running statistics. Data splits: Training set (~70-80%): used for weight optimization. Validation set (~10-15%): monitor training, tune hyperparameters. Test set (~10-15%): final unbiased performance estimate — never seen during training or hyperparameter tuning. K-Fold Cross-Validation: split data into k folds, rotate which fold is validation — more robust for small datasets. Evaluation metrics: Classification: accuracy, precision, recall, F1-score, AUC-ROC. Regression: MAE, MSE, RMSE, R². Object detection: mAP (mean Average Precision). NLP: BLEU (translation), ROUGE (summarization), BERTScore. Overfitting diagnosis: training accuracy >> validation accuracy → overfit. Both low → underfit. Learning curves: plot train and validation loss vs. epochs to diagnose training issues.

Pages: 1 2 3 4 5 6 7 8 9 10