Deep Learning MCQ Questions And Answers

Deep Learning MCQ Questions And Answers

91. What is the concept of adversarial examples in deep learning?

  1. Adversarial examples are incorrectly labeled training examples added to make training more challenging
  2. Adversarial examples are carefully crafted inputs with imperceptibly small perturbations that cause deep learning models to make confidently wrong predictions — revealing surprising brittleness in neural networks and motivating adversarial training for robustness
  3. Adversarial examples are examples from an adversary (competing company) used to test a model’s limits
  4. Adversarial examples are the hardest training examples selected by curriculum learning for later stages

Answer : B
Explanation: Adversarial Examples (Szegedy et al., 2013) revealed a fundamental vulnerability in deep learning. Classic example: take an image of a panda classified correctly with 57.7% confidence. Add a tiny, imperceptible noise pattern (determined by computing the gradient of the loss). New image: looks identical to humans, but classified as “gibbon” with 99.3% confidence. How they’re created: FGSM (Fast Gradient Sign Method): x_adv = x + ε·sign(∇_x L(f(x), y)). Just add gradient direction times small ε → fools the model. PGD (Projected Gradient Descent): stronger multi-step attack. Carlini-Wagner (C&W): optimization-based attack. Why they exist: neural networks learn to classify based on statistical patterns that may not match human visual perception. Small perturbations can shift the input across a decision boundary. Implications: Autonomous vehicles: sticker on stop sign → classified as speed limit. Face recognition: glasses with adversarial pattern → fool facial recognition. Medical AI: imperceptible perturbation → wrong diagnosis. Defense methods: Adversarial Training: include adversarial examples in training. Certified Defenses: mathematical guarantees of robustness within ε-ball. Randomized Smoothing: predict based on average of noisy inputs. Input preprocessing: denoising, JPEG compression. Note: adversarial robustness vs. natural accuracy often trade off — active research area.

92. What is the concept of neural radiance fields (NeRF) in deep learning?

  1. NeRF is a type of deep learning network that uses radial basis functions as activation functions
  2. Neural Radiance Fields (NeRF) is a deep learning technique that represents a 3D scene as a continuous volumetric function learned by a neural network — enabling synthesis of novel viewpoints of a scene from just a few 2D images by learning how light rays pass through 3D space
  3. NeRF is a neural network regularization technique that prevents overfitting in deep learning
  4. NeRF is a deep reinforcement learning method where the agent explores radial search fields

Answer : B
Explanation: NeRF (Neural Radiance Fields), introduced by Mildenhall et al. (UC Berkeley, 2020), revolutionized 3D scene representation. How it works: Input: ~20-100 images of a scene from known camera positions. NeRF model: a small MLP that takes (x,y,z,θ,φ) as input — 3D location + viewing direction. Outputs: RGB color + density (σ) at that 3D point. Volume rendering: cast rays through the scene, sample points along each ray, query the MLP for color and density at each point, integrate along the ray using volume rendering equations → final pixel color. Training: minimize difference between rendered and real pixel colors (photometric loss). Result: can render the scene from any new viewpoint — photorealistic novel view synthesis. Limitations: slow training (hours), slow rendering, doesn’t generalize across scenes. Improvements: Instant-NGP (NVIDIA, 2022): hash encoding → 100× faster training. Mip-NeRF: handles multi-scale. Block-NeRF: large outdoor scenes. 3D Gaussian Splatting (2023): represents scenes as 3D Gaussians → real-time rendering. Applications: VR/AR content creation, film visual effects, product visualization, robotics scene understanding, autonomous vehicle scene reconstruction. The field evolved rapidly — NeRF variants are now enabling real-time, generalizable 3D scene understanding.

93. What is the concept of model parallelism in deep learning?

  1. Model parallelism is a technique for training multiple different model architectures simultaneously
  2. Model parallelism distributes different parts of a neural network across multiple GPUs — used when a model is too large to fit in a single GPU’s memory — with pipeline parallelism, tensor parallelism, and expert parallelism as key variants
  3. Model parallelism means each GPU trains a complete copy of the model on different data batches
  4. Model parallelism is only needed for models with more than 1 trillion parameters

Answer : B
Explanation: As models grow (GPT-3: 175B params, requiring 350GB in FP16 — far exceeding any single GPU’s 80GB), parallelism strategies are essential. Data Parallelism: each GPU has a complete copy of the model. Different batches of data processed on each GPU. Gradients averaged across GPUs (AllReduce). Limited by single GPU memory — cannot fit models larger than one GPU. PyTorch DDP (Distributed Data Parallel). Model Parallelism (Tensor Parallelism): split individual layers across GPUs. Matrix multiplications partitioned across devices. E.g., large attention head split across 8 GPUs. Communication overhead between GPUs for each layer. Used in Megatron-LM (NVIDIA). Pipeline Parallelism: split model into stages across GPUs. GPU 1 processes layers 1-10, GPU 2 processes 11-20, etc. GPipe, PipeDream handle the pipeline bubble problem. 3D Parallelism (Megatron-DeepSpeed): combine data + tensor + pipeline parallelism. Used to train GPT-3 scale models. Mixture of Experts (MoE): model has many expert sub-networks. Only a subset of experts activated per token. Expert Parallelism: different experts on different GPUs. GPT-4, Mixtral use MoE architecture. Expert Parallelism enables training models with trillions of total parameters with reasonable compute cost.

94. What is the difference between discriminative and generative models in deep learning?

  1. Discriminative models are trained on easy examples; generative models train on harder examples
  2. Discriminative models learn the conditional probability P(y|x) to classify inputs directly; generative models learn the joint distribution P(x,y) or the data distribution P(x) — enabling generation of new samples, not just classification
  3. Discriminative models are always more accurate than generative models for classification tasks
  4. Generative models are only used for image generation and cannot be used for text or audio

Answer : B
Explanation: The discriminative vs. generative distinction is fundamental in deep learning. Discriminative Models: model P(y|x) — probability of label y given input x. Directly learn the decision boundary between classes. Examples: CNN classifiers, BERT fine-tuned for classification, SVMs, logistic regression. Advantages: typically higher accuracy for classification when you have labeled data. Don’t need to model the full data distribution. Generative Models: model P(x,y) (full joint) or P(x) (data distribution). Can generate new samples by sampling from P(x) or P(x|y). Examples: GANs, VAEs, diffusion models, autoregressive LMs (GPT). Can also perform classification via Bayes’ rule: P(y|x) = P(x|y)P(y)/P(x). Advantages: generate new data, data augmentation, semi-supervised learning, outlier detection (low P(x) = anomaly), understand the data distribution. Deep learning generative models: Autoregressive (GPT): model P(x) as product of conditional probabilities. Diffusion models: learn to denoise → generate. GANs: implicitly learn P(x) through adversarial training. VAEs: learn explicit latent distribution. Flow models: learn invertible transformations. Modern blur: LLMs are both discriminative (BERT) and generative (GPT). The boundary is increasingly blurry as models grow more capable.

95. What is the concept of curriculum learning in deep learning?

  1. Curriculum learning is a method for organizing the school curriculum for AI students
  2. Curriculum learning is a training strategy inspired by human learning — presenting training examples in a meaningful order from easier to harder, rather than randomly, enabling faster convergence and better final performance
  3. Curriculum learning means training a model on a curriculum (list) of different tasks in random order
  4. Curriculum learning is only applicable to language models and cannot be used in computer vision

Answer : B
Explanation: Curriculum Learning (Bengio et al., 2009) mimics how human learning works — start simple, gradually increase difficulty. Basic principle: sort training examples by difficulty. Train on easy examples first → build foundational knowledge. Progressively introduce harder examples. Difficulty measures: Model-based: examples the model gets wrong most often are hardest. Data-based: image clarity, noise level, sentence length, label ambiguity. Examples: NLP: start with short sentences, progress to longer and more complex. Computer Vision: clear images → augmented/noisy images. Speech: clean speech → noisy speech. Math: simple problems → complex proofs. Self-paced learning: the model itself determines what’s hard vs. easy — focuses on examples near its current capability boundary. Applications: Machine translation: shorter sentences first. Neural machine reading comprehension: factual questions first, inference questions later. Robotics: simple tasks before complex manipulation. Why it works: avoids confusing the model early in training with hard examples. Helps the model build robust representations on easier cases first. Can prevent local minima by establishing a good starting point. Adversarial curriculum: hard examples (near decision boundary) emphasized for robust training. Active learning: related concept — select the most informative examples for human labeling.

96. What is the concept of zero-shot and few-shot learning in deep learning?

  1. Zero-shot learning means training a model without any shots (no GPU) for faster training
  2. Zero-shot learning enables models to perform tasks without any task-specific training examples by leveraging prior knowledge; few-shot learning generalizes from very few examples (1-10) — both are enabled by large pre-trained models that learn transferable representations and in-context learning
  3. Zero-shot means the model achieves zero accuracy; few-shot means accuracy is a few percentage points
  4. Zero-shot and few-shot learning require retraining the entire model from scratch for each new task

Answer : B
Explanation: Zero-shot and few-shot learning represent the new paradigm of large language model usage. Zero-Shot: model performs a task with zero examples of that specific task. Works because large models learn transferable knowledge during pre-training. Example: “Translate ‘Hello’ to French” — the model has never been fine-tuned for translation yet can do it. CLIP: classifies images with text descriptions of unseen categories. GPT-4: answers questions about topics not in explicit training examples. Few-Shot (In-Context Learning): provide a few examples in the prompt (no gradient updates!). Example prompt: “Positive: I loved this movie. Negative: Terrible film. Positive: Amazing performance. The food was bland and service was slow: “. Model infers the pattern from the examples. GPT-3 (2020) demonstrated impressive few-shot learning. One-Shot: exactly one example provided. K-Shot: k examples provided. How it works: large models develop the ability to learn from context as an emergent capability of scale. Meta-learning (“learning to learn”): model learns to quickly adapt to new tasks from examples. MAML (Model-Agnostic Meta-Learning): explicitly train models to learn fast from few examples. Practical impact: dramatically reduces need for labeled data. Users can define new tasks by simply describing them in natural language.

97. What is the concept of representation learning in deep learning?

  1. Representation learning is the process of representing neural networks visually in diagrams
  2. Representation learning is the ability of deep learning models to automatically discover useful feature representations from raw data — transforming high-dimensional, unstructured inputs into lower-dimensional, structured latent spaces that capture meaningful patterns and relationships
  3. Representation learning only applies to image data and cannot extract representations from text or audio
  4. Representation learning always requires manual feature engineering before the neural network processes data

Answer : B
Explanation: Representation Learning is arguably the most important capability that makes deep learning powerful. Why it matters: traditional ML required manual feature engineering — domain experts laboriously designed features (HOG for computer vision, MFCC for audio, TF-IDF for text). Deep learning automatically discovers useful representations through end-to-end training. What good representations look like: Compressed: remove noise and irrelevant variation. Disentangled: different factors of variation encoded in different latent dimensions. Transferable: useful for multiple downstream tasks. Structured: similar inputs have similar representations. Hierarchy of representations: Layer 1: simple features (edges, n-grams, phonemes). Layer 2: more complex features (shapes, word combinations). Layer N: semantic concepts (faces, objects, sentence meaning). Evidence of good representations: Word2Vec: arithmetic on word vectors (king – man + woman ≈ queen). Visualization: t-SNE of penultimate CNN layer → meaningful clusters of related classes. Transfer success: ImageNet features work for medical imaging → representations truly transferable. Self-supervised learning: BERT, SimCLR, etc. learn rich representations without labels. Probing: simple classifiers on representations reveal they encode syntax, semantics, position, etc. The quality of the representation determines the quality of all downstream tasks.

98. What is the concept of graph neural networks (GNNs) in deep learning?

  1. Graph neural networks use graphs (diagrams) to visualize how information flows through training
  2. Graph Neural Networks (GNNs) extend deep learning to graph-structured data — where nodes represent entities and edges represent relationships — by iteratively aggregating and transforming features from each node’s neighbors to learn rich node, edge, or graph-level representations
  3. GNNs are neural networks that use graph coloring algorithms to partition the network into clusters
  4. Graph neural networks can only process undirected graphs and cannot handle directed networks

Answer : B
Explanation: GNNs handle non-Euclidean data that CNNs and RNNs cannot — social networks, molecules, knowledge graphs, road networks. Core message passing: for each node v: aggregate messages from neighbors N(v). Update node representation using aggregated messages + own features. h_v^(l+1) = UPDATE(h_v^(l), AGGREGATE({h_u^(l) : u ∈ N(v)})). Stack L layers: after L steps, each node’s representation captures its L-hop neighborhood. Key GNN variants: GCN (Graph Convolutional Network): spectral-based convolution. GraphSAGE: inductive — sample and aggregate from neighbors. GAT (Graph Attention Network): use attention weights for neighbor aggregation. Message Passing Neural Network (MPNN): general framework. Graph Transformer: apply Transformer to graphs. Applications: Molecular Biology: predict molecular properties for drug discovery. AlphaFold uses GNN for protein structure prediction. Social Networks: friend recommendation, spam detection. Knowledge Graphs: link prediction, entity classification. Recommendation Systems: user-item interaction graphs (Pinterest Pins). Traffic Prediction: road network graph (Google Maps). Chip Design: represent circuit as graph — Google’s AlphaChip uses GNNs. Physics: simulate particle systems as graphs. Limitations: over-smoothing (deep GNNs → all node representations converge), scalability to billion-node graphs.

99. What is the concept of scaling laws in deep learning?

  1. Scaling laws describe how the physical scale (size) of GPU clusters affects electricity consumption
  2. Scaling laws in deep learning are empirical relationships showing that model performance improves predictably as model size, training data, and compute are increased — enabling researchers to forecast performance before training a model and to optimally allocate compute budgets
  3. Scaling laws describe how the learning rate should scale with the number of layers in a deep network
  4. Scaling laws are legal regulations governing the maximum compute allowed for AI training

Answer : B
Explanation: Scaling Laws (Kaplan et al., OpenAI, 2020 — “Scaling Laws for Neural Language Models”) revealed one of the most important empirical findings in deep learning. Key finding: language model loss follows power laws with model size (N parameters), dataset size (D tokens), and compute (C = 6ND FLOPs). L(N) ∝ N^(-α_N), L(D) ∝ D^(-α_D), L(C) ∝ C^(-α_C). Each factor contributes independently with diminishing returns. For a given compute budget, there’s an optimal model size and number of training tokens. Implications: Bigger is better — performance improves predictably with scale. GPT-3 (175B) → GPT-4 (>1T params) → improvements were predictable. Chinchilla scaling law (Hoffmann et al., DeepMind, 2022): corrected earlier laws — for a given compute budget, should use a smaller model trained on more data. Chinchilla (70B params, 1.4T tokens) outperformed Gopher (280B, 300B tokens) despite being 4× smaller. Rule of thumb: train for ~20 tokens per parameter (Chinchilla-optimal). Why it matters: enables strategic planning of large-scale training runs. Predicts performance without actually training the full model. Explains why GPT-4 is so much better than GPT-3 (just more scale). Emergent abilities: certain capabilities (multi-step math, code generation) appear suddenly at scale thresholds.

100. What are the most important current trends and future directions in deep learning?

  1. Deep learning has peaked — all major problems are solved and no significant advances are expected
  2. Key current trends include Large Language Models and multimodal AI, diffusion models for generation, state space models (Mamba), test-time compute scaling, AI agents, scientific AI (AlphaFold, weather prediction), efficient AI for edge deployment, and AI safety and alignment research
  3. The future of deep learning is exclusively focused on making existing architectures more energy efficient
  4. Future deep learning will completely replace all human cognitive tasks within the next two years

Answer : B
Explanation: Deep learning is advancing at an extraordinary pace across multiple fronts. Large Language Models and Multimodal AI: GPT-4, Claude, Gemini — general-purpose AI systems. Multimodal: text + images + audio + video + code in one model. Foundation models that can be prompted for virtually any task. Diffusion Models: Stable Diffusion, DALL-E 3, Sora (video). Become dominant for high-quality image/video synthesis. State Space Models: Mamba, RWKV — potential Transformer alternatives. Linear complexity for long sequences. Test-Time Compute Scaling: OpenAI o1, o3 — allocate more compute during inference for harder problems. “Chain of thought” and “thinking” before responding. AI Agents: autonomous systems that plan and execute multi-step tasks. Tool use (web search, code execution, database queries). Multi-agent systems that collaborate. Scientific AI: AlphaFold 3 (predicts protein-DNA/RNA complexes), GNNs for drug discovery, AI weather forecasting (GraphCast), materials science. Efficient AI: quantization, pruning, distillation for edge deployment. Mixture of Experts (MoE): efficient scaling. Virtual Threads (TinyML). AI Safety and Alignment: RLHF, Constitutional AI, scalable oversight. Mechanistic interpretability. Preparing for AGI. The pace of progress suggests deep learning will continue transforming every field of human knowledge and endeavor.

Pages: 1 2 3 4 5 6 7 8 9 10