Neural Networks MCQ Questions And Answers

Neural Networks MCQ Questions And Answers

81. What is word embedding in neural networks for natural language processing?

  1. The process of physically embedding text data into a neural network’s memory chips
  2. A technique for representing words as dense numerical vectors in a continuous vector space where semantically similar words are positioned close together — enabling neural networks to process text mathematically and capture word relationships
  3. A text compression technique that encodes words as binary sequences for efficient storage
  4. A method of embedding one language’s words into another language’s vocabulary space

Answer : B
Explanation: Word Embeddings solve the problem of representing discrete text for neural networks. One-Hot Encoding (naive approach): each word is a binary vector of size V (vocabulary size). Problems: extremely sparse and high-dimensional, no semantic relationships captured (“king” and “queen” are equally distant from each other as from “car”). Word Embeddings: dense, low-dimensional vectors (typically 50-300 dimensions) learned from text data. Key property: captures semantic relationships. Famous example: king – man + woman ≈ queen (vector arithmetic). Word2Vec (Mikolov et al., 2013): learns embeddings by predicting context words (CBOW) or predicting from context (Skip-gram). GloVe (Pennington et al., 2014): learns embeddings from word co-occurrence statistics. FastText: handles out-of-vocabulary words using character n-grams. Contextual Embeddings: BERT, GPT learn different embeddings for the same word depending on context (polysemy). Pre-trained embeddings (GloVe, Word2Vec) can be loaded and fine-tuned for downstream tasks — a form of transfer learning.

82. What is knowledge distillation in neural networks?

  1. A technique that distills (summarizes) the knowledge learned by a neural network into a textual report
  2. A model compression technique where a smaller student network is trained to mimic the behavior of a larger teacher network — using soft probability outputs from the teacher instead of hard labels — producing a compact model that retains much of the teacher’s performance
  3. A process of extracting domain knowledge from human experts to train neural networks
  4. A technique that distills multiple neural network models into one unified architecture

Answer : B
Explanation: Knowledge Distillation, introduced by Hinton et al. (2015), compresses a large, accurate (but expensive) teacher model into a smaller, faster student model. Process: Train a large teacher model. Generate soft predictions from teacher using temperature T > 1 (e.g., [0.8, 0.15, 0.05] instead of one-hot [1,0,0]). Train student on: combination of hard labels (standard cross-entropy with true labels) + soft labels (KL divergence from teacher’s soft predictions). Why soft labels help: they contain “dark knowledge” — the teacher’s near-misses reveal relationships between classes (e.g., a cat image scoring 0.1 for dog reveals cat-dog similarity). This richer signal trains a better student than hard labels alone. Applications: Model compression for deployment on mobile/edge devices. Tiny BERT, DistilBERT (40% smaller BERT, 97% accuracy). Examples: DistilGPT-2 (GPT-2 compressed 6×), MobileNets distilled from ResNets. Knowledge distillation is essential for deploying state-of-the-art models in resource-constrained environments.

83. What is the purpose of data augmentation in neural network training?

  1. A technique for augmenting (adding complexity to) the neural network architecture during training
  2. A strategy of artificially expanding the training dataset by applying realistic transformations to existing examples — increasing diversity, reducing overfitting, and improving model generalization without collecting new data
  3. A method of augmenting GPU memory capacity to train larger neural networks
  4. A technique for automatically augmenting the training dataset with data from the internet

Answer : B
Explanation: Data Augmentation creates new training examples by applying label-preserving transformations: Image Augmentation: Random Horizontal/Vertical Flip, Random Rotation, Random Crop and Resize, Color Jitter (brightness, contrast, saturation, hue), Gaussian Noise, Cutout (randomly mask square regions), Mixup (blend two images and their labels), CutMix (cut and paste regions from different images). Text Augmentation: Synonym Replacement, Random Insertion/Deletion, Back-translation (English→French→English). Audio Augmentation: Time shifting, Pitch shifting, Adding background noise, Speed perturbation. Advanced: AutoAugment (learns optimal augmentation policies using RL), RandAugment (random subset of augmentations). Benefits: Effective regularization (model sees more diverse examples). Prevents memorization of specific visual artifacts. Teaches invariances (a cat is still a cat when flipped). Especially valuable with small datasets. In PyTorch: torchvision.transforms provides standard image augmentation. Albumentations is a popular high-performance library.

84. What is the difference between an encoder and decoder in neural networks?

  1. An encoder is the first half of a neural network; decoder is the second half in all architectures
  2. An encoder compresses input data into a lower-dimensional latent representation (feature extraction/compression); a decoder takes a latent representation and reconstructs or generates output — the encoder-decoder pattern is central to autoencoders, seq2seq, Transformers, and U-Net
  3. An encoder processes text; a decoder processes images in multimodal neural networks
  4. The encoder runs during training; the decoder runs only during inference

Answer : B
Explanation: The Encoder-Decoder architecture is one of the most important patterns in deep learning: Encoder: maps high-dimensional input to a compact latent representation. Extracts meaningful features while discarding irrelevant information. In Transformers: processes entire input sequence, each token attends to all others. Decoder: takes latent representation and generates output (reconstruction, translation, generation). In Transformers: generates output autoregressively, each token attends to previous tokens + encoder output. Applications: Autoencoder: encoder for compression, decoder for reconstruction. Seq2Seq: LSTM encoder compresses source sentence, LSTM decoder generates target language. Transformer for MT: BERT-like encoder + GPT-like decoder. U-Net: encoder (downsampling path) extracts features, decoder (upsampling path) reconstructs at full resolution — used for image segmentation. Image-to-Image translation (Pix2Pix): encoder extracts features from source image, decoder generates target image. The encoder-decoder pattern elegantly separates understanding (encoding) from generation (decoding).

85. What is neural network pruning and why is it important for deployment?

  1. A technique that removes incorrectly trained neural networks and starts fresh
  2. A model compression technique that removes redundant or unimportant weights, neurons, or layers from a trained neural network — producing a smaller, faster model with minimal accuracy loss for deployment on resource-constrained devices
  3. A regularization technique that prunes (caps) weight values to a maximum magnitude
  4. A training technique that prunes the training dataset to remove low-quality examples

Answer : B
Explanation: Neural Network Pruning removes unnecessary components to create smaller, faster models. Types: Weight Pruning (Unstructured): set individual weights below a threshold to zero. Creates sparse networks. Neuron/Filter Pruning (Structured): remove entire neurons or convolutional filters. More hardware-friendly (no sparse computation needed). Layer Pruning: remove entire layers deemed redundant. Approaches: Magnitude-based pruning (remove smallest weights — simplest). Gradient-based pruning (remove weights that contribute least to the loss). Lottery Ticket Hypothesis (Frankle & Carlin, 2019): within large networks exist small sub-networks (“winning tickets”) that can be trained from scratch to match the full network’s accuracy. Workflow: Train large model → prune → fine-tune → compress. Often combined with quantization (reduce bit precision) and knowledge distillation. Why it matters: ResNet-50 (98MB) → pruned to 10MB with <1% accuracy drop — deployable on smartphones. Mobile AI requires model efficiency: pruning, quantization, and NAS are essential skills for production ML engineers.

86. What is quantization in the context of neural networks?

  1. A technique that quantifies how well a neural network performs using standardized benchmarks
  2. A model compression technique that reduces the numerical precision of weights and activations from 32-bit floating point to lower bit representations (8-bit, 4-bit integers) — reducing model size and memory bandwidth while enabling faster inference on specialized hardware
  3. A process of converting continuous input features into discrete quantized categories
  4. A method of quantizing the learning rate during training for more stable convergence

Answer : B
Explanation: Quantization reduces numerical precision to shrink model size and accelerate inference: Standard training: FP32 (32-bit float) — 4 bytes per weight. FP16 (16-bit float, half-precision): 2× smaller, widely used for training on modern GPUs (mixed-precision training). INT8 (8-bit integer): 4× smaller than FP32. Sufficient for inference on most models with minimal accuracy loss. INT4 (4-bit integer): 8× smaller. Used in LLM quantization (GGUF, GPTQ formats for local LLM deployment). Post-Training Quantization (PTQ): quantize a pre-trained model without retraining. Fastest but slightly more accuracy loss. Quantization-Aware Training (QAT): simulate quantization during training. Better accuracy. Why it matters: GPT-3 at FP32: 700GB. At INT4: ~87GB — makes running LLMs locally feasible. INT8 inference is 2-4× faster than FP32 on CPUs and NPUs. 4-bit quantization enables running Llama-3 8B on a laptop. LLM.int8() and GPTQ are popular quantization methods for transformer models. Quantization + Pruning + Distillation = comprehensive model compression toolkit.

87. What is the difference between model accuracy and model loss during training?

  1. Model accuracy and loss are identical metrics — high accuracy always means low loss
  2. Loss measures the continuous numerical error between predictions and targets (what the optimizer minimizes); accuracy measures the fraction of correct discrete predictions (what we ultimately care about) — a model with lower loss doesn’t always have higher accuracy and vice versa
  3. Accuracy is used during training; loss is used only during evaluation on test data
  4. Loss is always between 0 and 1; accuracy has no upper bound

Answer : B
Explanation: Understanding both metrics is essential for neural network diagnostics: Loss: a continuous differentiable measure of prediction error (e.g., cross-entropy, MSE). Used by the optimizer (gradient descent) to update weights. Can be any non-negative value. Lower is better. Accuracy: the fraction of correctly classified examples. Directly interpretable to humans. Between 0% and 100%. Not directly optimized (not differentiable for classification). Relationship: Generally correlated — lower training loss → higher training accuracy. Can diverge when: Early training (loss decreasing but accuracy not yet improving), Class imbalance (loss can be low while accuracy is misleadingly high), Threshold effects (small changes in confidence can flip class predictions, changing accuracy without changing loss much). Typical training dynamics: Both training loss and accuracy should improve over epochs. If training loss decreases but validation loss increases → overfitting. If both losses plateau → learning rate may need adjustment or model may have converged. Monitor both metrics during training for complete diagnostics.

88. What is PyTorch and TensorFlow in the context of neural network implementation?

  1. Programming languages developed specifically for writing neural network algorithms
  2. PyTorch (Meta/Facebook) and TensorFlow (Google) are the two dominant open-source deep learning frameworks — providing automatic differentiation, GPU acceleration, pre-built layers, and optimization tools that make implementing neural networks practical
  3. Cloud platforms for deploying trained neural network models to production servers
  4. Database systems optimized for storing and retrieving neural network training datasets

Answer : B
Explanation: The two dominant deep learning frameworks: PyTorch (Facebook/Meta, 2016): Dynamic computation graph — defines the network by running it (define-by-run/eager execution). More Pythonic, flexible, and intuitive. Dominant in research (most papers implement in PyTorch). Tools: torch.nn (layers), torch.optim (optimizers), torchvision (CV), torchtext (NLP), torchaudio. Growing in production (TorchServe). TensorFlow (Google, 2015): Originally static computation graph (define-then-run). TensorFlow 2.0 (2019) adopted eager execution with Keras as the primary high-level API. Strong production ecosystem (TensorFlow Serving, TF Lite for mobile, TF.js for browser). JAX (Google, emerging): functional transformations, JIT compilation — increasingly popular for research. Keras: high-level API that runs on top of TensorFlow (and PyTorch via KerasCore). Industry trend: PyTorch dominates research; both are competitive in production. For interviews: be comfortable implementing a simple neural network in either framework — understanding concepts is more important than framework-specific syntax.

89. What is the concept of neural network depth vs. width?

  1. Depth refers to the training duration; width refers to the number of training examples
  2. Depth is the number of layers in a neural network; width is the number of neurons per layer — both increase model capacity, but deeper networks are more computationally efficient for complex tasks while wider networks can capture more parallel features at each representation level
  3. Depth measures the complexity of loss functions; width measures input feature dimensions
  4. A deep network always outperforms a wide network regardless of the task or dataset size

Answer : B
Explanation: Depth (number of layers) vs. Width (neurons per layer) are two dimensions of model capacity: Depth: enables hierarchical feature learning — early layers learn simple features, deep layers learn complex compositions. Deep networks can represent exponentially more functions than shallow networks with the same parameter count (exponential advantage of depth). Key challenge: training instability (vanishing/exploding gradients) — addressed by BatchNorm, ResNet skip connections, careful initialization. Width: more neurons per layer can capture more parallel features simultaneously. Wider layers can memorize more patterns. Easier to optimize (fewer gradient flow issues). EfficientNet (Tan & Le, 2019) systematically studied the relationship between depth, width, and input resolution — finding optimal scaling via a “compound coefficient” that scales all three dimensions together. In practice: modern architectures (GPT, BERT, ResNet) are both deep AND wide. For a given parameter budget: depth generally provides more expressive power for complex tasks. For simple tasks or small datasets: width with fewer layers may be preferable (less risk of vanishing gradients, easier training).

90. What is the concept of neural architecture search (NAS)?

  1. A database search technique for finding pre-trained neural networks on model repositories
  2. An automated machine learning (AutoML) technique that uses search algorithms or ML itself to find the optimal neural network architecture for a given task — automating the human expertise required for architecture design
  3. A security technique that searches for and patches vulnerabilities in neural network systems
  4. A research method for finding the best neural network paper published in a given year

Answer : B
Explanation: Neural Architecture Search (NAS) automates the design of neural network architectures. Search Space: defines which architectures can be considered (layer types, connections, filter sizes). Search Strategy: how to explore the search space. Random Search — baseline. Reinforcement Learning — a controller RNN generates architectures and is trained using validation accuracy as reward (original NAS by Zoph & Le, Google, 2016 — took 800 GPUs for 28 days). Evolutionary Algorithms — evolve architectures through mutation and selection. Gradient-based (DARTS — Differentiable Architecture Search) — makes architecture search differentiable, trainable with backpropagation — much faster. One-Shot NAS — train one supernetwork containing all possible architectures. Performance Estimation: predict performance without full training (weight sharing, learning curve extrapolation). Notable NAS results: NASNet, EfficientNet, MobileNetV3, RegNet — all found by NAS, outperforming human-designed architectures. NAS is a key enabler of AutoML — making deep learning accessible without deep architecture design expertise.

Pages: 1 2 3 4 5 6 7 8 9 10