Neural Networks MCQ Questions And Answers

Neural Networks MCQ Questions And Answers

51. What is transfer learning in neural networks?

  1. A technique for transferring trained neural network models between different hardware platforms
  2. A technique where a pre-trained model’s learned weights are reused as the starting point for a new but related task — dramatically reducing training time and data requirements for the new task
  3. A method for transferring knowledge between human experts and neural networks
  4. A technique that transfers gradient information between different layers during backpropagation

Answer : B
Explanation: Transfer Learning leverages knowledge gained from training on a large dataset (source task) for a different but related task (target task). How it works: Take a pre-trained model (e.g., ResNet-50 trained on ImageNet with 1.4M images). Remove the final classification layer. Add new layers for your specific task (e.g., cat vs. dog classifier). Fine-tune: either train only new layers (frozen base — faster, less data needed) or train all layers with a very low learning rate (full fine-tuning — more powerful, needs more data). Common in: Computer Vision (ImageNet pre-trained CNNs), NLP (BERT, GPT pre-trained transformers fine-tuned for sentiment analysis, QA, summarization). Why it’s powerful: Neural networks learn general, reusable features — edge detectors in early layers, shape detectors in middle layers. These transfer across different visual domains. Transfer learning has democratized deep learning by making it accessible without massive compute and data resources.

52. What is the ResNet (Residual Network) architecture and what problem does it solve?

  1. A network that resets weights to initial values when training plateaus
  2. A deep CNN architecture that uses skip connections (residual connections) to allow gradients to flow directly through the network — solving the degradation problem where adding more layers to a very deep network actually increases training error
  3. A network that processes residual (leftover) data ignored by the main network
  4. A residual memory network used for storing previously computed neural network states

Answer : B
Explanation: ResNet (Residual Network), developed by He et al. at Microsoft Research (2015), won the ImageNet competition with 152 layers — far deeper than any previous network. The Problem: as networks get deeper, training error actually increases (not just test error) — even with residual connections, training very deep networks was difficult. The Solution — Skip Connections: instead of learning the direct mapping H(x), layers learn the residual F(x) = H(x) – x. The skip connection adds the original input x back to the output: output = F(x) + x. Benefits: Gradients can flow directly through the identity shortcut (no gradient vanishing), enables training of extremely deep networks (50, 101, 152 layers), identity connections act as gradient highways. Impact: ResNet is one of the most cited papers in all of computer science, established the paradigm of residual learning that influenced Transformers (attention residuals), DenseNet, U-Net, and virtually every modern deep learning architecture.

53. What is the Transformer architecture in neural networks?

  1. A neural network that transforms input data between different modalities (text to image)
  2. A deep learning architecture based entirely on self-attention mechanisms — processing all input tokens in parallel rather than sequentially, enabling better capture of long-range dependencies and massively parallel training compared to RNNs
  3. A power transformer circuit that converts electrical signals for neural network hardware
  4. A modular neural network framework that transforms one model architecture into another

Answer : B
Explanation: The Transformer, introduced in “Attention Is All You Need” (Vaswani et al., Google, 2017), revolutionized deep learning. Key innovations: Self-Attention mechanism — every token attends to every other token simultaneously, capturing dependencies regardless of distance. Multi-Head Attention — runs attention multiple times in parallel with different learned projections, capturing different types of relationships. Positional Encoding — adds position information (since there is no recurrence). Encoder-Decoder structure (original): Encoder processes input sequence, Decoder generates output sequence. Key advantages over RNNs: Parallel processing of entire sequence (vs. sequential RNN steps) — dramatically faster training. Better long-range dependency capture. Scales extremely well with more data and compute. Foundation of all modern LLMs: BERT (encoder-only), GPT series (decoder-only), T5 (encoder-decoder). Vision Transformer (ViT) applies Transformers to images. The Transformer is arguably the most important deep learning architecture ever developed.

54. What is the attention mechanism in neural networks?

  1. A technique for directing human attention to the most important parts of a neural network diagram
  2. A mechanism that allows a neural network to dynamically focus on the most relevant parts of its input when producing each output — computing a weighted sum of input representations where weights reflect relevance
  3. A technique that pays extra attention (allocates more compute) to difficult training examples
  4. A regularization method that penalizes the network for attending to irrelevant features

Answer : B
Explanation: The Attention mechanism, introduced by Bahdanau et al. (2014) for neural machine translation, allows models to dynamically weight different parts of the input. How it works: for each output position, compute attention scores between that position and all input positions. Normalize scores with Softmax to get attention weights. Compute output as weighted sum of input representations. Self-Attention (in Transformers): every position attends to every other position in the same sequence. Computed via Query (Q), Key (K), Value (V) matrices: Attention(Q,K,V) = Softmax(QKᵀ/√d_k)V. The √d_k scaling prevents dot products from becoming too large. Multi-head attention runs this multiple times in parallel. Applications: Machine translation (align source and target words), Summarization (focus on key sentences), Question answering (attend to relevant passage parts), Image captioning (attend to relevant image regions). Attention is the single most important building block in modern NLP and increasingly in computer vision.

55. What is the difference between BERT and GPT in natural language processing?

  1. BERT and GPT are identical models trained on the same data with different names
  2. BERT is an encoder-only Transformer pre-trained using masked language modeling (bidirectional context) — best for understanding tasks; GPT is a decoder-only Transformer pre-trained using next-token prediction (left-to-right) — best for text generation tasks
  3. GPT is a smaller, less powerful version of BERT designed for edge device deployment
  4. BERT uses RNN architecture; GPT uses CNN architecture with attention mechanisms

Answer : B
Explanation: BERT (Bidirectional Encoder Representations from Transformers, Google, 2018): Encoder-only Transformer. Pre-training: Masked Language Modeling (MLM — predict randomly masked tokens from bidirectional context) + Next Sentence Prediction (NSP). Bidirectional context — understands each token using both left and right context. Best for: classification, question answering, named entity recognition, semantic similarity. Fine-tuned for downstream tasks. GPT (Generative Pre-trained Transformer, OpenAI): Decoder-only Transformer. Pre-training: Next-token prediction — predict the next word using only left context (autoregressive). Unidirectional (causal) — each token can only attend to previous tokens. Best for: text generation, summarization, translation, code generation, chatbots. ChatGPT is based on GPT-3.5/GPT-4. The key distinction: BERT understands (classification, comprehension), GPT generates (completion, conversation). T5 and many modern models use encoder-decoder combining both strengths.

56. What is a Generative Adversarial Network (GAN)?

  1. A network where multiple AI agents compete against each other for computational resources
  2. A deep learning framework consisting of two neural networks — a Generator (creates fake samples) and a Discriminator (distinguishes real from fake) — trained adversarially in a minimax game until the Generator produces indistinguishable fake samples
  3. A type of RNN that generates text by adversarially competing with a language model
  4. A network that generates adversarial examples to improve the robustness of other models

Answer : B
Explanation: GANs, introduced by Ian Goodfellow et al. (2014), consist of two competing networks: Generator (G): takes random noise vector z as input, produces synthetic samples (images, audio, text) trying to fool the discriminator. Discriminator (D): distinguishes real samples (from training data) from fake samples (from G). Training: G minimizes log(1-D(G(z))), D maximizes log(D(x)) + log(1-D(G(z))). The minimax game: G tries to fool D; D tries not to be fooled. At convergence, G produces samples indistinguishable from real data. Applications: Image synthesis (photorealistic human faces that don’t exist — ThisPersonDoesNotExist.com), Image-to-Image translation (CycleGAN — horse↔zebra), Super-resolution, Data augmentation, Style transfer, Deepfakes. Training challenges: Mode collapse (G generates limited variety), Training instability (D winning too fast prevents G from learning), Non-convergence. Variants: DCGAN, StyleGAN (photorealistic faces), Conditional GAN, Wasserstein GAN.

57. What is an autoencoder in neural networks?

  1. A neural network that automatically encodes data into machine code for efficient storage
  2. An unsupervised neural network architecture that learns to compress (encode) input data into a lower-dimensional latent representation, then reconstruct (decode) the original input — learning efficient data representations without labels
  3. A network that encodes audio signals automatically for speech recognition systems
  4. A self-training network that labels its own training data using automatic encoding

Answer : B
Explanation: An Autoencoder has two parts: Encoder — compresses input x into a lower-dimensional latent vector z (bottleneck). Decoder — reconstructs the input from z, producing x̂. Training objective: minimize reconstruction loss (||x – x̂||²). The bottleneck forces the network to learn a compressed, meaningful representation. Types: Undercomplete Autoencoder (standard — bottleneck smaller than input). Sparse Autoencoder (adds sparsity penalty — forces most latent neurons to be 0). Denoising Autoencoder (trained to reconstruct clean input from corrupted input — learns robust features). Variational Autoencoder (VAE — learns a probabilistic latent space, enables generation of new samples). Applications: Dimensionality reduction (like PCA but non-linear), Anomaly detection (high reconstruction error = anomaly), Image denoising, Feature learning, Data generation (VAEs), Pretraining for transfer learning. Connection to Q13: the auto-associative network in your existing questions is another name for an autoencoder.

58. What is the weight initialization problem in neural networks and how is it addressed?

  1. A memory allocation problem that occurs when initializing neural network weights on hardware
  2. The problem that poor initial weight values cause vanishing or exploding gradients from the very first forward and backward pass — addressed by initialization strategies like Xavier/Glorot and He initialization based on layer size and activation function
  3. A training problem where all weights converge to the same value during initialization
  4. A compatibility issue when loading pre-trained weights into a different model architecture

Answer : B
Explanation: Poor weight initialization causes training failure from the start. Zero initialization: all weights zero → all neurons compute the same output → symmetry problem (all neurons remain identical throughout training). Large random values: exploding gradients immediately. Small random values: vanishing gradients immediately. Solutions: Xavier/Glorot Initialization (for sigmoid/tanh): weights drawn from distribution with variance 2/(fan_in + fan_out) — keeps activation variance constant across layers. He/Kaiming Initialization (for ReLU): weights drawn from distribution with variance 2/fan_in — accounts for ReLU zeroing half the neurons, keeps variance stable. Batch normalization reduces sensitivity to weight initialization — with BN, initialization matters much less. LeCun Initialization (for SELU): variance = 1/fan_in. All major frameworks implement these: PyTorch uses Kaiming He init by default for Conv layers, Xavier for Linear. Proper initialization ensures training can begin learning from the first batch.

59. What is the cost function (loss function) in neural networks?

  1. A financial metric that calculates the monetary cost of running neural network training
  2. A mathematical function that measures the difference between the network’s predicted output and the true target — the training process minimizes this function to improve predictions
  3. A function that calculates the computational cost (FLOPS) of each neural network layer
  4. A penalty function that reduces the learning rate when training cost exceeds budget

Answer : B
Explanation: The Loss Function (Cost Function) quantifies how wrong the model’s predictions are — gradient descent minimizes this during training. Common loss functions: Mean Squared Error (MSE) — for regression: L = (1/n)Σ(yᵢ – ŷᵢ)². Penalizes large errors heavily (squaring amplifies outliers). Binary Cross-Entropy — for binary classification: L = -(1/n)Σ[yᵢlog(ŷᵢ) + (1-yᵢ)log(1-ŷᵢ)]. Categorical Cross-Entropy — for multi-class: L = -(1/n)ΣΣyᵢⱼlog(ŷᵢⱼ). Mean Absolute Error (MAE) — for regression, robust to outliers: L = (1/n)Σ|yᵢ – ŷᵢ|. Huber Loss — combination of MSE and MAE. Hinge Loss — for SVM and some neural networks. KL Divergence — for VAEs and distribution matching. Choosing the right loss function is crucial — using MSE for classification or cross-entropy for regression would produce poor results. The loss function directly determines what the model optimizes for.

60. What is overfitting in neural networks and how is it prevented?

  1. Overfitting occurs when a model fits the training data too slowly due to a low learning rate
  2. Overfitting occurs when a model learns the training data too well including noise — performing excellently on training data but poorly on new unseen data — prevented through regularization, dropout, early stopping, data augmentation, and cross-validation
  3. Overfitting occurs when too many training epochs cause the model to forget earlier patterns
  4. Overfitting is when a model overfits too many GPU resources during the training process

Answer : B
Explanation: Overfitting (high variance) happens when the model memorizes training data noise rather than learning generalizable patterns — training loss is very low but validation/test loss is high. Detection: training accuracy >> validation accuracy. Prevention strategies: More training data (most effective solution). Data Augmentation — artificially expand dataset (random flips, rotations, cropping for images; synonym replacement, back-translation for text). Dropout — randomly deactivate neurons during training. L1/L2 Regularization — add weight penalty to loss function. Early Stopping — stop training when validation loss starts increasing. Reduce model capacity — fewer layers or neurons. Cross-validation — robust evaluation of generalization. Batch normalization — mild regularization effect. Pretrained models + fine-tuning — transfer learning reduces overfitting. The bias-variance tradeoff: underfitting (high bias — model too simple) vs. overfitting (high variance — model too complex). Deep learning’s default strategy: use a large enough model and regularize, rather than using a small model.

Pages: 1 2 3 4 5 6 7 8 9 10