Neural Networks MCQ Questions And Answers

Neural Networks MCQ Questions And Answers

91. What is the number of trainable parameters in a neural network layer and why does it matter?

  1. The number of training examples that can be processed by the layer in one second
  2. The total count of weights and biases that are updated during training — determining model capacity, memory requirements, training time, and risk of overfitting — a fully connected layer of size m×n has m×n + n parameters
  3. The number of hyperparameters that need to be tuned for each individual layer
  4. The number of neurons that actively participate (non-zero activations) during one forward pass

Answer : B
Explanation: Trainable parameters are the weights and biases updated during gradient descent. Counting parameters: Dense Layer (m inputs → n outputs): m×n weights + n biases = m×n + n parameters. Convolutional Layer (F×F filter, Cin input channels, Cout output channels): F×F×Cin×Cout weights + Cout biases. Batch Normalization (size n): 2n parameters (scale γ and shift β per feature). Embedding Layer (V vocabulary, D dimensions): V×D parameters. Example: simple CNN for MNIST: Conv(1,32,3×3): 3×3×1×32 + 32 = 320 params. Conv(32,64,3×3): 3×3×32×64 + 64 = 18,496 params. Dense(9216,128): 9216×128 + 128 = 1,179,776 params. Dense(128,10): 128×10 + 10 = 1,290 params. Total ≈ 1.2M. Why it matters: determines model size (memory), computational cost (FLOPs per forward pass), risk of overfitting (more params relative to data → higher risk), training time, and hardware requirements. GPT-3 has 175 billion parameters requiring specialized hardware — counting parameters is how we understand model scale.

92. What is the Hopfield network and what is it used for?

  1. A feedforward network invented by Hopfield for fast image classification
  2. A fully connected recurrent neural network that functions as an associative memory (content-addressable memory) — storing patterns as energy minima and retrieving stored patterns from partial or noisy inputs by converging to the nearest stored pattern
  3. A convolutional network for processing Hopfield-format image data structures
  4. A network architecture invented for solving the Hopfield class of optimization problems

Answer : B
Explanation: The Hopfield Network (John Hopfield, 1982) is one of the most influential early neural network models and the most famous recurrent neural network (Q21 in your existing questions). Architecture: fully symmetric bidirectional connections between all N binary neurons (+1/-1). No self-connections. Energy Function: E = -½ΣᵢΣⱼwᵢⱼsᵢsⱼ — the network converges to states that minimize this energy. Working: Memories are stored by setting weights using Hebbian learning. When presented with a partial or noisy pattern, neurons update asynchronously until the network converges to a stored memory (local energy minimum). Capacity: approximately 0.14N patterns can be reliably stored in a network of N neurons. Applications: Content-addressable memory, error correction, combinatorial optimization (Traveling Salesman Problem). Historical significance: sparked renewed interest in neural networks in the 1980s. Modern Dense Associative Memory (Modern Hopfield Networks, 2016): exponentially larger storage capacity, connection to Transformer attention mechanisms — the attention mechanism in Transformers is mathematically equivalent to a Hopfield network retrieval operation.

93. What is the concept of feature maps in convolutional neural networks?

  1. Maps that show the geographic distribution of neural network training data
  2. The output volumes produced by convolutional filters — each filter produces one feature map (2D array) representing where in the image that filter’s pattern (edge, texture, shape) was detected, with multiple filters producing a stack of feature maps
  3. Maps showing which features in the training data are most correlated with the target class
  4. A visualization tool that maps neural network weight values to color-coded spatial grids

Answer : B
Explanation: A Feature Map (Activation Map) is the output of applying one convolutional filter to the input. If a convolutional layer has 64 filters, it produces 64 feature maps — one per filter. Each value in a feature map represents how strongly that filter’s pattern was detected at that spatial location. Visualization: visualizing feature maps reveals what each filter detects — early layer filters detect edges and colors, middle layers detect textures and parts, deep layers detect complex objects. Channel dimension: an image has 3 channels (RGB). After conv layer: the 64 feature maps constitute a 64-channel output volume. Feature map dimensions: (input_height – filter_height + 2×padding)/stride + 1 for each spatial dimension. Flattening before dense layers: feature maps are flattened into a 1D vector before the final fully connected classification layers (or replaced by Global Average Pooling in modern architectures). Feature maps are the internal representations that make CNNs so powerful — they are learned during training without any human guidance about what features to detect.

94. What is the concept of weight sharing in convolutional neural networks?

  1. A technique where multiple GPU cards share weight update computations during distributed training
  2. The principle that the same set of weights (filter) is applied to every position in the input — dramatically reducing parameters compared to fully connected layers and enforcing translation equivariance (the same feature detector works anywhere in the image)
  3. A method of sharing trained weights between different neural network architectures
  4. A technique where multiple neurons in the same layer share identical weight values

Answer : B
Explanation: Weight Sharing is the fundamental efficiency principle of CNNs. In a fully connected network applied to a 224×224×3 image: the first hidden layer would need 224×224×3 = 150,528 input connections per neuron. In a convolutional layer: a 3×3×3 filter has only 27 weights that are shared across all spatial positions. The same filter slides across every 3×3 patch of the image, detecting the same feature (e.g., horizontal edge) wherever it appears. This is the correct inductive bias for images: useful features like edges and textures can appear anywhere in an image — the same detector should work everywhere. Benefits: Dramatic parameter reduction (27 vs. 150K+ parameters per filter for first layer). Translation equivariance: shifting an input shifts the output feature map proportionally. Better generalization: fewer parameters → less overfitting. This principle appears in other architectures: Siamese networks share weights between twin branches; Transformers share attention weight matrices; LSTMs share weights across time steps.

95. What is a loss landscape in neural networks?

  1. A geographical map showing where neural network research labs are located globally
  2. The high-dimensional surface representing the loss function value at every possible combination of model weights — visualized as a landscape with peaks (high loss), valleys (low loss), saddle points, and flat plateaus that gradient descent must navigate to minimize loss
  3. A visualization tool that shows which training examples cause the highest loss values
  4. A plot of training loss values recorded during the training process over time

Answer : B
Explanation: The Loss Landscape (loss surface) is the function L(w) mapping all possible weight configurations w to a loss value. For a network with millions of parameters, this is a million-dimensional surface — impossible to visualize directly. 2D/3D visualizations project onto random directions to give intuition. Key features: Global Minimum: the lowest loss value — what we ultimately want to reach. Local Minima: valleys that trap gradient descent — not the global minimum. Saddle Points: gradient = 0, but saddle (minimum in some directions, maximum in others). More common than local minima in high-dimensional spaces. Plateaus: flat regions with near-zero gradients — slow training. Sharp vs. Flat Minima: flat minima generalize better (small perturbations → small loss change); sharp minima generalize poorly. Sharpness Aware Minimization (SAM, 2020) explicitly seeks flat minima. SGD with noise naturally finds flatter minima than batch GD. Understanding loss landscapes helps explain: why learning rate matters, why deep networks can be trained, why some regularization techniques improve generalization, and why batch size affects generalization.

96. What is self-supervised learning in neural networks?

  1. A learning paradigm where the neural network automatically supervises and grades its own outputs
  2. A learning paradigm where the model generates its own supervisory labels from unlabeled data through pretext tasks — enabling learning of rich representations from massive unlabeled datasets without expensive human annotation
  3. A training technique where the network learns to supervise other, smaller neural networks
  4. A reinforcement learning approach where the agent rewards itself based on internal success metrics

Answer : B
Explanation: Self-Supervised Learning (SSL) creates supervision signals automatically from the structure of data — no human labels required. Pretext Tasks — tasks designed so that solving them requires learning useful representations: Masked Language Modeling (BERT): predict randomly masked tokens using bidirectional context. Next Token Prediction (GPT): predict the next word from previous words. Contrastive Learning (SimCLR, MoCo): learn that different augmented views of the same image are similar (positive pairs), different images are dissimilar (negative pairs). Masked Autoencoders (MAE): predict masked image patches from visible patches. SimSiam, BYOL: learn representations without negative pairs. Why SSL is transformative: enables learning from the internet-scale unlabeled data (billions of images, trillions of text tokens). The resulting representations (foundation models) transfer to dozens of downstream tasks with minimal fine-tuning. BERT, GPT-4, CLIP, DALL-E, and virtually every state-of-the-art model today uses self-supervised pre-training. SSL has largely solved the data labeling bottleneck in deep learning.

97. What is contrastive learning in neural networks?

  1. A learning method where neural networks learn by contrasting their performance against other models
  2. A self-supervised learning approach where a model learns representations by attracting embeddings of similar samples (positive pairs) and repelling embeddings of dissimilar samples (negative pairs) — without requiring class labels
  3. A supervised learning technique that contrasts different class labels to improve discrimination
  4. A training method that alternates between contrasting easy and hard training examples

Answer : B
Explanation: Contrastive Learning trains models to produce similar representations for similar inputs and different representations for different inputs. Key frameworks: SimCLR (Chen et al., 2020): take one image, apply two random augmentations → two views. Push their representations together (positive pair). Push representations from different images apart (negative pairs). Use NT-Xent (Normalized Temperature-scaled Cross Entropy) loss. MoCo (He et al., Facebook): uses momentum-updated memory bank of negative examples. BYOL (Bootstrap Your Own Latent): learns without negative pairs — uses online + target network architecture. InfoNCE Loss: L = -log[exp(sim(z₁,z₂)/τ) / Σₖexp(sim(z₁,zₖ)/τ)] — attracts positive pair, repels K negative pairs. CLIP (OpenAI): contrastive learning between image and text pairs — learns joint vision-language representations. Results: SimCLR pre-trained features transfer better than supervised ImageNet features for many tasks. Contrastive learning is one of the most active research areas in deep learning, powering foundation models like CLIP that enable zero-shot image classification.

98. What is federated learning in the context of neural networks?

  1. A learning approach where multiple organizations cooperatively design neural network architectures
  2. A privacy-preserving distributed learning approach where neural network models are trained locally on distributed devices without sharing raw data — only model updates (gradients) are sent to a central server for aggregation
  3. A federated (government) regulation requiring neural networks to be trained on certified data
  4. A multi-model architecture where separate neural networks are federally combined at inference

Answer : B
Explanation: Federated Learning (FL), introduced by Google (2017), enables training neural networks on decentralized data without centralizing it. Process: Central server initializes global model. Model sent to each participating device (smartphone, hospital, bank). Each device trains on its local data, computes gradient updates. Only gradient updates sent back to server (not raw data). Server aggregates updates (FedAvg: weighted average of gradients). Repeat for many rounds until convergence. Why it matters: Privacy: sensitive data (medical records, financial transactions, personal messages) never leaves the device. Regulations: GDPR, HIPAA compliance by design. Bandwidth: only small model updates transmitted, not large datasets. Applications: Google Keyboard (Gboard) — next-word prediction improved using your typing without sending messages to Google. Healthcare: train disease detection models across hospitals without sharing patient records. Banking: fraud detection across institutions. Challenges: Non-IID data (data distribution differs across devices), communication efficiency, convergence guarantees, handling dropped devices. Differential Privacy is often combined with FL for stronger privacy guarantees.

99. What is the concept of attention heads in multi-head attention?

  1. The senior management team of researchers who direct the attention mechanism research project
  2. Multiple parallel attention mechanisms each learning different aspects of the relationship between tokens — each head uses separate Q, K, V projection matrices, enabling the model to jointly attend to information from different representation subspaces simultaneously
  3. The first and last attention layers in a Transformer that head (lead) the processing pipeline
  4. Attention operations performed on the head (beginning) and tail (end) tokens of a sequence

Answer : B
Explanation: Multi-Head Attention runs h parallel attention operations simultaneously: For each head i (i = 1…h): project Q, K, V using separate learned matrices: Qᵢ = QWᵢQ, Kᵢ = KWᵢK, Vᵢ = VWᵢV. Compute attention: headᵢ = Attention(Qᵢ, Kᵢ, Vᵢ). Concatenate: MultiHead(Q,K,V) = Concat(head₁,…,headₕ)W^O. Typical: 8 or 16 heads, each with d_model/h dimensions. Why multiple heads? Each head can attend to different types of relationships simultaneously: One head might capture syntactic dependencies (subject-verb agreement). Another captures semantic relationships (word meanings). Another captures positional relationships. Another captures co-reference (pronouns and their referents). Analogy: like using multiple types of dictionaries simultaneously — one for grammar, one for semantics, one for pragmatics. BERT-base uses 12 attention heads, GPT-3 uses 96. Research shows different heads indeed specialize in different linguistic phenomena — some can be pruned with minimal accuracy loss.

100. What are the current trends and frontier research areas in neural networks?

  1. The current research focuses exclusively on improving image classification accuracy on ImageNet
  2. Key frontier areas include: Large Language Models (LLMs) and scaling laws, Multimodal models (vision + language + audio), Efficient AI (quantization, pruning, distillation for edge deployment), Mixture of Experts (MoE), State Space Models (Mamba), AI safety and alignment, and Agentic AI systems
  3. Neural network research has peaked — all major problems are solved with current architectures
  4. Current research focuses only on reducing the environmental impact of training costs

Answer : B
Explanation: Neural network research is advancing rapidly across multiple frontiers: Large Language Models and Scaling: GPT-4, Claude, Gemini, Llama demonstrate emergent capabilities at scale. Scaling laws predict how performance improves with more data, compute, and parameters. Multimodal Models: GPT-4V, Gemini Ultra, CLIP — models that process text, images, audio, video together. Mixture of Experts (MoE): models with billions of parameters but only a fraction active per token — efficient scaling. Used in GPT-4 reportedly, Mixtral. State Space Models: Mamba — linear-time sequence modeling as a Transformer alternative, more efficient for long sequences. Efficient AI: 4-bit quantization making 70B models run on consumer hardware, speculative decoding for faster inference. Retrieval Augmented Generation (RAG): combining neural networks with external knowledge bases. AI Agents: autonomous agents using LLMs to plan and execute multi-step tasks. Neural Scaling + Interpretability: mechanistic interpretability research (Anthropic) — understanding what neural networks compute. Physics-informed Neural Networks: incorporating domain knowledge. These trends define the cutting edge of neural network research and are increasingly tested in advanced placement and graduate-level interviews.

Pages: 1 2 3 4 5 6 7 8 9 10