Neural Networks MCQ Questions And Answers

Neural Networks MCQ Questions And Answers

71. What are the main components of a biological neuron and how do they map to an artificial neuron?

  1. Biological neurons have circuits and transistors; artificial neurons have gates and registers
  2. Biological neuron components: dendrites (receive inputs) → cell body/soma (integrates signals) → axon (transmits output). Mapped to artificial: input signals (xᵢ) → weighted sum + activation function → output (aᵢ). Synaptic strength maps to weights (wᵢ)
  3. Biological neurons have RAM and ROM; artificial neurons simulate this with matrices and vectors
  4. All biological components map directly one-to-one with artificial neuron software modules

Answer : B
Explanation: The artificial neuron is inspired by (not a replica of) the biological neuron: Dendrites → Inputs (x₁, x₂, …, xₙ): dendrites receive electrical signals from other neurons; inputs are numerical values from previous layer neurons. Synapses → Weights (w₁, w₂, …, wₙ): synaptic strength (how strongly one neuron influences another) maps to connection weights. Stronger synapse = higher weight. Cell Body/Soma → Weighted Sum + Activation: the soma integrates all dendritic signals; the artificial neuron computes z = Σwᵢxᵢ + b, then applies f(z). Axon → Output (a): the axon transmits the cell’s output signal to other neurons; the artificial neuron passes its activation value to the next layer. All-or-nothing firing (action potential) is approximated by step function (perceptron) or smooth approximations (sigmoid, tanh, ReLU). Key differences: biological neurons are far more complex (thousands of dendritic compartments, spike timing matters, chemical signaling) — artificial neurons are dramatic simplifications that nonetheless achieve remarkable results.

72. What is the difference between supervised, unsupervised, and reinforcement learning in neural networks?

  1. Supervised uses more layers; unsupervised uses fewer layers; reinforcement uses no layers
  2. Supervised learning trains on labeled examples to predict outputs; unsupervised learning discovers patterns in unlabeled data; reinforcement learning trains agents through reward-based trial and error interactions with an environment
  3. Supervised is for regression; unsupervised is for classification; reinforcement is for clustering
  4. All three paradigms use identical neural network architectures but different loss functions only

Answer : B
Explanation: Three fundamental machine learning paradigms: Supervised Learning: labeled data (x, y pairs). Neural network learns f(x) → y. Loss function measures prediction error. Examples: CNNs for image classification, LSTMs for machine translation, MLPs for spam detection. Unsupervised Learning: unlabeled data (x only). Network discovers inherent structure. Examples: Autoencoders (compression), GANs (generation), k-means clustering, Variational Autoencoders (generative models), Self-Supervised Learning (BERT, GPT — create their own labels from data structure). Reinforcement Learning: agent interacts with environment, receives rewards/penalties. Learns policy to maximize cumulative reward. Neural network approximates value function or policy. Examples: Deep Q-Network (DQN) for Atari games, AlphaGo, robotic control, self-driving cars, ChatGPT’s RLHF training. Semi-Supervised Learning (intermediate): small labeled + large unlabeled dataset. The majority of practical deep learning applications use supervised learning, though self-supervised pre-training (GPT, BERT) is increasingly dominant for foundation models.

73. What is the forward pass in a neural network?

  1. The process of sending model weights forward to the next training iteration
  2. The process of propagating input data through the network layer by layer — computing weighted sums, applying activation functions, and producing a final output or prediction — before any weight updates occur
  3. The initial step in backpropagation where gradients move forward through the network
  4. A pass through only the first half of a neural network before splitting at the bottleneck layer

Answer : B
Explanation: The Forward Pass (Forward Propagation) computes the network’s prediction: Input layer: receives raw input data (pixel values, word embeddings, numerical features). For each hidden layer l: compute net input: z^l = W^l × a^(l-1) + b^l. Apply activation: a^l = f(z^l). Repeat until output layer. Output layer: compute final prediction (class probabilities, regression value). Compute loss: L(y, ŷ) — how wrong is the prediction? The forward pass runs once before backpropagation. During inference (prediction, not training), only the forward pass runs — no backpropagation needed. Time complexity: O(parameters) per example — with millions of parameters, efficient matrix operations (GPU acceleration) are essential. In code (PyTorch): output = model(input) runs the forward pass automatically by calling each layer’s forward() method. After the forward pass, backpropagation computes gradients to update weights.

74. What is backpropagation through time (BPTT) in recurrent neural networks?

  1. A technique for training RNNs on historical time series data from the past
  2. The algorithm for training RNNs by unrolling the network across time steps and applying standard backpropagation to the unrolled network — computing gradients with respect to all time steps simultaneously
  3. A training method that propagates gradients backward through the time dimension of the input data
  4. A real-time training technique where weight updates happen backward from the current time moment

Answer : B
Explanation: BPTT (Backpropagation Through Time) extends standard backpropagation to recurrent networks. Since RNNs process sequences by sharing weights across time steps, training requires: Unrolling the RNN across all T time steps — creating a very deep feedforward network where each “layer” corresponds to one time step. Applying standard backpropagation to this unrolled network. Summing gradients across all time steps (weights are shared). The problem: for long sequences (T=100), the gradient must propagate through T multiplication steps — leading to vanishing (or exploding) gradients. Solutions: Truncated BPTT — only backpropagate through the last k time steps instead of all T (limits gradient horizon). LSTM/GRU gates — preserve gradients over long sequences. Gradient Clipping — prevents explosion. In practice: PyTorch and TensorFlow implement BPTT automatically through their autograd systems when you call loss.backward() on an RNN’s output.

75. What is the purpose of the bias term in a neural network neuron?

  1. The bias introduces unfair advantage to some neurons, ensuring the strongest ones dominate
  2. The bias is a learnable offset added to the weighted sum before the activation function — allowing the activation threshold to be shifted independently of the input values, giving the neuron the flexibility to activate even when all inputs are zero
  3. The bias term balances the network’s positive and negative weight values during training
  4. The bias controls the learning rate for individual neurons during gradient descent

Answer : B
Explanation: The bias b in z = Σwᵢxᵢ + b allows the activation function to be shifted left or right. Why it’s essential: Without bias: if all inputs x = 0, then z = 0 regardless of weights. The neuron can only activate based on the magnitude of inputs — limited flexibility. With bias: the neuron can fire (activate) even when all inputs are zero, if bias is large enough. The bias shifts the “decision boundary” away from the origin. Geometric interpretation: in a 2D classification task, the decision boundary is a line. Without bias: line must pass through the origin (severely restricted). With bias: line can be anywhere in the plane. In matrix notation: z = Wx + b. The bias is a separate learnable parameter (not connected to inputs). Initialization: biases are typically initialized to 0 (unlike weights which need careful initialization to break symmetry). In PyTorch, bias=True is the default for all linear layers — you rarely need to change this.

76. What is the difference between online learning and batch learning in neural networks?

  1. Online learning uses internet data; batch learning uses locally stored datasets
  2. Online learning updates model weights after each individual training example — enabling continuous adaptation to new data; batch learning updates weights only after processing the entire dataset — more stable but cannot adapt in real-time
  3. Online learning trains only the final layer; batch learning trains all layers simultaneously
  4. Batch learning is faster for all neural network training tasks regardless of dataset size

Answer : B
Explanation: Learning paradigms differ in when weight updates occur: Online Learning (SGD with batch size=1): update weights after every single training example. Pros: adapts instantly to new patterns, less memory required, can handle streaming data, noisy updates can help escape local minima. Cons: very noisy gradient estimates, slower overall convergence. Batch Learning (Gradient Descent with full batch): process all training examples, then update once. Pros: accurate gradient estimate, stable convergence. Cons: very slow for large datasets (one update per full pass), cannot handle streaming data, requires all data in memory. Mini-batch Learning (most common in deep learning): process small batches (32-512 examples), update after each batch. Balances: more accurate gradients than online, faster than full batch, memory-efficient. In practice: most deep learning uses mini-batch SGD (confusingly often called “SGD” or “Adam”) with batch sizes of 32-256. “Online learning” in industry context also refers to models that continuously learn from new data in production.

77. What is a Boltzmann Machine in neural networks?

  1. A neural network architecture invented by Ludwig Boltzmann for thermodynamic simulation
  2. A stochastic generative neural network where all neurons are connected to each other (fully connected, bidirectional), learning probability distributions over input data — the Restricted Boltzmann Machine (RBM) removes connections within the same layer for tractable training
  3. A deterministic feedforward network that uses Boltzmann temperature scheduling for learning
  4. A deep convolutional network used for large-scale image generation tasks

Answer : B
Explanation: A Boltzmann Machine is a stochastic recurrent network where all neurons are bidirectionally connected. Each neuron is binary (0 or 1) and stochastic — it activates based on probability. The network learns a probability distribution over training data, enabling generation of new samples. Problem: training full Boltzmann Machines is computationally intractable for large networks. Solution — Restricted Boltzmann Machine (RBM): connections are restricted — visible layer (data) and hidden layer (features), but no connections within each layer. This makes training tractable using Contrastive Divergence. Deep Belief Network (DBN): stack of RBMs, pre-trained greedily layer by layer — was historically important for deep learning (Hinton et al., 2006 paper sparked the deep learning renaissance). Applications: collaborative filtering (Netflix recommendation), dimensionality reduction, feature learning. Today, RBMs and Boltzmann Machines are largely superseded by autoencoders, VAEs, and GANs — but they remain important historically and conceptually.

78. What is the difference between a dense (fully connected) layer and a convolutional layer?

  1. Dense layers are thicker than convolutional layers in terms of physical computation depth
  2. A dense layer connects every neuron to every neuron in the adjacent layers (O(n²) parameters) — suitable for learning global patterns from fixed-size inputs; a convolutional layer uses shared weight filters applied locally (O(filter_size² × channels) parameters) — exploiting spatial structure and requiring far fewer parameters
  3. Dense layers are only used in output layers; convolutional layers are only in input layers
  4. Convolutional layers use dropout; dense layers use batch normalization for regularization

Answer : B
Explanation: Dense (Fully Connected) Layer: each input neuron connects to each output neuron. Parameters: input_size × output_size + output_size (bias). For 1000 inputs → 1000 outputs: 1,001,000 parameters. No spatial awareness — cannot exploit local structure in images. Good for: final classification layer (global decision making), tabular data. Convolutional Layer: each output neuron connects only to a local region (filter size) of the input. Weights are shared across all positions (same filter scanned across entire input). Parameters: filter_size × filter_size × in_channels × out_channels + out_channels. For 3×3×64 filter with 128 output channels: only 73,856 parameters — regardless of input image size! Good for: exploiting spatial structure in images, audio, video. Efficiency comparison: a dense layer on a 224×224×3 image as input would have ~150 million parameters just for the first layer. A CNN first layer has perhaps 1,000 parameters. This is why CNNs are used for images — they are orders of magnitude more parameter-efficient.

79. What is the sequence-to-sequence (seq2seq) model in neural networks?

  1. A neural network that converts sequential (ordered) data into unordered set representations
  2. An encoder-decoder neural network architecture that maps an input sequence of one length to an output sequence of a different length — used for machine translation, text summarization, speech recognition, and question answering
  3. A model that processes sequences twice — once forward and once backward simultaneously
  4. A sequential series of neural networks where each model’s output becomes the next model’s input

Answer : B
Explanation: Seq2Seq (Sequence-to-Sequence), introduced by Sutskever et al. (Google, 2014), maps variable-length input sequences to variable-length output sequences. Architecture: Encoder — reads the entire input sequence and compresses it into a fixed-size context vector (the “thought vector”). Typically an LSTM or GRU. Decoder — generates the output sequence one token at a time, using the context vector and previously generated tokens. Problem: fixed-size context vector is a bottleneck — for long sequences, the encoder struggles to compress all information. Solution: Attention Mechanism (Bahdanau, 2014) — allows the decoder to “attend” to different parts of the encoder’s hidden states at each generation step, dramatically improving performance on long sequences. This attention-augmented seq2seq became the basis for the Transformer. Applications: Neural Machine Translation (Google Translate uses seq2seq with attention), Text Summarization, Speech Recognition, Image Captioning (CNN encoder + LSTM decoder), Chatbots.

80. What is the role of the softmax temperature parameter in neural networks?

  1. A physical temperature sensor that monitors GPU heat during softmax computation
  2. A parameter T in the scaled softmax σ(z/T) that controls the sharpness of the output probability distribution — higher temperature produces softer (more uniform) distributions, lower temperature produces sharper (more confident) distributions — used in knowledge distillation and language model sampling
  3. The processing temperature maintained by the network’s hardware during softmax inference
  4. A training schedule parameter that gradually increases softmax precision over epochs

Answer : B
Explanation: Temperature-scaled Softmax: σ(z/T) divides logits by temperature T before applying softmax. T = 1 (default): standard softmax behavior. T > 1 (high temperature): softer probabilities — more uniform distribution — more “creative” or “exploratory” outputs. Used in knowledge distillation (teacher model uses high T to provide “soft labels” that contain more information than hard one-hot labels). Used in language model sampling (ChatGPT temperature slider — higher T = more creative/random outputs). T < 1 (low temperature): sharper probabilities — model is more confident and deterministic. T → 0: equivalent to argmax (always pick the highest probability class). T → ∞: equivalent to uniform random sampling. In language model generation: T=0 (greedy/deterministic), T=0.7 (balanced creativity), T=1.0 (standard), T=1.5 (more random). Understanding temperature is important for deploying LLMs and for knowledge distillation, one of the most efficient model compression techniques.

Pages: 1 2 3 4 5 6 7 8 9 10