81. What is the difference between a data lake and a data warehouse?
- A data lake stores only structured data; a data warehouse stores only unstructured data
- A data lake stores raw data in its native format (structured, semi-structured, unstructured) at scale; a data warehouse stores processed, structured data optimized for analytical querying and reporting
- Data warehouses are always larger than data lakes in storage capacity
- A data lake uses SQL for all queries; a data warehouse requires Python for analysis
Answer : B Explanation: Data Lake: stores all types of raw data (structured, semi-structured, unstructured) in their native format at massive scale — cheap storage (AWS S3, Azure Data Lake). Schema is applied at read time (schema-on-read), making it flexible. Good for data exploration, ML training, and use cases that are not yet defined. Risk: can become a “data swamp” without proper governance. Data Warehouse: stores processed, cleaned, structured data optimized for analytical queries — fast, consistent, reliable. Schema-on-write. Good for business intelligence, dashboards, and reporting (Snowflake, BigQuery, Redshift). Modern “lakehouse” architectures (Delta Lake, Apache Iceberg) combine benefits of both.
82. What is the concept of data governance in data science?
- The process of governing which machine learning models are allowed to be deployed
- A framework of policies, processes, and standards that ensure data quality, security, privacy, and compliance throughout its lifecycle within an organization
- A voting system where data scientists govern which datasets are used for analysis
- The technical process of managing database access permissions for analysts
Answer : B Explanation: Data Governance is the organizational framework for managing data as a strategic asset. Key components: Data Quality (ensuring accuracy, completeness, consistency, timeliness), Data Catalog (documenting available datasets, their definitions, and owners), Data Lineage (tracking where data comes from and how it has been transformed), Data Privacy and Security (GDPR, CCPA compliance, access control), Master Data Management (single source of truth for critical entities like customers and products), and Data Stewardship (assigning responsibility for data domains). Good data governance ensures that data scientists work with trustworthy, well-documented data — improving analysis quality and regulatory compliance.
83. What is the role of statistics in data science?
- Statistics is only used for data visualization in data science projects
- Statistics provides the mathematical foundation for data collection, description, inference, hypothesis testing, probability modeling, and model evaluation — underpinning every stage of the data science workflow
- Statistics is replaced entirely by machine learning in modern data science projects
- Statistics is used only for analyzing historical data and cannot inform future predictions
Answer : B Explanation: Statistics is the backbone of data science. Its role spans the entire workflow: Descriptive Statistics (summarizing data — mean, std, percentiles, distributions), Inferential Statistics (drawing conclusions about populations from samples), Probability Theory (modeling uncertainty — distributions, Bayes’ theorem), Hypothesis Testing (A/B testing, determining if effects are statistically significant), Regression Analysis (modeling relationships for prediction), Experimental Design (designing valid experiments), and Model Evaluation (understanding precision, recall, confidence intervals). A data scientist without strong statistical foundations will misinterpret results, draw wrong conclusions, and make poor decisions — statistics is non-negotiable.
84. What is the difference between Pearson and Spearman correlation coefficients?
- Pearson measures non-linear correlations; Spearman measures only linear correlations
- Pearson measures the linear correlation between two continuous normally distributed variables; Spearman measures the monotonic relationship between variables using ranks — more robust to outliers and non-normal data
- Spearman correlation ranges from 0 to 1; Pearson ranges from -1 to 1
- Both coefficients always produce identical values for any given dataset
Answer : B Explanation: Pearson’s r measures the linear relationship between two continuous variables and assumes normal distribution, constant variance, and absence of significant outliers. It measures how much the variables move together proportionally. Spearman’s ρ (rho) is a non-parametric alternative that converts both variables to ranks and then computes Pearson on the ranks. It measures monotonic relationships (not necessarily linear) and is robust to outliers and non-normal distributions. Use Pearson for normally distributed continuous data with linear relationships. Use Spearman for ordinal data, skewed distributions, or when outliers are present. Both range from -1 to 1.
85. What is the difference between batch processing and stream processing in data science?
- Batch processing handles small data; stream processing handles large data only
- Batch processing collects and processes data in large groups at scheduled intervals; stream processing continuously ingests and processes data in real time as it arrives
- Stream processing is always more accurate than batch processing for analytics
- Both approaches produce identical results with different hardware requirements
Answer : B Explanation: Batch Processing collects data over a period and processes it all at once in scheduled jobs (hourly, daily, weekly). Tools: Apache Hadoop MapReduce, Apache Spark (batch mode). Good for: end-of-day reports, training ML models on historical data, periodic ETL jobs. Higher latency but simpler and more cost-effective for historical analysis. Stream Processing (Real-Time Processing) continuously processes data as it arrives, milliseconds to seconds after generation. Tools: Apache Kafka, Apache Flink, Apache Spark Streaming, Amazon Kinesis. Good for: fraud detection, real-time recommendations, monitoring dashboards, IoT sensor analysis. Lambda Architecture combines both batch and streaming for comprehensive solutions.
86. What is the purpose of data visualization in data science?
- Data visualization is used only for presenting final results to management
- To communicate data insights, patterns, trends, and relationships in a visual format that is easier for humans to understand and interpret than raw numbers or tables
- To decorate data science reports with attractive charts for publication purposes
- Data visualization replaces statistical analysis in modern data science workflows
Answer : B Explanation: Data Visualization serves multiple critical purposes in data science. During EDA: histograms reveal distributions; scatter plots show relationships; box plots expose outliers; heatmaps show correlations. During modeling: learning curves diagnose overfitting; confusion matrices show classification errors; ROC curves compare models. During communication: dashboards (Tableau, Power BI, Plotly Dash) convey insights to non-technical stakeholders. The human visual system processes patterns, trends, and anomalies far more efficiently than scanning numbers in tables. “A picture is worth a thousand data points.” Good data visualization is an essential data science communication skill alongside technical modeling ability.
87. What is the concept of data sampling bias in data science?
- A systematic error in a model caused by using an insufficiently biased loss function
- A systematic error that occurs when the sample used to train or analyze is not representative of the population it is meant to represent, causing skewed conclusions and biased models
- A computational bias in random number generators that affects data sampling quality
- A statistical bias caused by collecting too many samples from a population
Answer : B Explanation: Sampling Bias occurs when some members of a population are systematically more or less likely to be included in the sample, making conclusions non-generalizable. Types: Selection Bias (non-random selection — e.g., online surveys reach only internet users), Survivorship Bias (analyzing only “survivors” — e.g., studying only successful startups), Confirmation Bias (seeking data that confirms existing beliefs), and Availability Bias (using data that is easy to collect). Real-world example: training a hiring ML model on historical hiring decisions that favored certain demographics will perpetuate discrimination. Addressing sampling bias requires understanding data collection processes and using techniques like stratified sampling.
88. What is the difference between a supervised learning algorithm and a self-supervised learning algorithm?
- Self-supervised algorithms require more human supervision than supervised algorithms
- Supervised learning requires human-labeled data; self-supervised learning generates its own labels automatically from unlabeled data through pretext tasks, enabling training on massive datasets without expensive annotation
- Self-supervised learning is a subset of supervised learning using automated labeling
- Both require identical amounts of labeled training data to achieve similar performance
Answer : B Explanation: Supervised Learning requires expensive, time-consuming human annotation of training data. Self-Supervised Learning (SSL) automatically creates labels from the data itself. Examples: BERT predicts masked words — no human labeling needed; GPT predicts the next token; SimCLR learns image representations by treating augmented views of the same image as positives. SSL has enabled training on billions of unlabeled examples, producing powerful foundation models (LLMs, CLIP) that are fine-tuned for specific tasks with minimal labeled data. It represents a paradigm shift that dramatically reduces the data labeling bottleneck and has been the driving force behind modern large-scale AI models.
89. What is the CRISP-DM process model in data science?
- A Python library for cross-platform data science model development and deployment
- Cross-Industry Standard Process for Data Mining — a widely used framework with six phases: Business Understanding, Data Understanding, Data Preparation, Modeling, Evaluation, and Deployment
- A machine learning algorithm for clustering data using cross-industry benchmarks
- A data validation process that cross-checks models across different industry datasets
Answer : B Explanation: CRISP-DM (Cross-Industry Standard Process for Data Mining) is the most widely used data science project methodology. Its six cyclical phases: Business Understanding (define objectives and success criteria), Data Understanding (collect data and explore for quality issues), Data Preparation (clean, transform, and engineer features), Modeling (select and train algorithms), Evaluation (assess against business objectives — did we solve the problem?), and Deployment (deliver the solution to stakeholders or production). The process is iterative — insights often require returning to earlier phases. CRISP-DM provides a structured approach that ensures data science projects align with business goals, not just technical metrics.
90. What is the curse of dimensionality and how does it impact data science?
- A hardware limitation preventing data scientists from training on datasets with more than 1000 dimensions
- A phenomenon where algorithms become increasingly ineffective and computationally expensive as the number of features grows, because data becomes exponentially sparse in high-dimensional space
- A software bug that occurs when datasets with too many dimensions are loaded into memory
- A regulatory restriction limiting the number of variables used in financial ML models
Answer : B Explanation: The Curse of Dimensionality describes problems that arise as feature dimensions increase. In high-dimensional spaces: data becomes extremely sparse (more space to fill), distances between points lose meaning (all points become roughly equidistant), the volume of the feature space grows exponentially requiring exponentially more data to maintain statistical significance, and visualization becomes impossible. Impact on algorithms: distance-based methods (KNN, K-Means, SVM) degrade significantly. Solutions: Dimensionality Reduction (PCA, t-SNE, UMAP), Feature Selection (removing irrelevant features), and Feature Engineering (creating more informative features). Understanding this phenomenon is essential for designing effective data science solutions.
