A Data Scientist spends most of the time working with Pandas. This is why most practical questions in Data Science interviews are based on pandas concepts. So, if you are looking for Pandas concepts you should know about for Data Science interviews, this article is for you. In this article, I’ll take you through a guide to Pandas concepts for Data Science interviews, including example questions and how to solve them.
Pandas Concepts for Data Science Interviews
Here are must-know Pandas concepts for Data Science interviews, each explained in detail with an example question and its solution in Python.
MultiIndexing and Hierarchical Indexing
MultiIndexing allows you to work with multiple levels of row or column labels in your data. It is useful for working with high-dimensional data in a clear, structured way.
Example Question: You have sales data for multiple stores and products across different regions and years. Write a Python script to calculate the total sales for each store per product across the entire dataset.
Here’s the Python code to generate such data:
import pandas as pd
data = {
'Year': [2020, 2020, 2021, 2021, 2022, 2022],
'Region': ['North', 'South', 'North', 'South', 'North', 'South'],
'Store': ['A', 'A', 'B', 'B', 'C', 'C'],
'Product': ['X', 'Y', 'X', 'Y', 'X', 'Y'],
'Sales': [100, 200, 150, 250, 200, 300]
}
df = pd.DataFrame(data)
df
Here’s how to solve this problem:
# setting a multiindex on region, store, and product df.set_index(['Region', 'Store', 'Product'], inplace=True) # summing sales for each store and product combination total_sales = df.groupby(level=['Store', 'Product']).sum() print(total_sales)
Year Sales
Store Product
A X 2020 100
Y 2020 200
B X 2021 150
Y 2021 250
C X 2022 200
Y 2022 300
GroupBy with Custom Functions
GroupBy is a powerful feature in Pandas that allows you to group your data and apply aggregate functions. Using custom functions, you can go beyond the standard aggregations like sum, mean, etc., for more complex operations.
Example Question: You have a DataFrame of students’ exam scores across different subjects. Write a Python function to find the student with the maximum increase in scores between two consecutive exams.
Here’s the Python code to generate such data:
data = {
'Student': ['Alice', 'Bob', 'Alice', 'Bob', 'Alice', 'Bob'],
'Exam': [1, 1, 2, 2, 3, 3],
'Score': [85, 90, 88, 85, 95, 92]
}
df = pd.DataFrame(data)
df
Here’s how to solve this problem:
# define a custom function to calculate the score difference
def max_score_increase(group):
group = group.sort_values('Exam')
group['Score_Change'] = group['Score'].diff()
return group.loc[group['Score_Change'].idxmax()]
# apply the custom function using GroupBy
result = df.groupby('Student').apply(max_score_increase)
print(result[['Student', 'Exam', 'Score_Change']])Student Exam Score_Change
Student
Alice Alice 3 7.0
Bob Bob 3 7.0
Merging and Joining Complex DataFrames
Combining data from different sources is essential for real-world data analysis. Pandas offers different types of joins (inner, outer, left, right), and mastering how to merge complex DataFrames is critical.
Example Question: You have two DataFrames, one with product details and another with sales data. Merge them to find out which products have zero sales and display them with their details.
Here’s the Python code to generate such data:
products = pd.DataFrame({
'ProductID': [1, 2, 3, 4],
'Product': ['A', 'B', 'C', 'D']
})
sales = pd.DataFrame({
'SaleID': [101, 102],
'ProductID': [1, 3],
'Quantity': [10, 15]
})Here’s how to solve this problem:
# perform a left merge to keep all products and their sales (if any) merged_df = pd.merge(products, sales, on='ProductID', how='left') # find products with no sales (NaN in Quantity) no_sales = merged_df[merged_df['Quantity'].isna()] print(no_sales[['ProductID', 'Product']])
ProductID Product
1 2 B
3 4 D
Pivoting and Melting DataFrames
Pivoting reshapes the data to present it in a more readable format (wide to long format or vice versa) while melting unpivots it. These are crucial for handling data for visualization or advanced analysis.
Example Question: You have a dataset of monthly temperatures for several cities. Reshape the data so that you can see each city’s monthly temperatures in a wide format, where each column is a city and each row is a month.
Here’s the Python code to generate such data:
data = {
'Month': ['Jan', 'Feb', 'Mar', 'Jan', 'Feb', 'Mar'],
'City': ['New York', 'New York', 'New York', 'London', 'London', 'London'],
'Temperature': [30, 28, 35, 45, 43, 48]
}
df = pd.DataFrame(data)
df
Here’s how to solve this problem:
# pivot the DataFrame to have months as rows and cities as columns pivot_df = df.pivot(index='Month', columns='City', values='Temperature') print(pivot_df)
City London New York
Month
Feb 43 28
Jan 45 30
Mar 48 35
Time Series Manipulation
Handling time series data is crucial in many Data Science projects. Pandas provides a range of functions for parsing, resampling, and manipulating time series data effectively.
Example Question: You have a time series of daily sales for a retail store. Aggregate the data to show monthly total sales, and interpolate any missing values in the series.
Here’s the Python code to generate such data:
data = {
'Date': pd.date_range(start='2023-01-01', periods=10, freq='D'),
'Sales': [200, 250, None, 300, 350, None, 400, 450, 500, None]
}
df = pd.DataFrame(data)
df
Here’s how to solve this problem:
# convert the date column to a datetime type
df['Date'] = pd.to_datetime(df['Date'])
# set date as index
df.set_index('Date', inplace=True)
# interpolate missing sales values
df['Sales'] = df['Sales'].interpolate()
# resample to get monthly total sales
monthly_sales = df.resample('M').sum()
print(monthly_sales)Sales
Date
2023-01-31 3600.0
Summary
These Pandas concepts are frequently encountered in Data Science interviews, and understanding how to apply them in real-world scenarios will prepare you for a range of data manipulation and analysis tasks. I hope you liked this article on Pandas concepts you should know for Data Science interviews. Feel free to ask valuable questions in the comments section below. You can follow me on Instagram for many more resources.
