Guided Notes
Transform: Missing & Duplicate Data
Finding gaps, reshaping wide data with melt, filling with group means and forward/back fill, and removing duplicates.
Missing data: delete or impute?
Slide 1 / 5Many machine learning algorithms can't handle missing values, so a pipeline must decide what to do with them:
• Delete — drop rows (dropna) or whole columns that are mostly empty. Simple, but you lose information. • Impute — fill the gaps with a sensible estimate: a mean, median, mode, or neighboring value.
The right choice depends on how much is missing and why. Always count first: df.isnull().sum().