classification_features: Questions and suggestions for revision
Review these after reading the lab draft. I'll revise based on your responses.
Open questions
1. Dataset access. The lab downloads the SMS Spam Collection from the UCI repository at runtime. This is convenient but requires internet access and depends on a third-party URL being stable. Should we bundle the dataset in the repo instead? (It's ~500KB, small enough to include.)
2. Scope of data cleaning. The current lab covers duplicates, class imbalance, and encoding quirks. Should it also include a discussion of missing data (there is none in this dataset) and outliers (some messages are very long)? Or is it better to keep the data-cleaning section focused on what's actually in this dataset?
3. Cross-validation. The lab uses a single train/test split. Cross-validation (k-fold CV) would be more robust and is the industry standard—but it adds complexity. Should I add a brief section on CV, or save it for a later lab?
4. The matrices segue. The devnote sketch says "At this point, we introduce matrices." This is now not in the lab. I assumed the matrix introduction would come naturally in the neural networks lab when we discuss the forward pass. Is that right, or should this lab end with a brief introduction to matrix representation of feature vectors?
5. TF-IDF. The original sketch included TF-IDF as a way to go beyond hand-designed features. Currently TF-IDF is not in the lab—I moved it to classification_neural as an extension. Should it be here instead, as a bridge to learned features?
6. Duration. The lab is marked as 8 periods, matching the other labs. Do the four parts feel like they fit 8 periods? Part 4 (logistic regression + tuning) might run short—should I add more depth there, or use that time for a class competition to build the best classifier?
Suggestions for improvement
-
Add a confusion matrix visualization. Right now confusion matrices are printed as text. A heatmap (using seaborn or matplotlib) would be more readable and help students quickly identify which classes are confused.
-
Add a "most confused messages" section. After evaluating the logistic regression, show students the messages it got most wrong (highest confidence in the wrong direction). These are often illuminating and generate good discussion.
-
Feature engineering competition. After Part 3, consider a brief competition: which group can build the most informative single feature? Measure by the increase in F1 when that feature is added to a fixed baseline.
-
The spam context is slightly dated. SMS spam from 2011 differs from the phishing, scam texts, and robocalls students encounter today. Consider whether to update the examples used in the exploration section, or to frame the historical dataset as an artifact worth studying.