Your First ML Classifier: Building a Spam Filter

Your First ML Classifier: Building a Spam Filter

What is Spam and Why Filter It?

Spam: Unsolicited messages, often trying to trick users into clicking links or sharing personal info.

Types of Spam:

• Email spam: Phishing, promotional emails, scams
• SMS spam: OTP scams, loan offers, prize claims targeting Indian phone users
• Social media spam: Fake profiles, spam comments
• Web spam: Spammy comments, malicious links

Text Classification Pipeline

1. Collect spam and non-spam messages
2. Preprocess text (lowercase, remove punctuation, etc.)
3. Convert text to numerical features
4. Train classifier
5. Evaluate performance
6. Deploy and monitor

Step 1: Text Preprocessing

import re
import string

def preprocess_text(text): """Clean and preprocess text""" # Lowercase text = text.lower() # Remove URLs text = re.sub(r'httpS+|wwwS+', '', text) # Remove email addresses text = re.sub(r'S+@S+', '', text) # Remove punctuation text = text.translate(str.maketrans('', '', string.punctuation)) # Remove extra whitespace text = ' '.join(text.split()) return text

# Example
spam_msg = "CLICK NOW!!! Win Prize at www.scam.com Or call 9999999999!!!"
print(preprocess_text(spam_msg))
# output: click now win prize or call 9999999999

Step 2: Feature Extraction - Bag of Words

Bag of Words: Represent text as collection of word occurrences, ignoring order.

from sklearn.feature_extraction.text import CountVectorizer

# Sample messages
messages = [ "Click here to win free prize", "Meeting at 2pm tomorrow", "Urgent action required confirm account", "Can you call me back",
]

# Create bag of words
vectorizer = CountVectorizer()
X = vectorizer.fit_transform(messages)

# Show feature names (unique words)
print(vectorizer.get_feature_names_out())
# ['account', 'action', 'back', 'call', 'can', 'click', ...]

# Show sparse matrix (words count in each message)
print(X.toarray())
# [[0 0 0 0 0 1 0 1 ...],  # message 1
#  [0 0 0 0 0 0 0 0 ...],  # message 2
#  ...]

Step 3: Feature Extraction - TF-IDF

TF-IDF: Term Frequency-Inverse Document Frequency. Weight words: common words get lower weight, rare words get higher weight.

Why TF-IDF is better than Bag of Words:

• Common words like "the", "is", "a" get low weight (not useful for classification)
• Spam-specific words like "win", "prize", "click" get high weight
• Better discrimination between spam and ham

from sklearn.feature_extraction.text import TfidfVectorizer

# Create TF-IDF features
tfidf = TfidfVectorizer(max_features=100)  # top 100 words
X_tfidf = tfidf.fit_transform(messages)

print(X_tfidf.toarray())
# Values between 0 and 1, representing importance of each word

Step 4: Training - Naive Bayes Classifier

Naive Bayes: Probabilistic classifier based on Bayes theorem. "Naive" assumes features are independent.

How it works:

# P(Spam | Message) = P(Message | Spam) * P(Spam) / P(Message)
# 
# P(Spam | Message): Probability message is spam given words in it
# P(Message | Spam): Probability of words given message is spam
# P(Spam): Prior probability of spam (% of training data that is spam)
# P(Message): Probability of message (normalization constant)

# Simple example:
# Word "free" appears in 80% of spam, 5% of ham
# Word "meeting" appears in 2% of spam, 40% of ham
# Message contains "free": likely spam
# Message contains "meeting": likely ham

Complete Spam Filter Implementation

from sklearn.feature_extraction.text import TfidfVectorizer
from sklearn.naive_bayes import MultinomialNB
from sklearn.pipeline import Pipeline
from sklearn.model_selection import train_test_split
from sklearn.metrics import accuracy_score, confusion_matrix

# Sample dataset
messages = [ # Spam messages "Click here to win free money", "Congratulations you won a prize", "Urgent action required verify account", "Limited time offer claim now", # Ham (legitimate) messages "Hi how are you doing", "Meeting scheduled for tomorrow", "Can we discuss the project", "Thanks for your message",
]

labels = [1, 1, 1, 1, 0, 0, 0, 0]  # 1: spam, 0: ham

# Split data
X_train, X_test, y_train, y_test = train_test_split( messages, labels, test_size=0.25, random_state=42
)

# Create pipeline: TF-IDF + Naive Bayes
pipeline = Pipeline([ ('tfidf', TfidfVectorizer(max_features=50)), ('classifier', MultinomialNB())
])

# Train
pipeline.fit(X_train, y_train)

# Predict
y_pred = pipeline.predict(X_test)

# Evaluate
accuracy = accuracy_score(y_test, y_pred)
print(f"Accuracy: {accuracy:.2%}")

Step 5: Evaluation - Confusion Matrix

Confusion Matrix: Shows correct and incorrect predictions.

from sklearn.metrics import confusion_matrix, classification_report
import numpy as np

# Predictions vs actual
y_actual = [1, 0, 1, 0, 1, 0]
y_predicted = [1, 0, 0, 0, 1, 1]

# Confusion matrix
cm = confusion_matrix(y_actual, y_predicted)
print(cm)
# [[3 1]  -> True Negatives (TN=3), False Positives (FP=1)
#  [1 1]] -> False Negatives (FN=1), True Positives (TP=1)

# Detailed metrics
print(classification_report(y_actual, y_predicted))

Metrics Explained

Metric	Formula	Meaning	In Spam Filter
True Positive (TP)	-	Spam predicted as spam	Correctly blocked spam
True Negative (TN)	-	Ham predicted as ham	Legitimate message delivered
False Positive (FP)	-	Ham predicted as spam	Good email goes to spam (bad!)
False Negative (FN)	-	Spam predicted as ham	Spam reaches inbox (bad!)
Accuracy	(TP+TN)/(TP+TN+FP+FN)	% correct	Not always best metric
Precision	TP/(TP+FP)	Of predicted spam, % correct	Minimize false positives
Recall	TP/(TP+FN)	Of actual spam, % caught	Catch most spam

India Context: SMS Spam Problem

Indian phone users receive ~400 million spam SMS daily. Common patterns:

Common Spam Patterns in India

• Loan offers: "Loan upto 5 lakhs at 2% interest. Approve in 5 mins"
• OTP scams: "Your OTP is 123456. Don't share with anyone"
• Prize claims: "Congratulations! You won 1 lakh rupees. Claim now"
• Job offers: "Work from home, earn 50k/month. No investment needed"
• Crypto scams: "Bitcoin trading app - guaranteed 100% returns"

Build a Simple Hindi Spam Detector

# Hindi spam messages
hindi_messages = [ "क्लिक करें और जीतें 10 लाख रुपये",  # Click and win 10 lakh rupees "आपका OTP है 123456", # Your OTP is 123456 "घर बैठे 50,000 कमाएं", # Earn 50,000 sitting at home "कल मीटिंग है 2 बजे", # Meeting tomorrow at 2pm "धन्यवाद आपके संदेश के लिए", # Thanks for your message
]

labels = [1, 1, 1, 0, 0]  # spam, spam, spam, ham, ham

# TF-IDF will work with Hindi text (UTF-8 encoded)
tfidf = TfidfVectorizer()
X = tfidf.fit_transform(hindi_messages)

classifier = MultinomialNB()
classifier.fit(X, labels)

# Test on new message
new_msg = ["लाभ उठाएं खास ऑफर पर"]  # Take advantage of special offer
prediction = classifier.predict(tfidf.transform(new_msg))
print(f"Prediction: {'Spam' if prediction[0] else 'Ham'}")  # Spam

Advanced: Why Naive Bayes Works for Spam

• Fast: O(n) training, O(1) prediction
• Scalable: Can handle millions of messages
• Works with text: Naturally suited for word frequencies
• Probabilistic: Can output confidence scores
• Few parameters: Easy to tune

Why "Naive"?

Assumes all words are independent. In reality, they're not:

• "Free" and "money" often appear together
• "Meeting" and "tomorrow" often appear together

But despite this, Naive Bayes works surprisingly well in practice for text!

Improving Your Spam Filter

1. More data: Train on thousands of real spam/ham messages
2. Feature engineering: Add features like sender domain, message length, caps count
3. Ensemble methods: Combine multiple classifiers (voting)
4. Deep learning: Use neural networks for complex patterns
5. Online learning: Update model as new spam patterns emerge
6. Multilingual: Handle Hindi, Tamil, Telugu, etc.

Practice Problems

1. Build a spam filter using the provided code on your own dataset.
2. Compare Bag of Words vs TF-IDF features on spam classification.
3. Implement Naive Bayes from scratch (calculate probabilities manually).
4. Create confusion matrix and compute precision/recall for spam filter.
5. Build a Hindi SMS spam detector for Indian phone numbers.
6. Experiment with different hyperparameters (max_features, ngrams).

Complete Working Example

from sklearn.feature_extraction.text import TfidfVectorizer
from sklearn.naive_bayes import MultinomialNB
from sklearn.metrics import accuracy_score, precision_score, recall_score

# Real spam/ham dataset (simplified)
spam_messages = [ "Click now to win free money", "Congratulations you are selected", "Verify your account urgently", "Limited offer act now", "Claim free prize today",
]

ham_messages = [ "Can you call me back", "Meeting scheduled at 2pm", "Thanks for the update", "Let's discuss the project", "How are you doing today",
]

all_messages = spam_messages + ham_messages
labels = [1]*len(spam_messages) + [0]*len(ham_messages)

# Vectorize
vectorizer = TfidfVectorizer()
X = vectorizer.fit_transform(all_messages)

# Train classifier
classifier = MultinomialNB()
classifier.fit(X, labels)

# Test on new messages
test_messages = [ "Verify account now to win prize", "Can we meet tomorrow", "Claim free money instantly",
]

predictions = classifier.predict(vectorizer.transform(test_messages))
probabilities = classifier.predict_proba(vectorizer.transform(test_messages))

for msg, pred, prob in zip(test_messages, predictions, probabilities): print(f"Message: {msg}") print(f"Prediction: {'SPAM' if pred else 'HAM'}") print(f"Confidence: {max(prob):.2%}
")

Key Takeaways

• Spam filtering is a text classification problem with two classes: spam and ham.
• Preprocessing: lowercase, remove URLs, remove punctuation, clean whitespace.
• Bag of Words: count word frequencies (simple but ignores importance).
• TF-IDF: weight by importance (better - rare spam words get high weight).
• Naive Bayes: probabilistic classifier, fast, scalable, works great for text.
• Evaluation: Accuracy, Precision, Recall, F1-score, Confusion Matrix.
• In spam filters, precision is crucial (don't want legitimate emails in spam).
• India: Major problem with SMS spam (loan offers, OTP scams, job scams).
• Production: Continuous monitoring and retraining as spam patterns evolve.

  INPUT LAYER HIDDEN LAYERS OUTPUT LAYER (Raw Data) (Feature Extraction) (Decision) Pixel 1 ──┐ Pixel 2 ──┤ ┌─[Neuron]─┐ Pixel 3 ──┼───▶│ Edges & │───┐ Pixel 4 ──┤ │ Corners │ │ ┌─[Neuron]─┐ Pixel 5 ──┤ └───────────┘ ├───▶│ Face │──▶ "It's a cat!" (92%) ... │ ┌─[Neuron]─┐ │ │ Features │ "It's a dog" (7%) Pixel N ──┤ │ Shapes & │───┘ │ + Body │ "Other" (1%) └───▶│ Textures │───────▶│ Shape │ └───────────┘ └──────────┘ Layer 1: Detects simple features (edges, gradients) Layer 2: Combines into complex features (eyes, ears, whiskers) Layer 3: Makes the final decision based on all features

  BIG-O COMPARISON (n = 1,000,000 elements): O(1) Constant 1 operation Hash table lookup O(log n) Logarithmic  20 operations Binary search O(n) Linear 1,000,000 ops Linear search O(n log n)  Linearithmic 20,000,000 ops Merge sort, Quick sort O(n²) Quadratic 1,000,000,000,000 Bubble sort, Selection sort O(2ⁿ) Exponential  ∞ (universe dies) Brute force subset Time at 1 billion ops/sec: O(n log n): 0.02 seconds ← Perfectly usable O(n²): 11.5 DAYS ← Completely unusable! O(2ⁿ): Longer than the age of the universe # Python example: Merge Sort (O(n log n)) def merge_sort(arr): if len(arr) <= 1: return arr mid = len(arr) // 2 left = merge_sort(arr[:mid]) # Sort left half right = merge_sort(arr[mid:]) # Sort right half return merge(left, right) # Merge sorted halves def merge(left, right): result = [] i = j = 0 while i < len(left) and j < len(right): if left[i] <= right[j]: result.append(left[i]); i += 1 else: result.append(right[j]); j += 1 result.extend(left[i:]) result.extend(right[j:]) return result