Introduction to Machine Learning

What is Machine Learning?

Machine Learning (ML) is a branch of Artificial Intelligence that enables computers to learn patterns from data and make decisions without being explicitly programmed for every scenario. Instead of writing rules by hand, you feed the algorithm data and let it discover the rules itself.

Arthur Samuel (1959) defined ML as the "field of study that gives computers the ability to learn without being explicitly programmed." Tom Mitchell provided a more formal definition:

Types of Machine Learning

1. Supervised Learning

The algorithm learns from labeled data — each training example comes with the correct answer (label). The model learns a mapping from inputs to outputs.

• Classification: Predicting a category — spam vs. not spam, cat vs. dog
• Regression: Predicting a continuous value — house prices, temperature

2. Unsupervised Learning

The algorithm works with unlabeled data and tries to find hidden patterns or groupings without knowing the correct answers.

• Clustering: Grouping similar customers together
• Dimensionality Reduction: Compressing data while keeping important features (PCA)
• Association: Finding items that frequently co-occur (market basket analysis)

3. Reinforcement Learning

An agent learns by interacting with an environment, receiving rewards for good actions and penalties for bad ones. Used in game AI, robotics, and self-driving cars.

Real-World Applications

Machine Learning powers many products and services you use daily:

• Healthcare: Disease diagnosis from X-rays, drug discovery, patient risk prediction
• Finance: Fraud detection, algorithmic trading, credit scoring
• E-commerce: Product recommendations (Amazon, Netflix), dynamic pricing
• Transportation: Self-driving cars (Tesla), route optimization (Google Maps)
• Language: Machine translation (Google Translate), voice assistants (Siri, Alexa)
• Social Media: Content recommendation, face recognition, sentiment analysis

The ML Workflow

Every ML project follows a standard pipeline:

1. Define the Problem: What question are you trying to answer?
2. Collect Data: Gather relevant, high-quality data
3. Explore & Preprocess: Clean data, handle missing values, visualize patterns
4. Feature Engineering: Select and transform the most informative features
5. Train the Model: Choose an algorithm and fit it to the training data
6. Evaluate: Test the model on unseen data using appropriate metrics
7. Tune & Optimize: Adjust hyperparameters for better performance
8. Deploy: Put the model into production and monitor its performance

Setting Up Your Environment

bash
# Install essential ML libraries
pip install numpy pandas matplotlib scikit-learn
pip install tensorflow keras
pip install seaborn jupyter

# Verify installation
python -c "import sklearn; print(sklearn.__version__)"

Python Essentials for Machine Learning

NumPy: Numerical Computing Foundation

NumPy is the backbone of scientific computing in Python. It provides high-performance multidimensional arrays and tools for working with them.

Python
import numpy as np

# Creating arrays
a = np.array([1, 2, 3, 4, 5])
matrix = np.array([[1, 2, 3],
                   [4, 5, 6],
                   [7, 8, 9]])

# Useful array generators
zeros = np.zeros((3, 4))         # 3x4 matrix of zeros
ones = np.ones((2, 3))           # 2x3 matrix of ones
rng = np.arange(0, 10, 2)       # [0, 2, 4, 6, 8]
lin = np.linspace(0, 1, 5)      # [0, 0.25, 0.5, 0.75, 1.0]
rand = np.random.randn(3, 3)    # 3x3 random normal values

# Array properties
print(matrix.shape)    # (3, 3)
print(matrix.dtype)    # int64
print(matrix.ndim)     # 2

Vectorized Operations

Python
# Element-wise operations (much faster than loops)
a = np.array([1, 2, 3])
b = np.array([4, 5, 6])

print(a + b)      # [5, 7, 9]
print(a * b)      # [4, 10, 18]
print(a ** 2)     # [1, 4, 9]
print(np.dot(a, b))  # 32  (dot product)

# Statistical operations
data = np.array([14, 23, 18, 29, 35, 22])
print(np.mean(data))    # 23.5
print(np.std(data))     # 6.99
print(np.median(data))  # 22.5

# Matrix operations
A = np.array([[1, 2], [3, 4]])
B = np.array([[5, 6], [7, 8]])
print(A @ B)               # Matrix multiplication
print(np.linalg.inv(A))    # Matrix inverse
print(np.linalg.det(A))    # Determinant: -2.0

Pandas: Data Analysis Powerhouse

Python
import pandas as pd

# Creating a DataFrame
df = pd.DataFrame({
    'Name': ['Alice', 'Bob', 'Charlie', 'Diana'],
    'Age': [25, 30, 35, 28],
    'Salary': [50000, 60000, 75000, 55000],
    'Department': ['ML', 'Web', 'ML', 'Data']
})

# Exploring data
print(df.head())          # First 5 rows
print(df.info())          # Column types, non-null counts
print(df.describe())      # Statistical summary

# Selecting & filtering
ml_team = df[df['Department'] == 'ML']
high_salary = df[df['Salary'] > 55000]

# Groupby operations
avg_salary = df.groupby('Department')['Salary'].mean()

# Handling missing data
df.dropna()               # Remove rows with NaN
df.fillna(0)              # Replace NaN with 0
df.fillna(df.mean())      # Replace NaN with column mean

# Reading from CSV
df = pd.read_csv('data.csv')
df.to_csv('output.csv', index=False)

Matplotlib: Data Visualization

Python
import matplotlib.pyplot as plt
import numpy as np

# Line plot
x = np.linspace(0, 10, 100)
plt.figure(figsize=(10, 6))
plt.plot(x, np.sin(x), label='sin(x)', linewidth=2)
plt.plot(x, np.cos(x), label='cos(x)', linewidth=2)
plt.xlabel('x')
plt.ylabel('y')
plt.title('Trigonometric Functions')
plt.legend()
plt.grid(True, alpha=0.3)
plt.show()

# Scatter plot
np.random.seed(42)
x = np.random.randn(100)
y = 2 * x + np.random.randn(100) * 0.5
plt.scatter(x, y, alpha=0.7, c='#4f46e5')
plt.xlabel('Feature')
plt.ylabel('Target')
plt.title('Linear Relationship with Noise')
plt.show()

# Histogram
data = np.random.normal(100, 15, 1000)
plt.hist(data, bins=30, edgecolor='white', color='#7c3aed')
plt.title('Normal Distribution')
plt.show()

import numpy as np
arr = np.random.randint(1, 101, size=20)
print(f"Array: {arr}")
print(f"Mean: {np.mean(arr):.2f}")
print(f"Max: {np.max(arr)}")
print(f"Min: {np.min(arr)}")
print(f"Std: {np.std(arr):.2f}")

import pandas as pd
df = pd.DataFrame({
    'Name': ['Amit', 'Priya', 'Ravi', 'Sneha', 'Karan'],
    'Maths': [85, 92, 78, 95, 88],
    'Science': [90, 88, 82, 91, 76],
    'English': [78, 95, 85, 89, 92]
})
df['Total'] = df[['Maths','Science','English']].sum(axis=1)
df['Percentage'] = (df['Total'] / 300 * 100).round(2)
print(df)

subjects = ['Maths', 'Science', 'English']
averages = [df[s].mean() for s in subjects]
plt.bar(subjects, averages, color=['#4f46e5','#10b981','#f59e0b'])
plt.ylabel('Average Score')
plt.title('Average Score by Subject')
plt.ylim(70, 100)
plt.show()

Data Preprocessing & Feature Engineering

Why Preprocessing Matters

Raw data is messy — it contains missing values, inconsistent formats, outliers, and mixed data types. Garbage in = garbage out. Preprocessing transforms raw data into a clean, ML-ready format. Studies show that data scientists spend 60-80% of their time on data preprocessing.

Handling Missing Data

Python
import pandas as pd
import numpy as np

df = pd.DataFrame({
    'Age': [25, np.nan, 35, 28, np.nan],
    'Salary': [50000, 60000, np.nan, 55000, 70000],
    'City': ['Delhi', 'Mumbai', 'Delhi', None, 'Bangalore']
})

# Check missing values
print(df.isnull().sum())

# Strategy 1: Drop rows with any NaN
df_dropped = df.dropna()

# Strategy 2: Fill with mean/median (numerical)
df['Age'].fillna(df['Age'].mean(), inplace=True)
df['Salary'].fillna(df['Salary'].median(), inplace=True)

# Strategy 3: Fill with mode (categorical)
df['City'].fillna(df['City'].mode()[0], inplace=True)

# Strategy 4: Scikit-learn SimpleImputer
from sklearn.impute import SimpleImputer
imputer = SimpleImputer(strategy='mean')
df[['Age','Salary']] = imputer.fit_transform(df[['Age','Salary']])

Encoding Categorical Variables

Python
from sklearn.preprocessing import LabelEncoder, OneHotEncoder

# Label Encoding (for ordinal data: low < medium < high)
le = LabelEncoder()
df['City_encoded'] = le.fit_transform(df['City'])
# Delhi=1, Mumbai=2, Bangalore=0

# One-Hot Encoding (for nominal data — no order)
df_encoded = pd.get_dummies(df, columns=['City'], drop_first=True)
# Creates: City_Delhi, City_Mumbai (Bangalore is baseline)

Feature Scaling

Many algorithms (KNN, SVM, Neural Networks, Gradient Descent) are sensitive to the scale of features. A salary feature (50000-100000) would dominate an age feature (20-60) without scaling.

Python
from sklearn.preprocessing import StandardScaler, MinMaxScaler

# StandardScaler: mean=0, std=1 (z-score normalization)
scaler = StandardScaler()
df[['Age_scaled','Salary_scaled']] = scaler.fit_transform(df[['Age','Salary']])

# MinMaxScaler: scales to [0, 1]
mm_scaler = MinMaxScaler()
df[['Age_mm','Salary_mm']] = mm_scaler.fit_transform(df[['Age','Salary']])

Train-Test Split

Python
from sklearn.model_selection import train_test_split

X = df[['Age', 'Salary']]  # Features
y = df['Target']            # Label

X_train, X_test, y_train, y_test = train_test_split(
    X, y, test_size=0.2, random_state=42
)
print(f"Train: {X_train.shape}, Test: {X_test.shape}")

Python
import pandas as pd
from sklearn.preprocessing import StandardScaler
from sklearn.model_selection import train_test_split

# Load Titanic dataset
df = pd.read_csv('https://raw.githubusercontent.com/datasciencedojo/datasets/master/titanic.csv')

# Drop irrelevant columns
df = df.drop(['Name', 'Ticket', 'Cabin', 'PassengerId'], axis=1)

# Fill missing values
df['Age'].fillna(df['Age'].median(), inplace=True)
df['Embarked'].fillna(df['Embarked'].mode()[0], inplace=True)

# Encode categorical variables
df = pd.get_dummies(df, columns=['Sex', 'Embarked'], drop_first=True)

# Split features and target
X = df.drop('Survived', axis=1)
y = df['Survived']

# Scale features
scaler = StandardScaler()
X_scaled = scaler.fit_transform(X)

# Split into train and test
X_train, X_test, y_train, y_test = train_test_split(
    X_scaled, y, test_size=0.2, random_state=42)

print(f"Training set: {X_train.shape}")
print(f"Test set: {X_test.shape}")
print("Data is clean and ready for modeling!")

from sklearn.preprocessing import LabelEncoder
colors = ['Red', 'Green', 'Blue', 'Red', 'Blue']
le = LabelEncoder()
label_encoded = le.fit_transform(colors)
print("Label:", label_encoded)  # [2, 1, 0, 2, 0]

df = pd.DataFrame({'Color': colors})
one_hot = pd.get_dummies(df, columns=['Color'])
print("One-Hot:\n", one_hot)

Linear Regression

The Idea

Linear regression fits a straight line (or hyperplane) through data to predict a continuous target variable. It finds the best relationship between input features (X) and output (y).

Where w (weights) and b (bias) are learned from data. The goal is to find values of w and b that minimize the error between predicted ŷ and actual y.

Cost Function: Mean Squared Error

Gradient Descent

Gradient descent is an optimization algorithm that iteratively adjusts w and b to minimize the cost function by moving in the direction of the steepest descent.

Implementation from Scratch

Python
import numpy as np

class LinearRegressionScratch:
    def __init__(self, learning_rate=0.01, n_iterations=1000):
        self.lr = learning_rate
        self.n_iter = n_iterations
        self.weights = None
        self.bias = None
        self.cost_history = []

    def fit(self, X, y):
        n_samples, n_features = X.shape
        self.weights = np.zeros(n_features)
        self.bias = 0

        for _ in range(self.n_iter):
            y_pred = np.dot(X, self.weights) + self.bias

            # Compute gradients
            dw = (1 / n_samples) * np.dot(X.T, (y_pred - y))
            db = (1 / n_samples) * np.sum(y_pred - y)

            # Update parameters
            self.weights -= self.lr * dw
            self.bias -= self.lr * db

            # Track cost
            cost = np.mean((y_pred - y) ** 2)
            self.cost_history.append(cost)

    def predict(self, X):
        return np.dot(X, self.weights) + self.bias

# Usage
np.random.seed(42)
X = 2 * np.random.rand(100, 1)
y = 4 + 3 * X.flatten() + np.random.randn(100)

model = LinearRegressionScratch(learning_rate=0.1, n_iterations=1000)
model.fit(X, y)
print(f"Weight: {model.weights[0]:.3f}, Bias: {model.bias:.3f}")
# Should be close to Weight: 3.0, Bias: 4.0

Using Scikit-Learn

Python
from sklearn.linear_model import LinearRegression
from sklearn.metrics import mean_squared_error, r2_score

model = LinearRegression()
model.fit(X_train, y_train)

y_pred = model.predict(X_test)
print(f"Coefficients: {model.coef_}")
print(f"Intercept: {model.intercept_:.3f}")
print(f"MSE: {mean_squared_error(y_test, y_pred):.3f}")
print(f"R² Score: {r2_score(y_test, y_pred):.3f}")

Python
from sklearn.datasets import fetch_california_housing
from sklearn.linear_model import LinearRegression
from sklearn.model_selection import train_test_split
from sklearn.preprocessing import StandardScaler
from sklearn.metrics import mean_squared_error, r2_score
import numpy as np

# Load California housing dataset
housing = fetch_california_housing()
X, y = housing.data, housing.target

# Split and scale
X_train, X_test, y_train, y_test = train_test_split(X, y, test_size=0.2, random_state=42)
scaler = StandardScaler()
X_train_s = scaler.fit_transform(X_train)
X_test_s = scaler.transform(X_test)

# Train model
model = LinearRegression()
model.fit(X_train_s, y_train)

# Evaluate
y_pred = model.predict(X_test_s)
print(f"RMSE: ${np.sqrt(mean_squared_error(y_test, y_pred)) * 100000:.0f}")
print(f"R² Score: {r2_score(y_test, y_pred):.3f}")

# Feature importance
for name, coef in zip(housing.feature_names, model.coef_):
    print(f"  {name}: {coef:.4f}")

np.random.seed(0)
X = np.random.rand(200, 1) * 10
y = 5 * X.flatten() + 3 + np.random.randn(200) * 2

model = LinearRegressionScratch(lr=0.01, n_iterations=2000)
model.fit(X, y)
print(f"Learned: y = {model.weights[0]:.2f}x + {model.bias:.2f}")
# Expected: y ≈ 5.00x + 3.00

Classification: Logistic Regression

From Regression to Classification

Logistic regression predicts probabilities for classification. It wraps linear regression output through the sigmoid function to produce values between 0 and 1.

If σ(z) ≥ 0.5, predict class 1. Otherwise, predict class 0. The 0.5 threshold defines the decision boundary.

Binary Cross-Entropy Loss

Python
import numpy as np

def sigmoid(z):
    return 1 / (1 + np.exp(-z))

# Sigmoid outputs for different inputs
z_values = np.array([-5, -2, 0, 2, 5])
print(sigmoid(z_values))
# [0.007, 0.119, 0.500, 0.881, 0.993]

Scikit-Learn Implementation

Python
from sklearn.linear_model import LogisticRegression
from sklearn.metrics import accuracy_score, classification_report, confusion_matrix

model = LogisticRegression(max_iter=1000)
model.fit(X_train, y_train)

y_pred = model.predict(X_test)
y_proba = model.predict_proba(X_test)[:, 1]  # Probability of class 1

print(f"Accuracy: {accuracy_score(y_test, y_pred):.3f}")
print(classification_report(y_test, y_pred))
print("Confusion Matrix:\n", confusion_matrix(y_test, y_pred))

Python
from sklearn.feature_extraction.text import TfidfVectorizer
from sklearn.linear_model import LogisticRegression
from sklearn.model_selection import train_test_split
from sklearn.metrics import accuracy_score, classification_report

# Sample email data
emails = [
    "Win a free iPhone now! Click here!",
    "Meeting scheduled for tomorrow at 3pm",
    "Congratulations! You won $1000000",
    "Please review the attached report",
    "FREE OFFER! Buy one get one free!!!",
    "Can we reschedule the project discussion?",
    "URGENT: Your account will be suspended",
    "Hey, are you coming to lunch today?",
    "Claim your prize before it expires!!!",
    "The quarterly report is ready for review",
]
labels = [1,0,1,0,1,0,1,0,1,0]  # 1=spam, 0=ham

# Convert text to numerical features using TF-IDF
vectorizer = TfidfVectorizer(stop_words='english')
X = vectorizer.fit_transform(emails)
y = np.array(labels)

# Train classifier
model = LogisticRegression()
model.fit(X, y)

# Test on new emails
new_emails = [
    "You are selected for a cash prize!",
    "Let's finalize the budget this week"
]
X_new = vectorizer.transform(new_emails)
predictions = model.predict(X_new)

for email, pred in zip(new_emails, predictions):
    label = "SPAM" if pred == 1 else "HAM"
    print(f"[{label}] {email}")

Decision Trees & Ensemble Methods

Decision Trees

A decision tree splits data recursively based on feature thresholds, creating a tree structure of if-else decisions. It chooses splits that maximize information gain (or minimize Gini impurity).

Python
from sklearn.tree import DecisionTreeClassifier, export_text
from sklearn.datasets import load_iris
from sklearn.model_selection import train_test_split

iris = load_iris()
X_train, X_test, y_train, y_test = train_test_split(
    iris.data, iris.target, test_size=0.2, random_state=42)

tree = DecisionTreeClassifier(max_depth=3, random_state=42)
tree.fit(X_train, y_train)

print(f"Accuracy: {tree.score(X_test, y_test):.3f}")
print(export_text(tree, feature_names=iris.feature_names))

Random Forest (Bagging)

Random Forest trains multiple decision trees on random subsets of data and features, then combines their predictions (majority voting for classification, averaging for regression). This reduces overfitting.

Python
from sklearn.ensemble import RandomForestClassifier

rf = RandomForestClassifier(n_estimators=100, max_depth=5, random_state=42)
rf.fit(X_train, y_train)
print(f"Random Forest Accuracy: {rf.score(X_test, y_test):.3f}")

# Feature importance
for name, imp in zip(iris.feature_names, rf.feature_importances_):
    print(f"  {name}: {imp:.4f}")

Gradient Boosting & XGBoost

Gradient Boosting builds trees sequentially — each tree corrects errors of the previous one. XGBoost is an optimized, high-performance implementation.

Python
from sklearn.ensemble import GradientBoostingClassifier

gb = GradientBoostingClassifier(n_estimators=100, learning_rate=0.1, max_depth=3)
gb.fit(X_train, y_train)
print(f"Gradient Boosting Accuracy: {gb.score(X_test, y_test):.3f}")

Python
import pandas as pd
import numpy as np
from sklearn.ensemble import RandomForestClassifier
from sklearn.model_selection import train_test_split
from sklearn.metrics import classification_report

# Simulated telecom churn dataset
np.random.seed(42)
n = 1000
data = pd.DataFrame({
    'tenure': np.random.randint(1, 72, n),
    'monthly_charge': np.random.uniform(20, 120, n).round(2),
    'total_charges': np.random.uniform(100, 8000, n).round(2),
    'contract_type': np.random.choice([0,1,2], n),  # month/year/2year
    'support_tickets': np.random.randint(0, 10, n),
})
# Churn likelihood: short tenure + high charges + many tickets
churn_prob = (1 / (1 + np.exp(-(
    -0.05 * data['tenure'] +
    0.02 * data['monthly_charge'] +
    0.3 * data['support_tickets'] +
    -1.0 * data['contract_type'] - 1
))))
data['churned'] = (np.random.random(n) < churn_prob).astype(int)

X = data.drop('churned', axis=1)
y = data['churned']
X_train, X_test, y_train, y_test = train_test_split(X, y, test_size=0.2, random_state=42)

rf = RandomForestClassifier(n_estimators=200, max_depth=8, random_state=42)
rf.fit(X_train, y_train)

print(f"Accuracy: {rf.score(X_test, y_test):.3f}")
print(classification_report(y_test, rf.predict(X_test)))
print("Top churn predictors:")
for name, imp in sorted(zip(X.columns, rf.feature_importances_), key=lambda x: -x[1]):
    print(f"  {name}: {imp:.4f}")

Support Vector Machines

The Intuition

SVM finds the optimal hyperplane that separates classes with the maximum margin — the largest gap between the closest data points (support vectors) of each class. Wider margin = better generalization.

Kernel Trick

When data isn't linearly separable, the kernel trick maps data to a higher dimension where it becomes separable, without actually computing the transformation.

Python
from sklearn.svm import SVC
from sklearn.datasets import make_moons
from sklearn.model_selection import train_test_split
from sklearn.preprocessing import StandardScaler

# Non-linear data
X, y = make_moons(n_samples=500, noise=0.2, random_state=42)
X_train, X_test, y_train, y_test = train_test_split(X, y, test_size=0.2)

scaler = StandardScaler()
X_train_s = scaler.fit_transform(X_train)
X_test_s = scaler.transform(X_test)

# Compare kernels
for kernel in ['linear', 'rbf', 'poly']:
    svm = SVC(kernel=kernel, C=1.0, gamma='scale')
    svm.fit(X_train_s, y_train)
    acc = svm.score(X_test_s, y_test)
    print(f"{kernel:>8} kernel → Accuracy: {acc:.3f}")

Python
from sklearn.datasets import load_digits
from sklearn.svm import SVC
from sklearn.model_selection import train_test_split
from sklearn.metrics import classification_report, accuracy_score
from sklearn.preprocessing import StandardScaler

digits = load_digits()
X_train, X_test, y_train, y_test = train_test_split(
    digits.data, digits.target, test_size=0.2, random_state=42)

scaler = StandardScaler()
X_train_s = scaler.fit_transform(X_train)
X_test_s = scaler.transform(X_test)

svm = SVC(kernel='rbf', C=10, gamma=0.001)
svm.fit(X_train_s, y_train)

y_pred = svm.predict(X_test_s)
print(f"Accuracy: {accuracy_score(y_test, y_pred):.3f}")
print(classification_report(y_test, y_pred))

Unsupervised Learning

K-Means Clustering

K-Means groups data into k clusters by iteratively: (1) assigning each point to the nearest centroid, (2) recalculating centroids as the mean of assigned points.

Python
from sklearn.cluster import KMeans
import numpy as np

# Generate sample data
from sklearn.datasets import make_blobs
X, y_true = make_blobs(n_samples=300, centers=4, cluster_std=0.8, random_state=42)

# Fit K-Means
kmeans = KMeans(n_clusters=4, random_state=42, n_init=10)
labels = kmeans.fit_predict(X)

print(f"Cluster centers:\n{kmeans.cluster_centers_}")
print(f"Inertia: {kmeans.inertia_:.2f}")

Choosing k: Elbow Method

Python
inertias = []
K_range = range(1, 11)
for k in K_range:
    km = KMeans(n_clusters=k, random_state=42, n_init=10)
    km.fit(X)
    inertias.append(km.inertia_)

# Plot the elbow curve
import matplotlib.pyplot as plt
plt.plot(K_range, inertias, 'bo-')
plt.xlabel('Number of clusters (k)')
plt.ylabel('Inertia')
plt.title('Elbow Method')
plt.show()
# Look for the "elbow" where the curve bends

PCA: Dimensionality Reduction

Principal Component Analysis projects high-dimensional data onto fewer dimensions while preserving maximum variance.

Python
from sklearn.decomposition import PCA
from sklearn.datasets import load_iris

iris = load_iris()
X = iris.data  # 4 features

# Reduce to 2 dimensions for visualization
pca = PCA(n_components=2)
X_2d = pca.fit_transform(X)

print(f"Original shape: {X.shape}")      # (150, 4)
print(f"Reduced shape: {X_2d.shape}")     # (150, 2)
print(f"Variance retained: {sum(pca.explained_variance_ratio_):.2%}")

Python
import numpy as np
import pandas as pd
from sklearn.cluster import KMeans
from sklearn.preprocessing import StandardScaler

# Simulated customer data
np.random.seed(42)
customers = pd.DataFrame({
    'annual_income': np.concatenate([
        np.random.normal(30000, 5000, 100),
        np.random.normal(70000, 10000, 100),
        np.random.normal(120000, 15000, 100)
    ]),
    'spending_score': np.concatenate([
        np.random.normal(20, 10, 100),
        np.random.normal(60, 15, 100),
        np.random.normal(80, 8, 100)
    ])
})

# Scale features
scaler = StandardScaler()
X_scaled = scaler.fit_transform(customers)

# Find optimal k and cluster
kmeans = KMeans(n_clusters=3, random_state=42, n_init=10)
customers['segment'] = kmeans.fit_predict(X_scaled)

# Analyze segments
print(customers.groupby('segment').mean().round(0))
# Segment 0: Budget-conscious (low income, low spending)
# Segment 1: Middle market (medium income, medium spending)
# Segment 2: Premium (high income, high spending)

Neural Networks Fundamentals

The Perceptron

A perceptron is the simplest neural network — a single neuron that takes inputs, applies weights and bias, then passes through an activation function.

Activation Functions

Python
import numpy as np

def relu(x):
    return np.maximum(0, x)

def relu_derivative(x):
    return (x > 0).astype(float)

def sigmoid(x):
    return 1 / (1 + np.exp(-np.clip(x, -500, 500)))

def sigmoid_derivative(x):
    s = sigmoid(x)
    return s * (1 - s)

Building a Neural Network from Scratch

Python
import numpy as np

class NeuralNetwork:
    def __init__(self, layer_sizes):
        """layer_sizes: e.g., [2, 4, 1] = 2 inputs, 4 hidden, 1 output"""
        self.weights = []
        self.biases = []
        for i in range(len(layer_sizes) - 1):
            w = np.random.randn(layer_sizes[i], layer_sizes[i+1]) * 0.5
            b = np.zeros((1, layer_sizes[i+1]))
            self.weights.append(w)
            self.biases.append(b)

    def forward(self, X):
        self.activations = [X]
        self.z_values = []
        current = X
        for i in range(len(self.weights)):
            z = current @ self.weights[i] + self.biases[i]
            self.z_values.append(z)
            if i == len(self.weights) - 1:
                current = sigmoid(z)  # Output layer
            else:
                current = relu(z)     # Hidden layers
            self.activations.append(current)
        return current

    def backward(self, X, y, lr=0.01):
        m = X.shape[0]
        output = self.activations[-1]
        delta = output - y.reshape(-1, 1)

        for i in range(len(self.weights) - 1, -1, -1):
            dw = self.activations[i].T @ delta / m
            db = np.sum(delta, axis=0, keepdims=True) / m
            if i > 0:
                delta = (delta @ self.weights[i].T) * relu_derivative(self.z_values[i-1])
            self.weights[i] -= lr * dw
            self.biases[i] -= lr * db

    def train(self, X, y, epochs=1000, lr=0.1):
        for epoch in range(epochs):
            output = self.forward(X)
            self.backward(X, y, lr)
            if epoch % 200 == 0:
                loss = np.mean((output - y.reshape(-1,1))**2)
                print(f"Epoch {epoch}, Loss: {loss:.4f}")

# XOR Problem — not linearly separable!
X = np.array([[0,0],[0,1],[1,0],[1,1]])
y = np.array([0, 1, 1, 0])

nn = NeuralNetwork([2, 8, 1])
nn.train(X, y, epochs=2000, lr=0.5)

predictions = nn.forward(X)
print("\nPredictions:")
for inp, pred in zip(X, predictions):
    print(f"  {inp} → {pred[0]:.4f} (expected {int(inp[0]) ^ int(inp[1])})")

nn = NeuralNetwork([2, 8, 4, 1])
nn.train(X, y, epochs=3000, lr=0.5)
predictions = nn.forward(X)
for inp, pred in zip(X, predictions):
    print(f"  {inp} → {pred[0]:.4f}")

Deep Learning with TensorFlow & Keras

Keras: High-Level Deep Learning API

Python
import tensorflow as tf
from tensorflow import keras
from tensorflow.keras import layers

# Simple sequential model
model = keras.Sequential([
    layers.Dense(128, activation='relu', input_shape=(784,)),
    layers.Dropout(0.3),
    layers.Dense(64, activation='relu'),
    layers.Dropout(0.2),
    layers.Dense(10, activation='softmax')
])

model.compile(
    optimizer='adam',
    loss='sparse_categorical_crossentropy',
    metrics=['accuracy']
)

model.summary()

Convolutional Neural Networks (CNNs)

CNNs use convolutional filters to automatically detect visual features — edges, textures, shapes — making them ideal for image recognition.

Python
cnn_model = keras.Sequential([
    # Convolutional layers — feature extraction
    layers.Conv2D(32, (3,3), activation='relu', input_shape=(28,28,1)),
    layers.MaxPooling2D((2,2)),
    layers.Conv2D(64, (3,3), activation='relu'),
    layers.MaxPooling2D((2,2)),
    layers.Conv2D(64, (3,3), activation='relu'),

    # Dense layers — classification
    layers.Flatten(),
    layers.Dense(64, activation='relu'),
    layers.Dropout(0.5),
    layers.Dense(10, activation='softmax')
])

Recurrent Neural Networks (RNNs & LSTMs)

RNNs process sequential data (text, time series) by maintaining a hidden state. LSTMs solve the vanishing gradient problem with gates that control information flow.

Python
# LSTM for sequence classification
lstm_model = keras.Sequential([
    layers.Embedding(input_dim=10000, output_dim=128),
    layers.LSTM(64, return_sequences=True),
    layers.LSTM(32),
    layers.Dense(1, activation='sigmoid')
])
lstm_model.compile(optimizer='adam', loss='binary_crossentropy', metrics=['accuracy'])

Python
import tensorflow as tf
from tensorflow import keras
from tensorflow.keras import layers

# Load MNIST
(X_train, y_train), (X_test, y_test) = keras.datasets.mnist.load_data()

# Preprocess
X_train = X_train.reshape(-1, 28, 28, 1).astype('float32') / 255.0
X_test = X_test.reshape(-1, 28, 28, 1).astype('float32') / 255.0

# Build CNN
model = keras.Sequential([
    layers.Conv2D(32, (3,3), activation='relu', input_shape=(28,28,1)),
    layers.MaxPooling2D((2,2)),
    layers.Conv2D(64, (3,3), activation='relu'),
    layers.MaxPooling2D((2,2)),
    layers.Flatten(),
    layers.Dense(128, activation='relu'),
    layers.Dropout(0.5),
    layers.Dense(10, activation='softmax')
])

model.compile(optimizer='adam',
              loss='sparse_categorical_crossentropy',
              metrics=['accuracy'])

# Train
history = model.fit(X_train, y_train, epochs=5,
                    batch_size=64,
                    validation_split=0.1)

# Evaluate
test_loss, test_acc = model.evaluate(X_test, y_test)
print(f"\nTest Accuracy: {test_acc:.4f}")
# Expected: ~99% accuracy

Natural Language Processing

Text Preprocessing Pipeline

Python
import re
from sklearn.feature_extraction.text import CountVectorizer, TfidfVectorizer

def preprocess_text(text):
    # 1. Lowercase
    text = text.lower()
    # 2. Remove special characters and numbers
    text = re.sub(r'[^a-zA-Z\s]', '', text)
    # 3. Remove extra whitespace
    text = re.sub(r'\s+', ' ', text).strip()
    return text

# Example
raw = "This movie was AMAZING!!! I loved it 100%. Best film of 2024."
clean = preprocess_text(raw)
print(clean)  # "this movie was amazing i loved it best film of"

Bag of Words & TF-IDF

Python
corpus = [
    "machine learning is amazing",
    "deep learning is a subset of machine learning",
    "natural language processing uses machine learning"
]

# Bag of Words — simple word counts
bow = CountVectorizer()
X_bow = bow.fit_transform(corpus)
print("Vocabulary:", bow.get_feature_names_out())
print("BoW Matrix:\n", X_bow.toarray())

# TF-IDF — weights by importance
tfidf = TfidfVectorizer()
X_tfidf = tfidf.fit_transform(corpus)
print("\nTF-IDF Matrix:\n", X_tfidf.toarray().round(3))

Word Embeddings

Word embeddings represent words as dense vectors where similar words have similar vectors. King - Man + Woman ≈ Queen.

Python
# Using pre-trained embeddings with Keras
from tensorflow.keras.layers import Embedding

# Embedding layer: vocab_size → embedding_dim
# Learns 128-dim vector for each of 10000 words
embedding = Embedding(input_dim=10000, output_dim=128, input_length=200)

Python
from sklearn.feature_extraction.text import TfidfVectorizer
from sklearn.linear_model import LogisticRegression
from sklearn.model_selection import train_test_split
from sklearn.metrics import accuracy_score, classification_report

# Sample movie reviews
reviews = [
    "This movie was fantastic! Great acting and storyline.",
    "Terrible film. Waste of time and money.",
    "Loved every moment. A masterpiece of cinema.",
    "Boring and predictable. Would not recommend.",
    "An incredible journey with amazing performances.",
    "The worst movie I have ever seen. Awful.",
    "Brilliant storytelling and breathtaking visuals.",
    "Dull plot with terrible dialogue throughout.",
    "A beautiful and moving cinematic experience.",
    "Completely disappointing. Save your money.",
    "Outstanding performances by the entire cast.",
    "Unwatchable garbage from start to finish.",
    "One of the best films of the decade.",
    "Painfully slow and utterly forgettable.",
    "A thrilling and emotionally powerful movie.",
    "So bad I walked out of the theater.",
]
# 1=positive, 0=negative
sentiments = [1,0,1,0,1,0,1,0,1,0,1,0,1,0,1,0]

# TF-IDF features
tfidf = TfidfVectorizer(max_features=5000, stop_words='english')
X = tfidf.fit_transform(reviews)
y = np.array(sentiments)

# Train model
X_train, X_test, y_train, y_test = train_test_split(X, y, test_size=0.25, random_state=42)
model = LogisticRegression(max_iter=1000)
model.fit(X_train, y_train)

print(f"Accuracy: {model.score(X_test, y_test):.2f}")

# Test on new reviews
new_reviews = [
    "This is an absolutely wonderful and amazing movie!",
    "I hated this film, it was terrible and boring.",
    "A decent movie with some good moments."
]
X_new = tfidf.transform(new_reviews)
predictions = model.predict(X_new)

for review, pred in zip(new_reviews, predictions):
    sentiment = "POSITIVE ✓" if pred == 1 else "NEGATIVE ✗"
    print(f"[{sentiment}] {review}")

Model Evaluation, Tuning & Deployment

Cross-Validation

Instead of a single train-test split, k-fold cross-validation splits data into k parts, trains on k-1, tests on 1, and rotates. This gives a more reliable performance estimate.

Python
from sklearn.model_selection import cross_val_score
from sklearn.ensemble import RandomForestClassifier
from sklearn.datasets import load_iris

iris = load_iris()
model = RandomForestClassifier(n_estimators=100, random_state=42)

# 5-fold cross-validation
scores = cross_val_score(model, iris.data, iris.target, cv=5, scoring='accuracy')
print(f"CV Scores: {scores}")
print(f"Mean: {scores.mean():.3f} ± {scores.std():.3f}")

Hyperparameter Tuning

Python
from sklearn.model_selection import GridSearchCV, RandomizedSearchCV

# Grid Search — exhaustive search
param_grid = {
    'n_estimators': [50, 100, 200],
    'max_depth': [3, 5, 10, None],
    'min_samples_split': [2, 5, 10]
}

grid_search = GridSearchCV(
    RandomForestClassifier(random_state=42),
    param_grid, cv=5, scoring='accuracy', n_jobs=-1
)
grid_search.fit(iris.data, iris.target)

print(f"Best params: {grid_search.best_params_}")
print(f"Best score: {grid_search.best_score_:.3f}")
best_model = grid_search.best_estimator_

Comprehensive Evaluation Metrics

Python
from sklearn.metrics import (confusion_matrix, classification_report,
                             accuracy_score, precision_score, recall_score,
                             f1_score, roc_auc_score)

y_pred = best_model.predict(X_test)

print("Confusion Matrix:")
print(confusion_matrix(y_test, y_pred))
#              Predicted
#            Neg    Pos
# Actual Neg [TN]   [FP]
# Actual Pos [FN]   [TP]

print(f"\nAccuracy:  {accuracy_score(y_test, y_pred):.3f}")
print(f"Precision: {precision_score(y_test, y_pred, average='weighted'):.3f}")
print(f"Recall:    {recall_score(y_test, y_pred, average='weighted'):.3f}")
print(f"F1 Score:  {f1_score(y_test, y_pred, average='weighted'):.3f}")

Overfitting vs. Underfitting

Python
# save_model.py — Train and save model
import pickle
from sklearn.datasets import load_iris
from sklearn.ensemble import RandomForestClassifier

iris = load_iris()
model = RandomForestClassifier(n_estimators=100, random_state=42)
model.fit(iris.data, iris.target)

# Save model to file
with open('model.pkl', 'wb') as f:
    pickle.dump(model, f)

print("Model saved!")

Python
# app.py — Flask API
from flask import Flask, request, jsonify
import pickle
import numpy as np

app = Flask(__name__)

# Load trained model
with open('model.pkl', 'rb') as f:
    model = pickle.load(f)

species = ['setosa', 'versicolor', 'virginica']

@app.route('/predict', methods=['POST'])
def predict():
    data = request.json
    features = np.array(data['features']).reshape(1, -1)
    prediction = model.predict(features)[0]
    probability = model.predict_proba(features)[0].tolist()

    return jsonify({
        'prediction': species[prediction],
        'confidence': max(probability),
        'probabilities': dict(zip(species, probability))
    })

@app.route('/health')
def health():
    return jsonify({'status': 'healthy'})

if __name__ == '__main__':
    app.run(debug=True, port=5000)

bash
# Test the API
curl -X POST http://localhost:5000/predict \
  -H "Content-Type: application/json" \
  -d '{"features": [5.1, 3.5, 1.4, 0.2]}'

# Response:
# {"prediction": "setosa", "confidence": 1.0, "probabilities": {...}}