LAB 01 Linear Regression (Gradient Descent) REAL IMPLEMENTATION

Real linear regression using gradient descent optimization. Learns weights by minimizing Mean Squared Error.

Gradient Descent MSE Loss
Size (sqft)10001500200025003000
Price ($K)150200250300350
How big each step is when adjusting weights. Too high = overshoots, too low = slow. Use 0.0000001 for these large numbers.
Number of times to loop through all training data. More = better fit (usually 500-2000).
Enter a house size (in sqft) to predict its price using the trained model.
Linear Regression: y = wΓ—x + b Click "Train Model" to learn optimal w and b values.
LAB 02 Logistic Regression (Binary Classification) REAL IMPLEMENTATION

Binary classifier using sigmoid activation. Outputs probability between 0-1 for fraud detection.

Sigmoid Cross-Entropy Loss
Step size for gradient descent. 0.1 works well for normalized features.
Training iterations. 300-500 usually enough for convergence.
Dollar value of transaction. Higher amounts (>$1000) are more suspicious.
When transaction occurred. Late night (1-5 AM) = higher fraud risk.
How far from cardholder's usual location. >100 miles = suspicious.
Logistic Regression: P(fraud) = Οƒ(wΒ·x + b) Train the model first, then classify transactions.
LAB 03 Decision Tree (ID3/CART) REAL IMPLEMENTATION

Builds a tree by finding the best feature splits using Information Gain or Gini Impurity.

Information Gain Gini Impurity
Method to measure split quality. Gini is faster; Entropy often gives better trees.
Maximum tree depth. Deeper = more complex but risk of overfitting. 3-5 is typical.
Applicant's yearly earnings. Higher income improves approval odds.
FICO credit score. Above 700 = "good", below 600 = "poor".
Monthly debt payments as % of income. Below 36% is ideal.
Decision Tree: Recursive splits based on feature thresholds. Build the tree first to see its structure.
LAB 04 Random Forest (Bagging Ensemble) REAL IMPLEMENTATION

Multiple decision trees trained on bootstrapped samples, with random feature subsets.

More trees = more stable predictions. 5-100 typical. Each votes independently.
Depth limit per tree. Shallower trees reduce overfitting in ensemble.
Random Forest: Majority vote from multiple trees.
LAB 05 Gradient Boosting REAL IMPLEMENTATION

Sequential ensemble where each tree corrects errors of previous trees.

Boosting rounds. Each round adds a new tree that fits the residual errors.
Scales each tree's contribution. Lower = slower learning but better generalization.
Gradient Boosting: F(x) = Fβ‚€ + Ξ£(lr Γ— tree_m(x))
LAB 06 Neural Network (Backpropagation) REAL IMPLEMENTATION

Multi-layer perceptron with full backpropagation. Learns the XOR function (not linearly separable).

Comma-separated neuron counts. "8,8" = two layers with 8 neurons each.
Tanh often converges faster than sigmoid for this problem.
Weight update magnitude. 0.5-2.0 works for XOR problem.
Training iterations. XOR typically needs 3000-10000 to converge.

Test XOR: [0,0]β†’0, [0,1]β†’1, [1,0]β†’1, [1,1]β†’0

Neural Network with Backpropagation Train to learn XOR (requires hidden layers!).
LAB 07 Support Vector Machine REAL IMPLEMENTATION

Finds optimal separating hyperplane using hinge loss and sub-gradient descent.

Gradient step size. 0.001-0.01 typical for SVM.
Regularization strength. Higher = wider margin, more misclassifications allowed.
Training passes. 500-2000 usually sufficient.
SVM: Maximize margin between classes. Cyan = class +1, Pink = class -1
LAB 08 K-Nearest Neighbors REAL IMPLEMENTATION

Classifies by finding K closest training examples and voting.

How many nearest points vote. Odd numbers avoid ties. 3-7 typical.
Euclidean = straight line. Manhattan = grid/taxi distance.
Uniform = equal votes. Distance = closer neighbors count more.
KNN: Classification by similarity voting.
LAB 09 K-Means Clustering REAL IMPLEMENTATION

Unsupervised clustering using K-Means++ initialization and Lloyd's algorithm.

Number of groups to find. Try different values to see effect.
Number of data points to cluster.
K-Means: Iteratively assigns points to nearest centroid.
LAB 10 Naive Bayes (Multinomial) REAL IMPLEMENTATION

Text classification using Bayes theorem with Laplace smoothing.

Laplace smoothing prevents zero probabilities for unseen words. Ξ±=1 is standard.
Spam words: free, money, winner, prize, click, urgent, cash, offer, gift
Naive Bayes: P(class|words) ∝ P(class) Γ— ∏P(word|class)
LAB 11 Convolutional Neural Network REAL IMPLEMENTATION

CNN with learnable filters, ReLU, max pooling, and full backpropagation through conv layers.

Click cells to draw, or use presets. CNN classifies as horizontal/vertical/diagonal.
Number of 3Γ—3 convolutional filters to learn.
Backprop step size. 0.01-0.05 works well.
Training iterations. 100-300 for good accuracy.
CNN: Input β†’ Conv β†’ ReLU β†’ MaxPool β†’ Dense β†’ Softmax Train to classify line patterns!
LAB 12 Recurrent Neural Network REAL IMPLEMENTATION

Character-level RNN with hidden state, trained using Backpropagation Through Time.

Text for RNN to learn. Repetitive patterns work best for this small model.
Memory capacity. 32-64 for small texts.
BPTT step size. 0.05-0.2 typical.
300-1000 for character learning.
Starting character (must be in corpus).
Characters to generate.
RNN: hβ‚œ = tanh(Wβ‚“β‚•Γ—xβ‚œ + Wβ‚•β‚•Γ—hβ‚œβ‚‹β‚ + b) Hidden state carries memory through sequence.
LAB 13 Transformer (Self-Attention) REAL IMPLEMENTATION

Core self-attention mechanism with Q, K, V matrices - the foundation of GPT/BERT.

Enter words (max 8). Attention shows how each word "looks at" others.
Size of word vectors. 8-64 for demo.
Self-Attention: softmax(QK^T / √d) Γ— V Brighter cells = higher attention weight.
LAB 14 Q-Learning (Reinforcement Learning) REAL IMPLEMENTATION

Agent learns optimal policy through trial and error using Bellman equation.

πŸ€– Agent β†’ 🎯 Goal. Avoid 🚫 obstacles. Rewards: Goal=+10, Obstacle=-5, Step=-0.1
How much new info overrides old. 0.1-0.3 typical.
Value of future rewards. 0.9 = future worth 90%.
Random action probability. High early, low later.
Q-Learning: Q(s,a) ← Q + Ξ±[r + Ξ³Γ—maxQ' - Q]
LAB 15 Bayesian Network REAL IMPLEMENTATION

Medical diagnosis using conditional probability tables and Bayes theorem.

P(Disease)=1%. Symptoms: P(Fever|D)=90%, P(Cough|D)=80%, P(Fatigue|D)=70%
Bayesian Inference: P(D|E) = P(E|D)Γ—P(D) / P(E) Select symptoms to update disease probability.
LAB 16 Hierarchical Clustering REAL IMPLEMENTATION

Agglomerative clustering that builds a dendrogram by merging closest clusters.

Semicolon-separated coordinate pairs to cluster.
Single=min distance, Complete=max, Average=mean.
Hierarchical Clustering: Merge closest clusters iteratively.