Deep Learning Models: A collection of various deep learning architectures, models, & their implementations (PyTorch or TensorFlow) in Jupyter Notebooks.
- Author: Chizoba Obasi
- Github Repo: https://github.com/chizkidd/deep-learning-models
In [1]:
!pip install watermark
Collecting watermark Downloading watermark-2.6.0-py3-none-any.whl.metadata (15 kB) Requirement already satisfied: ipython>=6.0 in /usr/local/lib/python3.12/dist-packages (from watermark) (7.34.0) Requirement already satisfied: importlib-metadata>=1.4 in /usr/local/lib/python3.12/dist-packages (from watermark) (8.7.0) Requirement already satisfied: setuptools in /usr/local/lib/python3.12/dist-packages (from watermark) (75.2.0) Requirement already satisfied: zipp>=3.20 in /usr/local/lib/python3.12/dist-packages (from importlib-metadata>=1.4->watermark) (3.23.0) Requirement already satisfied: jedi>=0.16 in /usr/local/lib/python3.12/dist-packages (from ipython>=6.0->watermark) (0.19.2) Requirement already satisfied: decorator in /usr/local/lib/python3.12/dist-packages (from ipython>=6.0->watermark) (4.4.2) Requirement already satisfied: pickleshare in /usr/local/lib/python3.12/dist-packages (from ipython>=6.0->watermark) (0.7.5) Requirement already satisfied: traitlets>=4.2 in /usr/local/lib/python3.12/dist-packages (from ipython>=6.0->watermark) (5.7.1) Requirement already satisfied: prompt-toolkit!=3.0.0,!=3.0.1,<3.1.0,>=2.0.0 in /usr/local/lib/python3.12/dist-packages (from ipython>=6.0->watermark) (3.0.52) Requirement already satisfied: pygments in /usr/local/lib/python3.12/dist-packages (from ipython>=6.0->watermark) (2.19.2) Requirement already satisfied: backcall in /usr/local/lib/python3.12/dist-packages (from ipython>=6.0->watermark) (0.2.0) Requirement already satisfied: matplotlib-inline in /usr/local/lib/python3.12/dist-packages (from ipython>=6.0->watermark) (0.1.7) Requirement already satisfied: pexpect>4.3 in /usr/local/lib/python3.12/dist-packages (from ipython>=6.0->watermark) (4.9.0) Requirement already satisfied: parso<0.9.0,>=0.8.4 in /usr/local/lib/python3.12/dist-packages (from jedi>=0.16->ipython>=6.0->watermark) (0.8.5) Requirement already satisfied: ptyprocess>=0.5 in /usr/local/lib/python3.12/dist-packages (from pexpect>4.3->ipython>=6.0->watermark) (0.7.0) Requirement already satisfied: wcwidth in /usr/local/lib/python3.12/dist-packages (from prompt-toolkit!=3.0.0,!=3.0.1,<3.1.0,>=2.0.0->ipython>=6.0->watermark) (0.2.14) Downloading watermark-2.6.0-py3-none-any.whl (13 kB) Installing collected packages: watermark Successfully installed watermark-2.6.0
In [2]:
!pip install torchinfo
Requirement already satisfied: torchinfo in /usr/local/lib/python3.12/dist-packages (1.8.0)
In [3]:
%load_ext watermark
# %reload_ext watermark
%watermark -a 'Chizoba Obasi' -v -p torch
Author: Chizoba Obasi Python implementation: CPython Python version : 3.12.12 IPython version : 7.34.0 torch: 2.8.0+cu126
MobileNet Classifier on CIFAR-10¶
This notebook implements MobileNet [1] and applies it to the CIFAR-10 dataset. Now, instead of pushing for the largest network with biggest accuracy, we can consider tiny networks and accuracy/complexity tradeoff, hence the introduction of MobileNet [2]. MobileNets utilise depthwise separable convolutions, which significantly reduce the number of parameters and computation (mult-adds) compared to standard convolutions. This makes them ideal for mobile and embedded vision applications where computational resources are limited.
Network Architecture¶
The Core Innovation: Depthwise Separable Convolutions¶
In a standard convolution, the filter tries to learn both spatial (height/width) and cross-channel (depth) features simultaneously. Depthwise separable convolution breaks this down into two distinct steps:
- Depthwise (DW) Convolution:
- A spatial convolution is performed independently on each input channel.
CONV(3x3 DW)->BatchNORM->ReLU
- Pointwise (PW) Convolution:
- A convolution is then applied to project the channels into a new space.
CONV(1x1)->BatchNORM->ReLU
Standard Convolution ($L$) vs Depthwise Separable Convolutions ($R$)¶
Standard Convolution¶
| Single Feature | Multiple Features (128) |
|---|---|
 |
 |
Depthwise Separable Convolution¶
| Depthwise | Pointwise |
|---|---|
 |
 |
Depthwise + Pointwise
REFERENCES:
- Howard, A. G., et al. "MobileNets: Efficient Convolutional Neural Networks for Mobile Vision Applications" Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, 2017, pp. 1751-1760.
- Johnson, J. (2022, February 2). Lecture 11: CNN Architectures, Part 2 [PowerPoint Slides]. University of Michigan EECS 498/598: Deep Learning for Computer Vision. pp 44-53. https://web.eecs.umich.edu/~justincj/slides/eecs498/WI2022/598_WI2022_lecture11.pdf
In [4]:
import torch
import torchvision
import torch.nn as nn
import torch.nn.functional as F
import torch.optim as optim
from torch.utils.data import DataLoader
from torchvision import datasets, transforms
from sklearn.model_selection import train_test_split
from torchinfo import summary
import matplotlib.image as mpimg
import matplotlib.pyplot as plt
%matplotlib inline
import pandas as pd
import numpy as np
import math
import time
import os
import random
if torch.cuda.is_available():
torch.backends.cudnn.deterministic = True
In [5]:
def set_all_seeds(seed):
os.environ["PL_GLOBAL_SEED"] = str(seed)
random.seed(seed)
np.random.seed(seed)
torch.manual_seed(seed)
torch.cuda.manual_seed_all(seed)
In [6]:
# Check if GPU is available and set the device
if torch.cuda.is_available():
num_gpus = torch.cuda.device_count()
print("Num GPUs Available: ", num_gpus)
for i in range(num_gpus):
gpu_name = torch.cuda.get_device_name(i)
print(f"GPU {i}: {gpu_name}")
else:
print("No GPU available.")
Num GPUs Available: 2 GPU 0: Tesla T4 GPU 1: Tesla T4
In [7]:
##########################
### SETTINGS
##########################
# Hyperparameters
RANDOM_SEED = 123
BATCH_SIZE = 128 #256
LEARNING_RATE = 0.001
NUM_EPOCHS = 50
INPUT_SIZE = 32 # CIFAR-10 is 32x32
NUM_CLASSES = 10
# Set device
device = torch.device("cuda" if torch.cuda.is_available() else "cpu")
print(device)
cuda
In [8]:
set_all_seeds(RANDOM_SEED)
In [9]:
# CIFAR-10 specific stats (MEAN_arr, STD_arr)
stats = ((0.4914, 0.4822, 0.4465), (0.2471, 0.2435, 0.2616)) #(0.2023, 0.1994, 0.2010))
train_transforms = transforms.Compose([
transforms.RandomCrop(32, padding=4), # Standard CIFAR augmentation
transforms.RandomHorizontalFlip(),
transforms.ToTensor(),
transforms.Normalize(*stats)
])
test_transforms = transforms.Compose([
transforms.ToTensor(),
transforms.Normalize(*stats)
])
# Load Datasets
train_dataset = datasets.CIFAR10(root='./data', train=True, download=True, transform=train_transforms)
test_dataset = datasets.CIFAR10(root='./data', train=False, download=True, transform=test_transforms)
# Split training set into training and validation sets
train_size = int(0.9 * len(train_dataset))
val_size = len(train_dataset) - train_size
train_subset, val_subset = torch.utils.data.random_split(train_dataset, [train_size, val_size])
# Create DataLoaders
train_loader = DataLoader(train_subset, batch_size=BATCH_SIZE, shuffle=True, num_workers=2, pin_memory=True)
val_loader = DataLoader(val_subset, batch_size=BATCH_SIZE, shuffle=False, num_workers=2, pin_memory=True)
test_loader = DataLoader(test_dataset, batch_size=BATCH_SIZE, shuffle=False, num_workers=2, pin_memory=True)
classes = ('plane', 'car', 'bird', 'cat', 'deer', 'dog', 'frog', 'horse', 'ship', 'truck')
100%|██████████| 170M/170M [00:03<00:00, 50.0MB/s]
In [10]:
# Checking the dataset
for images, labels in train_loader:
print('Image batch dimensions:', images.shape)
print('Image label dimensions:', labels.shape)
print('Class labels of 10 examples:', labels[:10])
break
Image batch dimensions: torch.Size([128, 3, 32, 32]) Image label dimensions: torch.Size([128]) Class labels of 10 examples: tensor([2, 7, 4, 7, 0, 1, 1, 6, 9, 2])
In [11]:
plt.figure(figsize=(8, 8))
plt.axis("off")
plt.title("Training Images")
plt.imshow(np.transpose(torchvision.utils.make_grid(images[:64],
padding=2, normalize=True),
(1, 2, 0)))
Out[11]:
<matplotlib.image.AxesImage at 0x7df5d5183110>
2. Model Creation¶
Notes:
- The MobileNetV1 Building Block:
- The defining feature of V1 is the "Depthwise Separable" block, which splits a standard convolution into a spatial filter (Depthwise) and a channel combiner (Pointwise).
In [12]:
class DepthwiseSeparableConv(nn.Module):
def __init__(self, in_channels, out_channels, stride):
super(DepthwiseSeparableConv, self).__init__()
self.depthwise = nn.Sequential(
nn.Conv2d(in_channels, in_channels, kernel_size=3, stride=stride,
padding=1, groups=in_channels, bias=False),
nn.BatchNorm2d(in_channels),
nn.ReLU(inplace=True)
)
self.pointwise = nn.Sequential(
nn.Conv2d(in_channels, out_channels, kernel_size=1, stride=1,
padding=0, bias=False),
nn.BatchNorm2d(out_channels),
nn.ReLU(inplace=True)
)
def forward(self, x):
x = self.depthwise(x)
x = self.pointwise(x)
return x
class MobileNetV1(nn.Module):
def __init__(self, num_classes=10):
super(MobileNetV1, self).__init__()
# Initial standard convolution
# Stride=1 for CIFAR (originally 2 in paper)
self.model = nn.Sequential(
nn.Conv2d(3, 32, kernel_size=3, stride=1, padding=1, bias=False),
nn.BatchNorm2d(32),
nn.ReLU(inplace=True),
# (in_channels, out_channels, stride)
DepthwiseSeparableConv(32, 64, 1),
DepthwiseSeparableConv(64, 128, 1), # Changed from stride 2 to 1
DepthwiseSeparableConv(128, 128, 1),
DepthwiseSeparableConv(128, 256, 2),
DepthwiseSeparableConv(256, 256, 1),
DepthwiseSeparableConv(256, 512, 2),
# The 5 identical layers from the paper
DepthwiseSeparableConv(512, 512, 1),
DepthwiseSeparableConv(512, 512, 1),
DepthwiseSeparableConv(512, 512, 1),
DepthwiseSeparableConv(512, 512, 1),
DepthwiseSeparableConv(512, 512, 1),
DepthwiseSeparableConv(512, 1024, 2),
DepthwiseSeparableConv(1024, 1024, 1),
nn.AdaptiveAvgPool2d(1)
)
self.fc = nn.Linear(1024, num_classes)
def forward(self, x):
x = self.model(x)
x = x.view(x.size(0), -1)
x = self.fc(x)
return x
In [13]:
model = MobileNetV1().to(device)
In [14]:
# Model parameter details
summary(model,
input_size=(BATCH_SIZE, 3, INPUT_SIZE, INPUT_SIZE),
# verbose=1,
col_names=["input_size", "output_size", "num_params", "kernel_size", "mult_adds"]
)
Out[14]:
===================================================================================================================================================================== Layer (type:depth-idx) Input Shape Output Shape Param # Kernel Shape Mult-Adds ===================================================================================================================================================================== MobileNetV1 [128, 3, 32, 32] [128, 10] -- -- -- ├─Sequential: 1-1 [128, 3, 32, 32] [128, 1024, 1, 1] -- -- -- │ └─Conv2d: 2-1 [128, 3, 32, 32] [128, 32, 32, 32] 864 [3, 3] 113,246,208 │ └─BatchNorm2d: 2-2 [128, 32, 32, 32] [128, 32, 32, 32] 64 -- 8,192 │ └─ReLU: 2-3 [128, 32, 32, 32] [128, 32, 32, 32] -- -- -- │ └─DepthwiseSeparableConv: 2-4 [128, 32, 32, 32] [128, 64, 32, 32] -- -- -- │ │ └─Sequential: 3-1 [128, 32, 32, 32] [128, 32, 32, 32] 352 -- 37,756,928 │ │ └─Sequential: 3-2 [128, 32, 32, 32] [128, 64, 32, 32] 2,176 -- 268,451,840 │ └─DepthwiseSeparableConv: 2-5 [128, 64, 32, 32] [128, 128, 32, 32] -- -- -- │ │ └─Sequential: 3-3 [128, 64, 32, 32] [128, 64, 32, 32] 704 -- 75,513,856 │ │ └─Sequential: 3-4 [128, 64, 32, 32] [128, 128, 32, 32] 8,448 -- 1,073,774,592 │ └─DepthwiseSeparableConv: 2-6 [128, 128, 32, 32] [128, 128, 32, 32] -- -- -- │ │ └─Sequential: 3-5 [128, 128, 32, 32] [128, 128, 32, 32] 1,408 -- 151,027,712 │ │ └─Sequential: 3-6 [128, 128, 32, 32] [128, 128, 32, 32] 16,640 -- 2,147,516,416 │ └─DepthwiseSeparableConv: 2-7 [128, 128, 32, 32] [128, 256, 16, 16] -- -- -- │ │ └─Sequential: 3-7 [128, 128, 32, 32] [128, 128, 16, 16] 1,408 -- 37,781,504 │ │ └─Sequential: 3-8 [128, 128, 16, 16] [128, 256, 16, 16] 33,280 -- 1,073,807,360 │ └─DepthwiseSeparableConv: 2-8 [128, 256, 16, 16] [128, 256, 16, 16] -- -- -- │ │ └─Sequential: 3-9 [128, 256, 16, 16] [128, 256, 16, 16] 2,816 -- 75,563,008 │ │ └─Sequential: 3-10 [128, 256, 16, 16] [128, 256, 16, 16] 66,048 -- 2,147,549,184 │ └─DepthwiseSeparableConv: 2-9 [128, 256, 16, 16] [128, 512, 8, 8] -- -- -- │ │ └─Sequential: 3-11 [128, 256, 16, 16] [128, 256, 8, 8] 2,816 -- 18,939,904 │ │ └─Sequential: 3-12 [128, 256, 8, 8] [128, 512, 8, 8] 132,096 -- 1,073,872,896 │ └─DepthwiseSeparableConv: 2-10 [128, 512, 8, 8] [128, 512, 8, 8] -- -- -- │ │ └─Sequential: 3-13 [128, 512, 8, 8] [128, 512, 8, 8] 5,632 -- 37,879,808 │ │ └─Sequential: 3-14 [128, 512, 8, 8] [128, 512, 8, 8] 263,168 -- 2,147,614,720 │ └─DepthwiseSeparableConv: 2-11 [128, 512, 8, 8] [128, 512, 8, 8] -- -- -- │ │ └─Sequential: 3-15 [128, 512, 8, 8] [128, 512, 8, 8] 5,632 -- 37,879,808 │ │ └─Sequential: 3-16 [128, 512, 8, 8] [128, 512, 8, 8] 263,168 -- 2,147,614,720 │ └─DepthwiseSeparableConv: 2-12 [128, 512, 8, 8] [128, 512, 8, 8] -- -- -- │ │ └─Sequential: 3-17 [128, 512, 8, 8] [128, 512, 8, 8] 5,632 -- 37,879,808 │ │ └─Sequential: 3-18 [128, 512, 8, 8] [128, 512, 8, 8] 263,168 -- 2,147,614,720 │ └─DepthwiseSeparableConv: 2-13 [128, 512, 8, 8] [128, 512, 8, 8] -- -- -- │ │ └─Sequential: 3-19 [128, 512, 8, 8] [128, 512, 8, 8] 5,632 -- 37,879,808 │ │ └─Sequential: 3-20 [128, 512, 8, 8] [128, 512, 8, 8] 263,168 -- 2,147,614,720 │ └─DepthwiseSeparableConv: 2-14 [128, 512, 8, 8] [128, 512, 8, 8] -- -- -- │ │ └─Sequential: 3-21 [128, 512, 8, 8] [128, 512, 8, 8] 5,632 -- 37,879,808 │ │ └─Sequential: 3-22 [128, 512, 8, 8] [128, 512, 8, 8] 263,168 -- 2,147,614,720 │ └─DepthwiseSeparableConv: 2-15 [128, 512, 8, 8] [128, 1024, 4, 4] -- -- -- │ │ └─Sequential: 3-23 [128, 512, 8, 8] [128, 512, 4, 4] 5,632 -- 9,568,256 │ │ └─Sequential: 3-24 [128, 512, 4, 4] [128, 1024, 4, 4] 526,336 -- 1,074,003,968 │ └─DepthwiseSeparableConv: 2-16 [128, 1024, 4, 4] [128, 1024, 4, 4] -- -- -- │ │ └─Sequential: 3-25 [128, 1024, 4, 4] [128, 1024, 4, 4] 11,264 -- 19,136,512 │ │ └─Sequential: 3-26 [128, 1024, 4, 4] [128, 1024, 4, 4] 1,050,624 -- 2,147,745,792 │ └─AdaptiveAvgPool2d: 2-17 [128, 1024, 4, 4] [128, 1024, 1, 1] -- -- -- ├─Linear: 1-2 [128, 1024] [128, 10] 10,250 -- 1,312,000 ===================================================================================================================================================================== Total params: 3,217,226 Trainable params: 3,217,226 Non-trainable params: 0 Total mult-adds (Units.GIGABYTES): 22.47 ===================================================================================================================================================================== Input size (MB): 1.57 Forward/backward pass size (MB): 2566.92 Params size (MB): 12.87 Estimated Total Size (MB): 2581.37 =====================================================================================================================================================================
In [15]:
criterion = nn.CrossEntropyLoss()
optimizer = optim.Adam(model.parameters(), lr=LEARNING_RATE, weight_decay=5e-4)
# optimizer = optim.SGD(model.parameters(), lr=LEARNING_RATE, momentum=0.9, weight_decay=5e-4) # Use weight decay (L2) to combat overfitting
# scheduler = optim.lr_scheduler.ReduceLROnPlateau(optimizer, mode='min', factor=0.1, patience=5)
def calculate_accuracy(loader):
model.eval()
correct = 0
total = 0
with torch.no_grad():
for inputs, labels in loader:
inputs, labels = inputs.to(device), labels.to(device)
outputs = model(inputs)
_, predicted = torch.max(outputs.data, 1)
total += labels.size(0)
correct += (predicted == labels).sum().item()
return 100 * correct / total
# Training Loop
start_time = time.time()
history = {'train_loss': [], 'val_loss': [], 'train_acc': [], 'val_acc': [], 'lrs': []}
print(f"Starting training on {device}...")
for epoch in range(NUM_EPOCHS):
model.train()
running_loss = 0.0
for i, (inputs, labels) in enumerate(train_loader):
inputs, labels = inputs.to(device), labels.to(device)
optimizer.zero_grad()
outputs = model(inputs)
loss = criterion(outputs, labels)
loss.backward()
optimizer.step()
running_loss += loss.item() * inputs.size(0)
train_loss = running_loss / len(train_subset)
train_acc = calculate_accuracy(train_loader)
# Calculate validation loss and accuracy
model.eval()
val_loss = 0.0
with torch.no_grad():
for inputs, labels in val_loader:
inputs, labels = inputs.to(device), labels.to(device)
outputs = model(inputs)
loss = criterion(outputs, labels)
val_loss += loss.item() * inputs.size(0)
val_loss /= len(val_subset)
val_acc = calculate_accuracy(val_loader)
# scheduler.step(val_loss)
# current_lr = optimizer.param_groups[0]['lr']
# Log metrics
history['train_loss'].append(train_loss)
history['val_loss'].append(val_loss)
history['train_acc'].append(train_acc)
history['val_acc'].append(val_acc)
# history['lrs'].append(current_lr)
print(f'Epoch {epoch + 1}/{NUM_EPOCHS} | '
f'Train Loss: {train_loss:.5f}, '
f'Train Acc.: {train_acc:.2f}% | '
f'Val Loss: {val_loss:.5f}, '
f'Val Acc.: {val_acc:.2f}% | '
# f'LR: {current_lr:.8f}')
)
print('\nTotal Training Time: %.2f min' % ((time.time() - start_time)/60))
# --- Final Evaluation ---
test_accuracy = calculate_accuracy(test_loader)
print(f'\nTest Accuracy: {test_accuracy:.2f}%')
Starting training on cuda... Epoch 1/50 | Train Loss: 1.64079, Train Acc.: 48.22% | Val Loss: 1.46047, Val Acc.: 46.86% | Epoch 2/50 | Train Loss: 1.25678, Train Acc.: 58.51% | Val Loss: 1.20850, Val Acc.: 57.16% | Epoch 3/50 | Train Loss: 1.02700, Train Acc.: 68.01% | Val Loss: 0.97419, Val Acc.: 65.02% | Epoch 4/50 | Train Loss: 0.85995, Train Acc.: 73.91% | Val Loss: 0.82695, Val Acc.: 71.44% | Epoch 5/50 | Train Loss: 0.74043, Train Acc.: 76.74% | Val Loss: 0.75273, Val Acc.: 73.78% | Epoch 6/50 | Train Loss: 0.65644, Train Acc.: 78.99% | Val Loss: 0.69230, Val Acc.: 75.56% | Epoch 7/50 | Train Loss: 0.60376, Train Acc.: 79.54% | Val Loss: 0.69671, Val Acc.: 76.58% | Epoch 8/50 | Train Loss: 0.55649, Train Acc.: 81.78% | Val Loss: 0.62899, Val Acc.: 78.00% | Epoch 9/50 | Train Loss: 0.51982, Train Acc.: 83.25% | Val Loss: 0.61162, Val Acc.: 79.56% | Epoch 10/50 | Train Loss: 0.48877, Train Acc.: 83.92% | Val Loss: 0.57651, Val Acc.: 79.40% | Epoch 11/50 | Train Loss: 0.45820, Train Acc.: 85.57% | Val Loss: 0.56024, Val Acc.: 80.72% | Epoch 12/50 | Train Loss: 0.43873, Train Acc.: 87.18% | Val Loss: 0.52572, Val Acc.: 82.40% | Epoch 13/50 | Train Loss: 0.42062, Train Acc.: 86.75% | Val Loss: 0.51081, Val Acc.: 82.68% | Epoch 14/50 | Train Loss: 0.40477, Train Acc.: 87.81% | Val Loss: 0.50711, Val Acc.: 82.64% | Epoch 15/50 | Train Loss: 0.39010, Train Acc.: 87.96% | Val Loss: 0.49031, Val Acc.: 83.08% | Epoch 16/50 | Train Loss: 0.37258, Train Acc.: 89.37% | Val Loss: 0.45802, Val Acc.: 84.24% | Epoch 17/50 | Train Loss: 0.36669, Train Acc.: 87.90% | Val Loss: 0.51312, Val Acc.: 82.96% | Epoch 18/50 | Train Loss: 0.35128, Train Acc.: 88.81% | Val Loss: 0.46996, Val Acc.: 83.74% | Epoch 19/50 | Train Loss: 0.34055, Train Acc.: 89.27% | Val Loss: 0.47333, Val Acc.: 84.00% | Epoch 20/50 | Train Loss: 0.33007, Train Acc.: 89.81% | Val Loss: 0.45105, Val Acc.: 85.38% | Epoch 21/50 | Train Loss: 0.32189, Train Acc.: 88.88% | Val Loss: 0.49285, Val Acc.: 83.56% | Epoch 22/50 | Train Loss: 0.31552, Train Acc.: 90.57% | Val Loss: 0.41849, Val Acc.: 85.70% | Epoch 23/50 | Train Loss: 0.30682, Train Acc.: 90.04% | Val Loss: 0.46608, Val Acc.: 84.42% | Epoch 24/50 | Train Loss: 0.29913, Train Acc.: 91.11% | Val Loss: 0.41873, Val Acc.: 85.82% | Epoch 25/50 | Train Loss: 0.29292, Train Acc.: 90.11% | Val Loss: 0.45570, Val Acc.: 84.46% | Epoch 26/50 | Train Loss: 0.29292, Train Acc.: 90.81% | Val Loss: 0.44096, Val Acc.: 84.60% | Epoch 27/50 | Train Loss: 0.28567, Train Acc.: 90.92% | Val Loss: 0.42735, Val Acc.: 85.58% | Epoch 28/50 | Train Loss: 0.28020, Train Acc.: 90.62% | Val Loss: 0.43850, Val Acc.: 84.68% | Epoch 29/50 | Train Loss: 0.27304, Train Acc.: 90.13% | Val Loss: 0.47988, Val Acc.: 84.56% | Epoch 30/50 | Train Loss: 0.26935, Train Acc.: 90.09% | Val Loss: 0.45858, Val Acc.: 85.88% | Epoch 31/50 | Train Loss: 0.26775, Train Acc.: 91.21% | Val Loss: 0.42217, Val Acc.: 85.78% | Epoch 32/50 | Train Loss: 0.25962, Train Acc.: 92.30% | Val Loss: 0.40230, Val Acc.: 86.70% | Epoch 33/50 | Train Loss: 0.25692, Train Acc.: 92.00% | Val Loss: 0.40784, Val Acc.: 86.68% | Epoch 34/50 | Train Loss: 0.25927, Train Acc.: 91.38% | Val Loss: 0.43140, Val Acc.: 86.14% | Epoch 35/50 | Train Loss: 0.25001, Train Acc.: 92.32% | Val Loss: 0.41445, Val Acc.: 86.28% | Epoch 36/50 | Train Loss: 0.24987, Train Acc.: 92.47% | Val Loss: 0.41148, Val Acc.: 86.48% | Epoch 37/50 | Train Loss: 0.24776, Train Acc.: 91.86% | Val Loss: 0.41339, Val Acc.: 86.62% | Epoch 38/50 | Train Loss: 0.24610, Train Acc.: 92.61% | Val Loss: 0.39342, Val Acc.: 86.74% | Epoch 39/50 | Train Loss: 0.23935, Train Acc.: 92.28% | Val Loss: 0.41510, Val Acc.: 86.32% | Epoch 40/50 | Train Loss: 0.24228, Train Acc.: 91.97% | Val Loss: 0.40183, Val Acc.: 86.98% | Epoch 41/50 | Train Loss: 0.23453, Train Acc.: 92.85% | Val Loss: 0.39250, Val Acc.: 86.22% | Epoch 42/50 | Train Loss: 0.23311, Train Acc.: 92.87% | Val Loss: 0.40533, Val Acc.: 87.10% | Epoch 43/50 | Train Loss: 0.22906, Train Acc.: 92.56% | Val Loss: 0.41489, Val Acc.: 86.26% | Epoch 44/50 | Train Loss: 0.23078, Train Acc.: 93.27% | Val Loss: 0.38408, Val Acc.: 87.80% | Epoch 45/50 | Train Loss: 0.22391, Train Acc.: 92.72% | Val Loss: 0.38287, Val Acc.: 87.04% | Epoch 46/50 | Train Loss: 0.21816, Train Acc.: 91.80% | Val Loss: 0.39818, Val Acc.: 86.98% | Epoch 47/50 | Train Loss: 0.22660, Train Acc.: 92.36% | Val Loss: 0.42409, Val Acc.: 86.42% | Epoch 48/50 | Train Loss: 0.21980, Train Acc.: 93.04% | Val Loss: 0.40863, Val Acc.: 87.40% | Epoch 49/50 | Train Loss: 0.21891, Train Acc.: 93.25% | Val Loss: 0.38614, Val Acc.: 87.42% | Epoch 50/50 | Train Loss: 0.21517, Train Acc.: 93.61% | Val Loss: 0.37717, Val Acc.: 87.64% | Total Training Time: 45.72 min Test Accuracy: 88.61%
In [16]:
def plot_history(history):
"""
Plots the training and validation loss, accuracy, and learning rate
from a PyTorch history dictionary.
Args:
history (dict): Dictionary containing training metrics.
Expected keys: 'train_loss', 'val_loss', 'train_acc', 'val_acc', 'lrs'.
"""
epochs = range(len(history['train_loss']))
plt.figure(figsize=(18, 5))
# Plot 1: Loss
plt.subplot(1, 3, 1)
plt.plot(epochs, history['train_loss'], label='Training Loss')
plt.plot(epochs, history['val_loss'], label='Validation Loss')
plt.title('Loss')
plt.xlabel('Epochs')
plt.ylabel('Loss')
plt.legend()
# plt.grid(True)
# Plot 2: Accuracy
plt.subplot(1, 3, 2)
plt.plot(epochs, history['train_acc'], label='Training Accuracy')
plt.plot(epochs, history['val_acc'], label='Validation Accuracy')
plt.title('Accuracy')
plt.xlabel('Epochs')
plt.ylabel('Accuracy')
plt.legend()
# plt.grid(True)
plt.tight_layout()
plt.show()
# plt.savefig()
In [17]:
plot_history(history)
In [19]:
from sklearn.metrics import confusion_matrix
import seaborn as sns
def _compute_confusion_matrix(model, data_loader, device):
"""
Computes a confusion matrix for the given model and data.
Uses scikit-learn for robustness and speed.
"""
model.eval()
all_targets = []
all_predictions = []
with torch.no_grad():
for features, targets in data_loader:
features = features.to(device)
logits = model(features)
_, predicted_labels = torch.max(logits, 1)
# Move to CPU and convert to list/numpy immediately
all_targets.extend(targets.view(-1).cpu().numpy())
all_predictions.extend(predicted_labels.view(-1).cpu().numpy())
return confusion_matrix(all_targets, all_predictions)
def _plot_confusion_matrix(mat, class_names):
"""
Visualizes the confusion matrix using Seaborn for professional reporting.
"""
plt.figure(figsize=(10, 8))
sns.heatmap(mat, annot=True, fmt='d', cmap='Blues',
xticklabels=class_names, yticklabels=class_names)
plt.xlabel('Predicted Label')
plt.ylabel('True Label')
plt.title('Confusion Matrix: MobileNetV2 on CIFAR-10')
plt.show()
In [21]:
class_dict = {0: 'plane',
1: 'car',
2: 'bird',
3: 'cat',
4: 'deer',
5: 'dog',
6: 'frog',
7: 'horse',
8: 'ship',
9: 'truck'}
_mat = _compute_confusion_matrix(model=model, data_loader=test_loader, device=device)
_plot_confusion_matrix(_mat, class_names=class_dict.values())
In [22]:
%watermark -iv
matplotlib : 3.10.0 numpy : 2.0.2 pandas : 2.2.2 seaborn : 0.13.2 sklearn : 1.6.1 torch : 2.8.0+cu126 torchinfo : 1.8.0 torchvision: 0.23.0+cu126
In [ ]: