Deep Learning Models: A collection of various deep learning architectures, models, & their implementations (PyTorch or TensorFlow) in Jupyter Notebooks.
- Author: Chizoba Obasi
- Github Repo: https://github.com/chizkidd/deep-learning-models
In [13]:
!pip install watermark
Requirement already satisfied: watermark in /usr/local/lib/python3.12/dist-packages (2.6.0) Requirement already satisfied: ipython>=6.0 in /usr/local/lib/python3.12/dist-packages (from watermark) (7.34.0) Requirement already satisfied: importlib-metadata>=1.4 in /usr/local/lib/python3.12/dist-packages (from watermark) (8.7.0) Requirement already satisfied: setuptools in /usr/local/lib/python3.12/dist-packages (from watermark) (75.2.0) Requirement already satisfied: zipp>=3.20 in /usr/local/lib/python3.12/dist-packages (from importlib-metadata>=1.4->watermark) (3.23.0) Requirement already satisfied: jedi>=0.16 in /usr/local/lib/python3.12/dist-packages (from ipython>=6.0->watermark) (0.19.2) Requirement already satisfied: decorator in /usr/local/lib/python3.12/dist-packages (from ipython>=6.0->watermark) (4.4.2) Requirement already satisfied: pickleshare in /usr/local/lib/python3.12/dist-packages (from ipython>=6.0->watermark) (0.7.5) Requirement already satisfied: traitlets>=4.2 in /usr/local/lib/python3.12/dist-packages (from ipython>=6.0->watermark) (5.7.1) Requirement already satisfied: prompt-toolkit!=3.0.0,!=3.0.1,<3.1.0,>=2.0.0 in /usr/local/lib/python3.12/dist-packages (from ipython>=6.0->watermark) (3.0.52) Requirement already satisfied: pygments in /usr/local/lib/python3.12/dist-packages (from ipython>=6.0->watermark) (2.19.2) Requirement already satisfied: backcall in /usr/local/lib/python3.12/dist-packages (from ipython>=6.0->watermark) (0.2.0) Requirement already satisfied: matplotlib-inline in /usr/local/lib/python3.12/dist-packages (from ipython>=6.0->watermark) (0.1.7) Requirement already satisfied: pexpect>4.3 in /usr/local/lib/python3.12/dist-packages (from ipython>=6.0->watermark) (4.9.0) Requirement already satisfied: parso<0.9.0,>=0.8.4 in /usr/local/lib/python3.12/dist-packages (from jedi>=0.16->ipython>=6.0->watermark) (0.8.5) Requirement already satisfied: ptyprocess>=0.5 in /usr/local/lib/python3.12/dist-packages (from pexpect>4.3->ipython>=6.0->watermark) (0.7.0) Requirement already satisfied: wcwidth in /usr/local/lib/python3.12/dist-packages (from prompt-toolkit!=3.0.0,!=3.0.1,<3.1.0,>=2.0.0->ipython>=6.0->watermark) (0.2.14)
In [14]:
!pip install torchinfo
Requirement already satisfied: torchinfo in /usr/local/lib/python3.12/dist-packages (1.8.0)
In [15]:
%load_ext watermark
# %reload_ext watermark
%watermark -a 'Chizoba Obasi' -v -p torch
The watermark extension is already loaded. To reload it, use: %reload_ext watermark Author: Chizoba Obasi Python implementation: CPython Python version : 3.12.12 IPython version : 7.34.0 torch: 2.8.0+cu126
Network-in-Network (NiN) Classifier on CIFAR-10¶
Network Architecture:
Network in Network essentially removed the "parameter-heavy" fully connected layers seen in AlexNet, LeNet and VGG by using Global Average Pooling and also introduced $1\times1$ convolutions.
Improvements of NiN over LeNet, AlexNet, VGG:¶
- 1×1 Convolutions (MLPconv):
NiNreplaced standard linear convolutional layers with "micro-networks" (MLPs) implemented as $1\times1$ convolutions that add local nonlinearities across the channel activations. WhileVGGandAlexNetfilters only extract spatial features, NiN's $1\times1$ layers allow the model to learn complex non-linear relationships across channels at every single pixel location. - Global Average Pooling (GAP): This is NiN's most famous contribution which was used to integrate across all locations in the last representation layer. Instead of using massive "Flatten + Dense" layers at the end of the network (which account for ~90% of AlexNet's parameters), NiN takes the average of each feature map. This drastically reduces the parameter count and acts as a strong regularizer to prevent overfitting. Note that global average pooling would not be effective, were it not for the added nonlinearities.
- Deep Cross-Channel Pooling: By stacking multiple $1\times1$ convolutions, NiN performs a form of "channel-wise" dimensionality reduction or expansion. This allows the network to increase its representational power (depth) without the explosive memory cost associated with the large 4096-unit fully connected layers found in VGG.
Quick Comparison Table¶
| Feature | AlexNet / VGG | Network In Network (NiN) |
|---|---|---|
| Top-level Classifier | Fully Connected (Dense) Layers | Global Average Pooling |
| Feature Extraction | Linear Convolutional Filters | MLPconv (1×1 Convolutions) |
| Parameter Density | Extremely High (due to FC layers) | Very Low (more efficient) |
| Main Innovation | Deeper stacks of 3x3/5x5 | Cross-channel non-linearity |
REFERENCES:
- Lin, M., Chen, Q., & Yan, S. "Network In Network." International Conference on Learning Representations (ICLR), 2014. arXiv:1312.4400.
- Zhang, Aston, et al. "Network in Network (NiN) (Chapter 8.3)." Dive into Deep Learning. Cambridge University Press, 2023.
- Google. (2025). "Gemini 3 Flash" [Large language model]. Google AI.
In [16]:
import torch
import torch.nn as nn
import torch.nn.functional as F
import torch.optim as optim
from torch.utils.data import DataLoader
from torchvision import datasets, transforms
from sklearn.model_selection import train_test_split
# from torch.cuda.amp import GradScaler, autocast
from torch.amp import GradScaler, autocast
from torchinfo import summary
import matplotlib.image as mpimg
import matplotlib.pyplot as plt
%matplotlib inline
import pandas as pd
import numpy as np
import math
import time
import os
import random
if torch.cuda.is_available():
torch.backends.cudnn.deterministic = True
In [17]:
def set_all_seeds(seed):
os.environ["PL_GLOBAL_SEED"] = str(seed)
random.seed(seed)
np.random.seed(seed)
torch.manual_seed(seed)
torch.cuda.manual_seed_all(seed)
In [18]:
##########################
### SETTINGS
##########################
# Hyperparameters
RANDOM_SEED = 123
BATCH_SIZE = 128
LEARNING_RATE = 0.001
NUM_EPOCHS = 80
INPUT_SIZE = 32 # CIFAR-10 is 32x32
NUM_CLASSES = 10
# Set device
device = torch.device("cuda" if torch.cuda.is_available() else "cpu")
print(device)
cuda
In [19]:
# Mean and Std Dev for CIFAR-10 (3 channels)
CIFAR_MEAN = (0.4914, 0.4822, 0.4465)
CIFAR_STD = (0.2471, 0.2435, 0.2616)
# CIFAR-10 preprocessing
train_transforms = transforms.Compose([
transforms.RandomCrop(32, padding=4),
transforms.RandomHorizontalFlip(),
transforms.ToTensor(),
transforms.Normalize(CIFAR_MEAN, CIFAR_STD)
])
test_transforms = transforms.Compose([
transforms.ToTensor(),
transforms.Normalize(CIFAR_MEAN, CIFAR_STD)
])
# Load Datasets
train_dataset = datasets.CIFAR10(root='./data', train=True, download=True, transform=train_transforms)
test_dataset = datasets.CIFAR10(root='./data', train=False, download=True, transform=test_transforms)
# Split training set into training and validation sets
train_size = int(0.9 * len(train_dataset))
val_size = len(train_dataset) - train_size
train_subset, val_subset = torch.utils.data.random_split(train_dataset, [train_size, val_size])
# Create DataLoaders
train_loader = DataLoader(train_subset, batch_size=BATCH_SIZE, shuffle=True, num_workers=2, pin_memory=True)
val_loader = DataLoader(val_subset, batch_size=BATCH_SIZE, shuffle=False, num_workers=2, pin_memory=True)
test_loader = DataLoader(test_dataset, batch_size=BATCH_SIZE, shuffle=False, num_workers=2, pin_memory=True)
In [20]:
def nin_block(in_channels, out_channels, kernel_size, strides, padding):
return nn.Sequential(
nn.Conv2d(in_channels, out_channels, kernel_size, strides, padding),
nn.BatchNorm2d(out_channels),
nn.ReLU(inplace=True),
# The "Network in Network" part: two 1x1 convolutions (MLPs)
nn.Conv2d(out_channels, out_channels, kernel_size=1),
nn.BatchNorm2d(out_channels),
nn.ReLU(inplace=True),
nn.Conv2d(out_channels, out_channels, kernel_size=1),
nn.BatchNorm2d(out_channels),
nn.ReLU(inplace=True)
)
class NiN(nn.Module):
def __init__(self, num_classes=10):
super(NiN, self).__init__()
self.net = nn.Sequential(
nin_block(3, 192, kernel_size=5, strides=1, padding=2),
nn.MaxPool2d(3, stride=2, padding=1),
nn.Dropout(0.5),
nin_block(192, 160, kernel_size=5, strides=1, padding=2),
nn.MaxPool2d(3, stride=2, padding=1),
nn.Dropout(0.5),
nin_block(160, 96, kernel_size=3, strides=1, padding=1),
nn.MaxPool2d(3, stride=2, padding=1),
nn.Dropout(0.5),
# Final block reduces channels to number of classes
nin_block(96, num_classes, kernel_size=3, strides=1, padding=1),
# Global Average Pooling replaces Fully Connected layers
nn.AdaptiveAvgPool2d((1, 1)),
nn.Flatten()
)
def forward(self, x):
return self.net(x)
print(device)
cuda
In [21]:
def test_model_dims():
# Create a dummy batch: 2 images, 3 channels, 32x32 size
x = torch.randn(2, 3, 32, 32).to(device)
# Initialize model
net = NiN().to(device)
# Forward pass
y = net(x)
print(f"Input Shape: {x.shape}")
print(f"Output Shape: {y.shape}") # Should be [2, 10]
if y.shape == (2, 10):
print("✅ Sanity Check Passed: Dimensions are correct for CIFAR-10!")
else:
print("❌ Sanity Check Failed: Check your stride or pooling layers.")
test_model_dims()
Input Shape: torch.Size([2, 3, 32, 32]) Output Shape: torch.Size([2, 10]) ✅ Sanity Check Passed: Dimensions are correct for CIFAR-10!
In [22]:
set_all_seeds(RANDOM_SEED)
model = NiN(num_classes=NUM_CLASSES).to(device)
# Model parameter details
summary(model,
input_size=(BATCH_SIZE, 3, INPUT_SIZE, INPUT_SIZE), # verbose=1,
col_names=["input_size", "output_size", "num_params", "kernel_size", "mult_adds"]
)
Out[22]:
===================================================================================================================================================================== Layer (type:depth-idx) Input Shape Output Shape Param # Kernel Shape Mult-Adds ===================================================================================================================================================================== NiN [128, 3, 32, 32] [128, 10] -- -- -- ├─Sequential: 1-1 [128, 3, 32, 32] [128, 10] -- -- -- │ └─Sequential: 2-1 [128, 3, 32, 32] [128, 192, 32, 32] -- -- -- │ │ └─Conv2d: 3-1 [128, 3, 32, 32] [128, 192, 32, 32] 14,592 [5, 5] 1,912,602,624 │ │ └─BatchNorm2d: 3-2 [128, 192, 32, 32] [128, 192, 32, 32] 384 -- 49,152 │ │ └─ReLU: 3-3 [128, 192, 32, 32] [128, 192, 32, 32] -- -- -- │ │ └─Conv2d: 3-4 [128, 192, 32, 32] [128, 192, 32, 32] 37,056 [1, 1] 4,857,004,032 │ │ └─BatchNorm2d: 3-5 [128, 192, 32, 32] [128, 192, 32, 32] 384 -- 49,152 │ │ └─ReLU: 3-6 [128, 192, 32, 32] [128, 192, 32, 32] -- -- -- │ │ └─Conv2d: 3-7 [128, 192, 32, 32] [128, 192, 32, 32] 37,056 [1, 1] 4,857,004,032 │ │ └─BatchNorm2d: 3-8 [128, 192, 32, 32] [128, 192, 32, 32] 384 -- 49,152 │ │ └─ReLU: 3-9 [128, 192, 32, 32] [128, 192, 32, 32] -- -- -- │ └─MaxPool2d: 2-2 [128, 192, 32, 32] [128, 192, 16, 16] -- 3 -- │ └─Dropout: 2-3 [128, 192, 16, 16] [128, 192, 16, 16] -- -- -- │ └─Sequential: 2-4 [128, 192, 16, 16] [128, 160, 16, 16] -- -- -- │ │ └─Conv2d: 3-10 [128, 192, 16, 16] [128, 160, 16, 16] 768,160 [5, 5] 25,171,066,880 │ │ └─BatchNorm2d: 3-11 [128, 160, 16, 16] [128, 160, 16, 16] 320 -- 40,960 │ │ └─ReLU: 3-12 [128, 160, 16, 16] [128, 160, 16, 16] -- -- -- │ │ └─Conv2d: 3-13 [128, 160, 16, 16] [128, 160, 16, 16] 25,760 [1, 1] 844,103,680 │ │ └─BatchNorm2d: 3-14 [128, 160, 16, 16] [128, 160, 16, 16] 320 -- 40,960 │ │ └─ReLU: 3-15 [128, 160, 16, 16] [128, 160, 16, 16] -- -- -- │ │ └─Conv2d: 3-16 [128, 160, 16, 16] [128, 160, 16, 16] 25,760 [1, 1] 844,103,680 │ │ └─BatchNorm2d: 3-17 [128, 160, 16, 16] [128, 160, 16, 16] 320 -- 40,960 │ │ └─ReLU: 3-18 [128, 160, 16, 16] [128, 160, 16, 16] -- -- -- │ └─MaxPool2d: 2-5 [128, 160, 16, 16] [128, 160, 8, 8] -- 3 -- │ └─Dropout: 2-6 [128, 160, 8, 8] [128, 160, 8, 8] -- -- -- │ └─Sequential: 2-7 [128, 160, 8, 8] [128, 96, 8, 8] -- -- -- │ │ └─Conv2d: 3-19 [128, 160, 8, 8] [128, 96, 8, 8] 138,336 [3, 3] 1,133,248,512 │ │ └─BatchNorm2d: 3-20 [128, 96, 8, 8] [128, 96, 8, 8] 192 -- 24,576 │ │ └─ReLU: 3-21 [128, 96, 8, 8] [128, 96, 8, 8] -- -- -- │ │ └─Conv2d: 3-22 [128, 96, 8, 8] [128, 96, 8, 8] 9,312 [1, 1] 76,283,904 │ │ └─BatchNorm2d: 3-23 [128, 96, 8, 8] [128, 96, 8, 8] 192 -- 24,576 │ │ └─ReLU: 3-24 [128, 96, 8, 8] [128, 96, 8, 8] -- -- -- │ │ └─Conv2d: 3-25 [128, 96, 8, 8] [128, 96, 8, 8] 9,312 [1, 1] 76,283,904 │ │ └─BatchNorm2d: 3-26 [128, 96, 8, 8] [128, 96, 8, 8] 192 -- 24,576 │ │ └─ReLU: 3-27 [128, 96, 8, 8] [128, 96, 8, 8] -- -- -- │ └─MaxPool2d: 2-8 [128, 96, 8, 8] [128, 96, 4, 4] -- 3 -- │ └─Dropout: 2-9 [128, 96, 4, 4] [128, 96, 4, 4] -- -- -- │ └─Sequential: 2-10 [128, 96, 4, 4] [128, 10, 4, 4] -- -- -- │ │ └─Conv2d: 3-28 [128, 96, 4, 4] [128, 10, 4, 4] 8,650 [3, 3] 17,715,200 │ │ └─BatchNorm2d: 3-29 [128, 10, 4, 4] [128, 10, 4, 4] 20 -- 2,560 │ │ └─ReLU: 3-30 [128, 10, 4, 4] [128, 10, 4, 4] -- -- -- │ │ └─Conv2d: 3-31 [128, 10, 4, 4] [128, 10, 4, 4] 110 [1, 1] 225,280 │ │ └─BatchNorm2d: 3-32 [128, 10, 4, 4] [128, 10, 4, 4] 20 -- 2,560 │ │ └─ReLU: 3-33 [128, 10, 4, 4] [128, 10, 4, 4] -- -- -- │ │ └─Conv2d: 3-34 [128, 10, 4, 4] [128, 10, 4, 4] 110 [1, 1] 225,280 │ │ └─BatchNorm2d: 3-35 [128, 10, 4, 4] [128, 10, 4, 4] 20 -- 2,560 │ │ └─ReLU: 3-36 [128, 10, 4, 4] [128, 10, 4, 4] -- -- -- │ └─AdaptiveAvgPool2d: 2-11 [128, 10, 4, 4] [128, 10, 1, 1] -- -- -- │ └─Flatten: 2-12 [128, 10, 1, 1] [128, 10] -- -- -- ===================================================================================================================================================================== Total params: 1,076,962 Trainable params: 1,076,962 Non-trainable params: 0 Total mult-adds (Units.GIGABYTES): 39.79 ===================================================================================================================================================================== Input size (MB): 1.57 Forward/backward pass size (MB): 1498.35 Params size (MB): 4.31 Estimated Total Size (MB): 1504.23 =====================================================================================================================================================================
In [23]:
# criterion = nn.CrossEntropyLoss()
# optimizer = optim.Adam(model.parameters(), lr=LEARNING_RATE)
# # optimizer = optim.SGD(model.parameters(), lr=LEARNING_RATE, momentum=0.9, weight_decay=5e-4)
# # scheduler = optim.lr_scheduler.ReduceLROnPlateau(optimizer, mode='min', factor=0.1, patience=5)
def calculate_accuracy(loader):
model.eval()
correct = 0
total = 0
with torch.no_grad():
for inputs, labels in loader:
inputs, labels = inputs.to(device), labels.to(device)
outputs = model(inputs)
_, predicted = torch.max(outputs.data, 1)
total += labels.size(0)
correct += (predicted == labels).sum().item()
return 100 * correct / total
# # Training Loop
# start_time = time.time()
# history = {'train_loss': [], 'val_loss': [], 'train_acc': [], 'val_acc': [], 'lrs': []}
# print(f"Starting training on {device}...")
# for epoch in range(NUM_EPOCHS):
# model.train()
# running_loss = 0.0
# for i, (inputs, labels) in enumerate(train_loader):
# inputs, labels = inputs.to(device), labels.to(device)
# optimizer.zero_grad()
# outputs = model(inputs)
# loss = criterion(outputs, labels)
# loss.backward()
# optimizer.step()
# running_loss += loss.item() * inputs.size(0)
# train_loss = running_loss / len(train_subset)
# train_acc = calculate_accuracy(train_loader)
# # Calculate validation loss and accuracy
# model.eval()
# val_loss = 0.0
# with torch.no_grad():
# for inputs, labels in val_loader:
# inputs, labels = inputs.to(device), labels.to(device)
# outputs = model(inputs)
# loss = criterion(outputs, labels)
# val_loss += loss.item() * inputs.size(0)
# val_loss /= len(val_subset)
# val_acc = calculate_accuracy(val_loader)
# # scheduler.step(val_loss)
# # current_lr = optimizer.param_groups[0]['lr']
# # Log metrics
# history['train_loss'].append(train_loss)
# history['val_loss'].append(val_loss)
# history['train_acc'].append(train_acc)
# history['val_acc'].append(val_acc)
# # history['lrs'].append(current_lr)
# print(f'Epoch {epoch + 1}/{NUM_EPOCHS} | '
# f'Train Loss: {train_loss:.5f}, '
# f'Train Acc.: {train_acc:.2f}% | '
# f'Val Loss: {val_loss:.5f}, '
# f'Val Acc.: {val_acc:.2f}% | '
# # f'LR: {current_lr:.8f}'
# )
# print('\nTotal Training Time: %.2f min' % ((time.time() - start_time)/60))
# # --- Final Evaluation ---
# test_accuracy = calculate_accuracy(test_loader)
# print(f'\nTest Accuracy: {test_accuracy:.2f}%')
In [24]:
# 1. Update Optimizer and Scheduler
criterion = nn.CrossEntropyLoss()
optimizer = optim.AdamW(model.parameters(), lr=LEARNING_RATE, weight_decay=0.01)
# OneCycleLR setup
scheduler = optim.lr_scheduler.OneCycleLR(
optimizer,
max_lr=0.01, # Peak LR (higher than standard Adam)
steps_per_epoch=len(train_loader), # Steps per epoch
epochs=NUM_EPOCHS, # Total epochs
pct_start=0.3, # Spend 30% of time warming up
anneal_strategy='cos' # Cosine annealing
)
# Initialize AMP GradScaler
scaler = GradScaler('cuda')
# Training Loop
start_time = time.time()
history = {'train_loss': [], 'val_loss': [], 'train_acc': [], 'val_acc': [], 'lrs': []}
print(f"Starting training on {device}...")
for epoch in range(NUM_EPOCHS):
model.train()
running_loss = 0.0
for i, (inputs, labels) in enumerate(train_loader):
inputs, labels = inputs.to(device), labels.to(device)
optimizer.zero_grad()
with autocast('cuda'):
outputs = model(inputs)
loss = criterion(outputs, labels)
scaler.scale(loss).backward() # Scale the loss and call backward()
# scaler.step() first unscales the gradients of the optimizer's assigned params.
# If these gradients do not contain infs or NaNs, optimizer.step() is then called.
scaler.step(optimizer)
scaler.update()
scheduler.step()
running_loss += loss.item() * inputs.size(0)
# Log current LR from the optimizer
current_lr = optimizer.param_groups[0]['lr']
train_loss = running_loss / len(train_subset)
train_acc = calculate_accuracy(train_loader)
# Calculate validation loss and accuracy
model.eval()
val_loss = 0.0
with torch.no_grad():
for inputs, labels in val_loader:
inputs, labels = inputs.to(device), labels.to(device)
with autocast('cuda'):
outputs = model(inputs)
loss = criterion(outputs, labels)
val_loss += loss.item() * inputs.size(0)
val_loss /= len(val_subset)
val_acc = calculate_accuracy(val_loader)
# Log metrics
history['train_loss'].append(train_loss)
history['val_loss'].append(val_loss)
history['train_acc'].append(train_acc)
history['val_acc'].append(val_acc)
history['lrs'].append(current_lr)
print(f'Epoch {epoch + 1}/{NUM_EPOCHS} | '
f'Train Loss: {train_loss:.5f}, '
f'Train Acc.: {train_acc:.2f}% | '
f'Val Loss: {val_loss:.5f}, '
f'Val Acc.: {val_acc:.2f}% | '
f'LR: {current_lr:.6f}')
print('\nTotal Training Time: %.2f min' % ((time.time() - start_time)/60))
# --- Final Evaluation ---
test_accuracy = calculate_accuracy(test_loader)
print(f'\nTest Accuracy: {test_accuracy:.2f}%')
Starting training on cuda... Epoch 1/80 | Train Loss: 1.90253, Train Acc.: 44.84% | Val Loss: 1.62372, Val Acc.: 44.40% | LR: 0.000441 Epoch 2/80 | Train Loss: 1.44622, Train Acc.: 60.61% | Val Loss: 1.26193, Val Acc.: 60.00% | LR: 0.000564 Epoch 3/80 | Train Loss: 1.20882, Train Acc.: 59.00% | Val Loss: 1.25214, Val Acc.: 58.28% | LR: 0.000765 Epoch 4/80 | Train Loss: 1.07695, Train Acc.: 55.58% | Val Loss: 1.33912, Val Acc.: 55.44% | LR: 0.001043 Epoch 5/80 | Train Loss: 0.99023, Train Acc.: 70.49% | Val Loss: 0.92761, Val Acc.: 68.56% | LR: 0.001392 Epoch 6/80 | Train Loss: 0.91351, Train Acc.: 64.56% | Val Loss: 1.11073, Val Acc.: 63.50% | LR: 0.001806 Epoch 7/80 | Train Loss: 0.86316, Train Acc.: 72.45% | Val Loss: 0.87032, Val Acc.: 71.36% | LR: 0.002278 Epoch 8/80 | Train Loss: 0.81705, Train Acc.: 76.04% | Val Loss: 0.77137, Val Acc.: 74.24% | LR: 0.002801 Epoch 9/80 | Train Loss: 0.77947, Train Acc.: 73.34% | Val Loss: 0.86698, Val Acc.: 71.24% | LR: 0.003364 Epoch 10/80 | Train Loss: 0.74242, Train Acc.: 77.40% | Val Loss: 0.72543, Val Acc.: 75.58% | LR: 0.003958 Epoch 11/80 | Train Loss: 0.71350, Train Acc.: 78.70% | Val Loss: 0.70994, Val Acc.: 76.06% | LR: 0.004574 Epoch 12/80 | Train Loss: 0.68859, Train Acc.: 77.81% | Val Loss: 0.69876, Val Acc.: 75.90% | LR: 0.005201 Epoch 13/80 | Train Loss: 0.67498, Train Acc.: 75.01% | Val Loss: 0.83561, Val Acc.: 73.26% | LR: 0.005827 Epoch 14/80 | Train Loss: 0.65950, Train Acc.: 77.79% | Val Loss: 0.74667, Val Acc.: 75.60% | LR: 0.006443 Epoch 15/80 | Train Loss: 0.64951, Train Acc.: 77.68% | Val Loss: 0.72949, Val Acc.: 75.96% | LR: 0.007038 Epoch 16/80 | Train Loss: 0.63008, Train Acc.: 76.92% | Val Loss: 0.75225, Val Acc.: 75.10% | LR: 0.007601 Epoch 17/80 | Train Loss: 0.62458, Train Acc.: 80.40% | Val Loss: 0.65568, Val Acc.: 77.56% | LR: 0.008123 Epoch 18/80 | Train Loss: 0.61064, Train Acc.: 79.14% | Val Loss: 0.68725, Val Acc.: 77.38% | LR: 0.008595 Epoch 19/80 | Train Loss: 0.60154, Train Acc.: 83.49% | Val Loss: 0.57551, Val Acc.: 81.36% | LR: 0.009009 Epoch 20/80 | Train Loss: 0.58831, Train Acc.: 81.50% | Val Loss: 0.63472, Val Acc.: 78.34% | LR: 0.009358 Epoch 21/80 | Train Loss: 0.58013, Train Acc.: 83.83% | Val Loss: 0.57436, Val Acc.: 80.94% | LR: 0.009635 Epoch 22/80 | Train Loss: 0.58147, Train Acc.: 81.91% | Val Loss: 0.63800, Val Acc.: 79.26% | LR: 0.009837 Epoch 23/80 | Train Loss: 0.55888, Train Acc.: 79.51% | Val Loss: 0.66393, Val Acc.: 77.16% | LR: 0.009959 Epoch 24/80 | Train Loss: 0.55165, Train Acc.: 83.55% | Val Loss: 0.58955, Val Acc.: 80.14% | LR: 0.010000 Epoch 25/80 | Train Loss: 0.54598, Train Acc.: 80.25% | Val Loss: 0.68080, Val Acc.: 77.28% | LR: 0.009992 Epoch 26/80 | Train Loss: 0.53314, Train Acc.: 85.35% | Val Loss: 0.51544, Val Acc.: 82.44% | LR: 0.009968 Epoch 27/80 | Train Loss: 0.52842, Train Acc.: 85.56% | Val Loss: 0.52410, Val Acc.: 82.22% | LR: 0.009929 Epoch 28/80 | Train Loss: 0.51619, Train Acc.: 86.35% | Val Loss: 0.50836, Val Acc.: 82.08% | LR: 0.009874 Epoch 29/80 | Train Loss: 0.50116, Train Acc.: 83.90% | Val Loss: 0.58979, Val Acc.: 80.78% | LR: 0.009804 Epoch 30/80 | Train Loss: 0.49832, Train Acc.: 84.78% | Val Loss: 0.53527, Val Acc.: 81.96% | LR: 0.009719 Epoch 31/80 | Train Loss: 0.48072, Train Acc.: 86.78% | Val Loss: 0.51124, Val Acc.: 82.96% | LR: 0.009619 Epoch 32/80 | Train Loss: 0.48340, Train Acc.: 87.40% | Val Loss: 0.48391, Val Acc.: 83.54% | LR: 0.009505 Epoch 33/80 | Train Loss: 0.46901, Train Acc.: 86.78% | Val Loss: 0.49999, Val Acc.: 83.14% | LR: 0.009376 Epoch 34/80 | Train Loss: 0.46657, Train Acc.: 83.99% | Val Loss: 0.61034, Val Acc.: 80.50% | LR: 0.009233 Epoch 35/80 | Train Loss: 0.45237, Train Acc.: 85.50% | Val Loss: 0.55393, Val Acc.: 81.80% | LR: 0.009077 Epoch 36/80 | Train Loss: 0.44923, Train Acc.: 85.74% | Val Loss: 0.55699, Val Acc.: 81.56% | LR: 0.008909 Epoch 37/80 | Train Loss: 0.44516, Train Acc.: 87.81% | Val Loss: 0.47020, Val Acc.: 83.76% | LR: 0.008728 Epoch 38/80 | Train Loss: 0.43236, Train Acc.: 88.12% | Val Loss: 0.47663, Val Acc.: 84.34% | LR: 0.008535 Epoch 39/80 | Train Loss: 0.42329, Train Acc.: 89.62% | Val Loss: 0.43928, Val Acc.: 85.94% | LR: 0.008331 Epoch 40/80 | Train Loss: 0.41743, Train Acc.: 89.04% | Val Loss: 0.47179, Val Acc.: 84.84% | LR: 0.008117 Epoch 41/80 | Train Loss: 0.40293, Train Acc.: 89.56% | Val Loss: 0.45510, Val Acc.: 84.82% | LR: 0.007893 Epoch 42/80 | Train Loss: 0.40254, Train Acc.: 88.32% | Val Loss: 0.47881, Val Acc.: 84.06% | LR: 0.007659 Epoch 43/80 | Train Loss: 0.39263, Train Acc.: 89.94% | Val Loss: 0.43850, Val Acc.: 85.58% | LR: 0.007418 Epoch 44/80 | Train Loss: 0.38790, Train Acc.: 86.66% | Val Loss: 0.54527, Val Acc.: 82.42% | LR: 0.007169 Epoch 45/80 | Train Loss: 0.38006, Train Acc.: 88.98% | Val Loss: 0.48027, Val Acc.: 84.78% | LR: 0.006913 Epoch 46/80 | Train Loss: 0.36605, Train Acc.: 91.31% | Val Loss: 0.41927, Val Acc.: 86.34% | LR: 0.006651 Epoch 47/80 | Train Loss: 0.36067, Train Acc.: 90.71% | Val Loss: 0.42674, Val Acc.: 86.22% | LR: 0.006383 Epoch 48/80 | Train Loss: 0.35229, Train Acc.: 89.42% | Val Loss: 0.47246, Val Acc.: 84.06% | LR: 0.006112 Epoch 49/80 | Train Loss: 0.34320, Train Acc.: 91.89% | Val Loss: 0.40775, Val Acc.: 86.34% | LR: 0.005837 Epoch 50/80 | Train Loss: 0.33886, Train Acc.: 91.85% | Val Loss: 0.40924, Val Acc.: 85.70% | LR: 0.005559 Epoch 51/80 | Train Loss: 0.32865, Train Acc.: 91.84% | Val Loss: 0.39967, Val Acc.: 86.50% | LR: 0.005280 Epoch 52/80 | Train Loss: 0.31851, Train Acc.: 92.74% | Val Loss: 0.37164, Val Acc.: 87.48% | LR: 0.004999 Epoch 53/80 | Train Loss: 0.30973, Train Acc.: 93.26% | Val Loss: 0.37240, Val Acc.: 87.62% | LR: 0.004719 Epoch 54/80 | Train Loss: 0.29975, Train Acc.: 93.39% | Val Loss: 0.37586, Val Acc.: 88.22% | LR: 0.004439 Epoch 55/80 | Train Loss: 0.29632, Train Acc.: 93.88% | Val Loss: 0.36374, Val Acc.: 88.24% | LR: 0.004162 Epoch 56/80 | Train Loss: 0.28587, Train Acc.: 93.90% | Val Loss: 0.37280, Val Acc.: 87.80% | LR: 0.003887 Epoch 57/80 | Train Loss: 0.27816, Train Acc.: 94.65% | Val Loss: 0.35496, Val Acc.: 88.36% | LR: 0.003615 Epoch 58/80 | Train Loss: 0.26835, Train Acc.: 94.46% | Val Loss: 0.36689, Val Acc.: 88.10% | LR: 0.003348 Epoch 59/80 | Train Loss: 0.25625, Train Acc.: 94.76% | Val Loss: 0.35964, Val Acc.: 88.34% | LR: 0.003086 Epoch 60/80 | Train Loss: 0.24764, Train Acc.: 95.47% | Val Loss: 0.33960, Val Acc.: 88.72% | LR: 0.002830 Epoch 61/80 | Train Loss: 0.24207, Train Acc.: 95.73% | Val Loss: 0.34561, Val Acc.: 88.86% | LR: 0.002581 Epoch 62/80 | Train Loss: 0.23365, Train Acc.: 96.10% | Val Loss: 0.32637, Val Acc.: 89.52% | LR: 0.002339 Epoch 63/80 | Train Loss: 0.22667, Train Acc.: 95.77% | Val Loss: 0.34045, Val Acc.: 89.32% | LR: 0.002106 Epoch 64/80 | Train Loss: 0.21231, Train Acc.: 96.23% | Val Loss: 0.33263, Val Acc.: 89.52% | LR: 0.001882 Epoch 65/80 | Train Loss: 0.20584, Train Acc.: 96.60% | Val Loss: 0.33528, Val Acc.: 89.50% | LR: 0.001668 Epoch 66/80 | Train Loss: 0.20450, Train Acc.: 96.80% | Val Loss: 0.32995, Val Acc.: 89.34% | LR: 0.001464 Epoch 67/80 | Train Loss: 0.19413, Train Acc.: 96.74% | Val Loss: 0.32989, Val Acc.: 89.68% | LR: 0.001271 Epoch 68/80 | Train Loss: 0.19229, Train Acc.: 97.28% | Val Loss: 0.31275, Val Acc.: 89.84% | LR: 0.001090 Epoch 69/80 | Train Loss: 0.18385, Train Acc.: 97.20% | Val Loss: 0.32126, Val Acc.: 89.80% | LR: 0.000922 Epoch 70/80 | Train Loss: 0.17799, Train Acc.: 97.38% | Val Loss: 0.32366, Val Acc.: 89.72% | LR: 0.000766 Epoch 71/80 | Train Loss: 0.17443, Train Acc.: 97.57% | Val Loss: 0.31418, Val Acc.: 90.84% | LR: 0.000624 Epoch 72/80 | Train Loss: 0.16518, Train Acc.: 97.51% | Val Loss: 0.31764, Val Acc.: 90.14% | LR: 0.000495 Epoch 73/80 | Train Loss: 0.16197, Train Acc.: 97.80% | Val Loss: 0.30111, Val Acc.: 90.60% | LR: 0.000380 Epoch 74/80 | Train Loss: 0.16284, Train Acc.: 97.78% | Val Loss: 0.31148, Val Acc.: 90.22% | LR: 0.000280 Epoch 75/80 | Train Loss: 0.16011, Train Acc.: 97.78% | Val Loss: 0.31743, Val Acc.: 90.36% | LR: 0.000195 Epoch 76/80 | Train Loss: 0.16076, Train Acc.: 97.90% | Val Loss: 0.31457, Val Acc.: 90.22% | LR: 0.000125 Epoch 77/80 | Train Loss: 0.15830, Train Acc.: 97.88% | Val Loss: 0.32026, Val Acc.: 89.94% | LR: 0.000071 Epoch 78/80 | Train Loss: 0.15811, Train Acc.: 97.87% | Val Loss: 0.32391, Val Acc.: 90.44% | LR: 0.000031 Epoch 79/80 | Train Loss: 0.15659, Train Acc.: 97.86% | Val Loss: 0.30832, Val Acc.: 90.36% | LR: 0.000008 Epoch 80/80 | Train Loss: 0.15498, Train Acc.: 97.93% | Val Loss: 0.30580, Val Acc.: 89.92% | LR: 0.000000 Total Training Time: 35.19 min Test Accuracy: 90.43%
In [25]:
def plot_history(history):
"""
Plots the training and validation loss, accuracy from a PyTorch history dictionary.
Args:
history (dict): Dictionary containing training metrics.
Expected keys: 'train_loss', 'val_loss', 'train_acc', 'val_acc'.
"""
epochs = range(len(history['train_loss']))
plt.figure(figsize=(18, 5))
# Plot 1: Loss
plt.subplot(1, 3, 1)
plt.plot(epochs, history['train_loss'], label='Training Loss')
plt.plot(epochs, history['val_loss'], label='Validation Loss')
plt.title('Loss')
plt.xlabel('Epochs')
plt.ylabel('Loss')
plt.legend()
# plt.grid(True)
# Plot 2: Accuracy
plt.subplot(1, 3, 2)
plt.plot(epochs, history['train_acc'], label='Training Accuracy')
plt.plot(epochs, history['val_acc'], label='Validation Accuracy')
plt.title('Accuracy')
plt.xlabel('Epochs')
plt.ylabel('Accuracy')
plt.legend()
# plt.grid(True)
plt.tight_layout()
plt.show()
# plt.savefig()
plot_history(history)
In [26]:
%watermark -iv
matplotlib : 3.10.0 numpy : 2.0.2 pandas : 2.2.2 sklearn : 1.6.1 torch : 2.8.0+cu126 torchinfo : 1.8.0 torchvision: 0.23.0+cu126
In [ ]: