▶ video

Deep Learning & PyTorch

Difficulty: PhDRead Time: ~15 min

Lecture Notes

## Introduction to Neural Networks Deep learning is a subset of machine learning based on artificial neural networks with representation learning. The adjective "deep" in deep learning refers to the use of multiple layers in the network. ## Why PyTorch? PyTorch is an open source machine learning framework that accelerates the path from research prototyping to production deployment. Unlike TensorFlow's static computation graphs (historically), PyTorch uses **Dynamic Computation Graphs** (Define-by-Run), making debugging significantly easier. ### Tensors The core data structure in PyTorch is the `Tensor`. It is similar to a NumPy array but has added superpowers: it can run on GPUs to accelerate computing. ```python import torch import torch.nn as nn # Create a random tensor x = torch.rand(5, 3) print(x) # Move tensor to GPU if available device = torch.device("cuda" if torch.cuda.is_available() else "cpu") x = x.to(device) ``` ### Building a Neural Network In PyTorch, you define neural networks by subclassing `nn.Module`. ```python class SimpleNN(nn.Module): def __init__(self): super(SimpleNN, self).__init__() # Input layer to hidden layer (e.g., 784 inputs -> 128 hidden) self.fc1 = nn.Linear(784, 128) # Activation function self.relu = nn.ReLU() # Hidden layer to output layer (e.g., 10 classes) self.fc2 = nn.Linear(128, 10) def forward(self, x): out = self.fc1(x) out = self.relu(out) out = self.fc2(out) return out model = SimpleNN().to(device) print(model) ``` ## Backpropagation & Optimization To train this network, PyTorch provides `Autograd`, an automatic differentiation engine that calculates gradients for all neural network parameters. ```python criterion = nn.CrossEntropyLoss() optimizer = torch.optim.Adam(model.parameters(), lr=0.001) # Inside the training loop: optimizer.zero_grad() # 1. Clear previous gradients outputs = model(inputs) # 2. Forward pass loss = criterion(outputs, labels) # 3. Compute loss loss.backward() # 4. Backward pass (calculate gradients) optimizer.step() # 5. Update weights ``` Deep learning forms the foundation of modern [Natural Language Processing](/courses/advanced-data-science-and-ai/1) and Generative AI.