{ "cells": [ { "cell_type": "markdown", "metadata": {}, "source": [ "# PyTorch Tutorial - Deep learning framework\n", "\n", "### EE-556: Mathematics of Data: From Theory to Computation. Instructor: Prof. Volkan Cevher\n", "\n", "\n", "This tutorial introduces the basics for working with deep learning methods in PyTorch.\n", "\n", "## Defining a neural network architecture\n", "\n", "In PyTorch it is easy to define a network architecture. We only have to define a subclass of the `torch.nn.Module` class and implement a forward method that defines the output of the network given an input. We can reuse the components in the `torch.nn` submodule and the functions in the `torch.nn.functional` submodule. The following example shows how to define a two-layer fully connected network with ReLU activations." ] }, { "cell_type": "code", "execution_count": 1, "metadata": {}, "outputs": [], "source": [ "import torch\n", "import torch.nn.functional as F\n", "from torch import nn\n", "\n", "class TwoLayerNet(nn.Module):\n", " def __init__(self, input_dim, hidden_dim, output_dim):\n", " \"\"\"\n", " Create two nn.Linear objects and assign them as attributes\n", " :param input_dim: dimension of the input\n", " :param hidden_dim: number of hidden neurons\n", " :param output_dim: dimension of the output\n", " \"\"\"\n", " super().__init__()\n", " self.linear1 = nn.Linear(input_dim, hidden_dim)\n", " self.linear2 = nn.Linear(hidden_dim, output_dim)\n", "\n", " def forward(self, x):\n", " \"\"\"\n", " In the forward method we define what is the output of the network\n", " given an input x. In this example we use the ReLU as our activation function\n", " \"\"\"\n", " x = F.relu(self.linear1(x))\n", " x = self.linear2(x)\n", " return x" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "Now we can create an instance of the architecture we just defined, and compute its output on some tensor. Notice that we can pass different values for the input dimension, number of hidden neurons and output dimension when creating an instance of the class." ] }, { "cell_type": "code", "execution_count": 2, "metadata": {}, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "Output of the network at input x: tensor([ 0.0964, -0.0889, 0.0039, 0.0765, 0.4046, -0.3576, -0.0808, -0.2414,\n", " -0.0047, 0.1155], grad_fn=)\n" ] } ], "source": [ "net = TwoLayerNet(input_dim=784, hidden_dim=100, output_dim=10)\n", "x = torch.randn(784)\n", "result = net(x)\n", "print('Output of the network at input x:', result)\n" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "\n", "## Accessing the parameters of the layers\n", "\n", "Most often we want to directly modify the parameters of a network. Depending on the type of modules you use to define your network, this procedure might be different. We will show how to clip the values of the weight and biases of linear layers in the following example:\n" ] }, { "cell_type": "code", "execution_count": 3, "metadata": {}, "outputs": [], "source": [ "\n", "net = TwoLayerNet(input_dim=784, hidden_dim=100, output_dim=10)\n", "net.linear1.weight.data = torch.clip(net.linear1.weight.data, min=-1., max=1.)\n", "net.linear1.bias.data = torch.clip(net.linear1.bias.data, min=-1., max=1.)\n", "net.linear2.weight.data = torch.clip(net.linear2.weight.data, min=-1., max=1.)\n", "net.linear2.bias.data = torch.clip(net.linear2.bias.data, min=-1., max=1.)" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "In this example, the linear layers have two attributes weight and bias where the actual tensors are stored. Note that we modify the data attribute which stores the raw values of the tensor, so that this operation is not taken into account when computing derivatives. Recall that PyTorch keeps track of all operations involving tensors that have the attribute `requires_grad` set to True.\n" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "## Loading the training data\n", "\n", "PyTorch implements several classes that make it easy to work with large datasets. Also, the `torchvision` module lets us download popular benchmark datasets. The following example shows how to load the MNIST dataset, and create two instances of `torch.utils.data.DataLoader`, which is an object that lets us iterate through the dataset in batches. Because MNIST consists of image-label pairs, each batch is made of a tensor containing the images, and a tensor containing the corresponding labels.\n" ] }, { "cell_type": "code", "execution_count": 4, "metadata": {}, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "Batch size: 100\n", "Input dimension: torch.Size([1, 28, 28])\n", "Batch size: 100\n", "Input dimension: torch.Size([1, 28, 28])\n", "Batch size: 100\n", "Input dimension: torch.Size([1, 28, 28])\n", "Batch size: 100\n", "Input dimension: torch.Size([1, 28, 28])\n", "Batch size: 100\n", "Input dimension: torch.Size([1, 28, 28])\n", "Batch size: 100\n", "Input dimension: torch.Size([1, 28, 28])\n", "Batch size: 100\n", "Input dimension: torch.Size([1, 28, 28])\n", "Batch size: 100\n", "Input dimension: torch.Size([1, 28, 28])\n", "Batch size: 100\n", "Input dimension: torch.Size([1, 28, 28])\n", "Batch size: 100\n", "Input dimension: torch.Size([1, 28, 28])\n", "Batch size: 100\n", "Input dimension: torch.Size([1, 28, 28])\n", "Batch size: 100\n", "Input dimension: torch.Size([1, 28, 28])\n", "Batch size: 100\n", "Input dimension: torch.Size([1, 28, 28])\n", "Batch size: 100\n", "Input dimension: torch.Size([1, 28, 28])\n", "Batch size: 100\n", "Input dimension: torch.Size([1, 28, 28])\n", "Batch size: 100\n", "Input dimension: torch.Size([1, 28, 28])\n", "Batch size: 100\n", "Input dimension: torch.Size([1, 28, 28])\n", "Batch size: 100\n", "Input dimension: torch.Size([1, 28, 28])\n", "Batch size: 100\n", "Input dimension: torch.Size([1, 28, 28])\n", "Batch size: 100\n", "Input dimension: torch.Size([1, 28, 28])\n", "Batch size: 100\n", "Input dimension: torch.Size([1, 28, 28])\n", "Batch size: 100\n", "Input dimension: torch.Size([1, 28, 28])\n", "Batch size: 100\n", "Input dimension: torch.Size([1, 28, 28])\n", "Batch size: 100\n", "Input dimension: torch.Size([1, 28, 28])\n", "Batch size: 100\n", "Input dimension: torch.Size([1, 28, 28])\n", "Batch size: 100\n", "Input dimension: torch.Size([1, 28, 28])\n", "Batch size: 100\n", "Input dimension: torch.Size([1, 28, 28])\n", "Batch size: 100\n", "Input dimension: torch.Size([1, 28, 28])\n", "Batch size: 100\n", "Input dimension: torch.Size([1, 28, 28])\n", "Batch size: 100\n", "Input dimension: torch.Size([1, 28, 28])\n", "Batch size: 100\n", "Input dimension: torch.Size([1, 28, 28])\n", "Batch size: 100\n", "Input dimension: torch.Size([1, 28, 28])\n", "Batch size: 100\n", "Input dimension: torch.Size([1, 28, 28])\n", "Batch size: 100\n", "Input dimension: torch.Size([1, 28, 28])\n", "Batch size: 100\n", "Input dimension: torch.Size([1, 28, 28])\n", "Batch size: 100\n", "Input dimension: torch.Size([1, 28, 28])\n", "Batch size: 100\n", "Input dimension: torch.Size([1, 28, 28])\n", "Batch size: 100\n", "Input dimension: torch.Size([1, 28, 28])\n", "Batch size: 100\n", "Input dimension: torch.Size([1, 28, 28])\n", "Batch size: 100\n", "Input dimension: torch.Size([1, 28, 28])\n", "Batch size: 100\n", "Input dimension: torch.Size([1, 28, 28])\n", "Batch size: 100\n", "Input dimension: torch.Size([1, 28, 28])\n", "Batch size: 100\n", "Input dimension: torch.Size([1, 28, 28])\n", "Batch size: 100\n", "Input dimension: torch.Size([1, 28, 28])\n", "Batch size: 100\n", "Input dimension: torch.Size([1, 28, 28])\n", "Batch size: 100\n", "Input dimension: torch.Size([1, 28, 28])\n", "Batch size: 100\n", "Input dimension: torch.Size([1, 28, 28])\n", "Batch size: 100\n", "Input dimension: torch.Size([1, 28, 28])\n", "Batch size: 100\n", "Input dimension: torch.Size([1, 28, 28])\n", "Batch size: 100\n", "Input dimension: torch.Size([1, 28, 28])\n", "Batch size: 100\n", "Input dimension: torch.Size([1, 28, 28])\n", "Batch size: 100\n", "Input dimension: torch.Size([1, 28, 28])\n", "Batch size: 100\n", "Input dimension: torch.Size([1, 28, 28])\n", "Batch size: 100\n", "Input dimension: torch.Size([1, 28, 28])\n", "Batch size: 100\n", "Input dimension: torch.Size([1, 28, 28])\n", "Batch size: 100\n", "Input dimension: torch.Size([1, 28, 28])\n", "Batch size: 100\n", "Input dimension: torch.Size([1, 28, 28])\n", "Batch size: 100\n", "Input dimension: torch.Size([1, 28, 28])\n", "Batch size: 100\n", "Input dimension: torch.Size([1, 28, 28])\n", "Batch size: 100\n", "Input dimension: torch.Size([1, 28, 28])\n", "Batch size: 100\n", "Input dimension: torch.Size([1, 28, 28])\n", "Batch size: 100\n", "Input dimension: torch.Size([1, 28, 28])\n", "Batch size: 100\n", "Input dimension: torch.Size([1, 28, 28])\n", "Batch size: 100\n", "Input dimension: torch.Size([1, 28, 28])\n", "Batch size: 100\n", "Input dimension: torch.Size([1, 28, 28])\n", "Batch size: 100\n", "Input dimension: torch.Size([1, 28, 28])\n", "Batch size: 100\n", "Input dimension: torch.Size([1, 28, 28])\n", "Batch size: 100\n", "Input dimension: torch.Size([1, 28, 28])\n", "Batch size: 100\n", "Input dimension: torch.Size([1, 28, 28])\n", "Batch size: 100\n", "Input dimension: torch.Size([1, 28, 28])\n", "Batch size: 100\n", "Input dimension: torch.Size([1, 28, 28])\n", "Batch size: 100\n", "Input dimension: torch.Size([1, 28, 28])\n", "Batch size: 100\n", "Input dimension: torch.Size([1, 28, 28])\n", "Batch size: 100\n", "Input dimension: torch.Size([1, 28, 28])\n", "Batch size: 100\n", "Input dimension: torch.Size([1, 28, 28])\n", "Batch size: 100\n", "Input dimension: torch.Size([1, 28, 28])\n", "Batch size: 100\n", "Input dimension: torch.Size([1, 28, 28])\n", "Batch size: 100\n", "Input dimension: torch.Size([1, 28, 28])\n", "Batch size: 100\n", "Input dimension: torch.Size([1, 28, 28])\n", "Batch size: 100\n", "Input dimension: torch.Size([1, 28, 28])\n", "Batch size: 100\n", "Input dimension: torch.Size([1, 28, 28])\n", "Batch size: 100\n", "Input dimension: torch.Size([1, 28, 28])\n", "Batch size: 100\n", "Input dimension: torch.Size([1, 28, 28])\n", "Batch size: 100\n", "Input dimension: torch.Size([1, 28, 28])\n", "Batch size: 100\n", "Input dimension: torch.Size([1, 28, 28])\n", "Batch size: 100\n", "Input dimension: torch.Size([1, 28, 28])\n", "Batch size: 100\n", "Input dimension: torch.Size([1, 28, 28])\n", "Batch size: 100\n", "Input dimension: torch.Size([1, 28, 28])\n", "Batch size: 100\n", "Input dimension: torch.Size([1, 28, 28])\n", "Batch size: 100\n", "Input dimension: torch.Size([1, 28, 28])\n", "Batch size: 100\n", "Input dimension: torch.Size([1, 28, 28])\n", "Batch size: 100\n", "Input dimension: torch.Size([1, 28, 28])\n", "Batch size: 100\n", "Input dimension: torch.Size([1, 28, 28])\n", "Batch size: 100\n", "Input dimension: torch.Size([1, 28, 28])\n", "Batch size: 100\n", "Input dimension: torch.Size([1, 28, 28])\n", "Batch size: 100\n", "Input dimension: torch.Size([1, 28, 28])\n", "Batch size: 100\n", "Input dimension: torch.Size([1, 28, 28])\n", "Batch size: 100\n", "Input dimension: torch.Size([1, 28, 28])\n", "Batch size: 100\n", "Input dimension: torch.Size([1, 28, 28])\n", "Batch size: 100\n", "Input dimension: torch.Size([1, 28, 28])\n", "Batch size: 100\n", "Input dimension: torch.Size([1, 28, 28])\n", "Batch size: 100\n", "Input dimension: torch.Size([1, 28, 28])\n", "Batch size: 100\n", "Input dimension: torch.Size([1, 28, 28])\n", "Batch size: 100\n", "Input dimension: torch.Size([1, 28, 28])\n", "Batch size: 100\n", "Input dimension: torch.Size([1, 28, 28])\n", "Batch size: 100\n", "Input dimension: torch.Size([1, 28, 28])\n", "Batch size: 100\n", "Input dimension: torch.Size([1, 28, 28])\n", "Batch size: 100\n", "Input dimension: torch.Size([1, 28, 28])\n", "Batch size: 100\n", "Input dimension: torch.Size([1, 28, 28])\n", "Batch size: 100\n", "Input dimension: torch.Size([1, 28, 28])\n", "Batch size: 100\n", "Input dimension: torch.Size([1, 28, 28])\n", "Batch size: 100\n", "Input dimension: torch.Size([1, 28, 28])\n", "Batch size: 100\n", "Input dimension: torch.Size([1, 28, 28])\n", "Batch size: 100\n", "Input dimension: torch.Size([1, 28, 28])\n", "Batch size: 100\n", "Input dimension: torch.Size([1, 28, 28])\n", "Batch size: 100\n", "Input dimension: torch.Size([1, 28, 28])\n", "Batch size: 100\n", "Input dimension: torch.Size([1, 28, 28])\n", "Batch size: 100\n", "Input dimension: torch.Size([1, 28, 28])\n", "Batch size: 100\n", "Input dimension: torch.Size([1, 28, 28])\n", "Batch size: 100\n", "Input dimension: torch.Size([1, 28, 28])\n", "Batch size: 100\n", "Input dimension: torch.Size([1, 28, 28])\n", "Batch size: 100\n", "Input dimension: torch.Size([1, 28, 28])\n", "Batch size: 100\n", "Input dimension: torch.Size([1, 28, 28])\n", "Batch size: 100\n", "Input dimension: torch.Size([1, 28, 28])\n", "Batch size: 100\n", "Input dimension: torch.Size([1, 28, 28])\n", "Batch size: 100\n", "Input dimension: torch.Size([1, 28, 28])\n", "Batch size: 100\n", "Input dimension: torch.Size([1, 28, 28])\n", "Batch size: 100\n", "Input dimension: torch.Size([1, 28, 28])\n", "Batch size: 100\n", "Input dimension: torch.Size([1, 28, 28])\n", "Batch size: 100\n", "Input dimension: torch.Size([1, 28, 28])\n", "Batch size: 100\n", "Input dimension: torch.Size([1, 28, 28])\n", "Batch size: 100\n", "Input dimension: torch.Size([1, 28, 28])\n", "Batch size: 100\n", "Input dimension: torch.Size([1, 28, 28])\n", "Batch size: 100\n", "Input dimension: torch.Size([1, 28, 28])\n", "Batch size: 100\n", "Input dimension: torch.Size([1, 28, 28])\n", "Batch size: 100\n", "Input dimension: torch.Size([1, 28, 28])\n", "Batch size: 100\n", "Input dimension: torch.Size([1, 28, 28])\n", "Batch size: 100\n", "Input dimension: torch.Size([1, 28, 28])\n", "Batch size: 100\n", "Input dimension: torch.Size([1, 28, 28])\n", "Batch size: 100\n", "Input dimension: torch.Size([1, 28, 28])\n", "Batch size: 100\n", "Input dimension: torch.Size([1, 28, 28])\n", "Batch size: 100\n", "Input dimension: torch.Size([1, 28, 28])\n", "Batch size: 100\n", "Input dimension: torch.Size([1, 28, 28])\n", "Batch size: 100\n", "Input dimension: torch.Size([1, 28, 28])\n", "Batch size: 100\n", "Input dimension: torch.Size([1, 28, 28])\n", "Batch size: 100\n", "Input dimension: torch.Size([1, 28, 28])\n", "Batch size: 100\n", "Input dimension: torch.Size([1, 28, 28])\n", "Batch size: 100\n", "Input dimension: torch.Size([1, 28, 28])\n", "Batch size: 100\n", "Input dimension: torch.Size([1, 28, 28])\n", "Batch size: 100\n", "Input dimension: torch.Size([1, 28, 28])\n", "Batch size: 100\n", "Input dimension: torch.Size([1, 28, 28])\n", "Batch size: 100\n", "Input dimension: torch.Size([1, 28, 28])\n", "Batch size: 100\n", "Input dimension: torch.Size([1, 28, 28])\n", "Batch size: 100\n", "Input dimension: torch.Size([1, 28, 28])\n", "Batch size: 100\n", "Input dimension: torch.Size([1, 28, 28])\n", "Batch size: 100\n", "Input dimension: torch.Size([1, 28, 28])\n", "Batch size: 100\n", "Input dimension: torch.Size([1, 28, 28])\n", "Batch size: 100\n", "Input dimension: torch.Size([1, 28, 28])\n", "Batch size: 100\n", "Input dimension: torch.Size([1, 28, 28])\n", "Batch size: 100\n", "Input dimension: torch.Size([1, 28, 28])\n", "Batch size: 100\n", "Input dimension: torch.Size([1, 28, 28])\n", "Batch size: 100\n", "Input dimension: torch.Size([1, 28, 28])\n", "Batch size: 100\n", "Input dimension: torch.Size([1, 28, 28])\n", "Batch size: 100\n", "Input dimension: torch.Size([1, 28, 28])\n", "Batch size: 100\n", "Input dimension: torch.Size([1, 28, 28])\n", "Batch size: 100\n", "Input dimension: torch.Size([1, 28, 28])\n", "Batch size: 100\n", "Input dimension: torch.Size([1, 28, 28])\n", "Batch size: 100\n", "Input dimension: torch.Size([1, 28, 28])\n", "Batch size: 100\n", "Input dimension: torch.Size([1, 28, 28])\n", "Batch size: 100\n", "Input dimension: torch.Size([1, 28, 28])\n", "Batch size: 100\n", "Input dimension: torch.Size([1, 28, 28])\n", "Batch size: 100\n", "Input dimension: torch.Size([1, 28, 28])\n", "Batch size: 100\n", "Input dimension: torch.Size([1, 28, 28])\n", "Batch size: 100\n", "Input dimension: torch.Size([1, 28, 28])\n", "Batch size: 100\n", "Input dimension: torch.Size([1, 28, 28])\n", "Batch size: 100\n", "Input dimension: torch.Size([1, 28, 28])\n", "Batch size: 100\n", "Input dimension: torch.Size([1, 28, 28])\n", "Batch size: 100\n", "Input dimension: torch.Size([1, 28, 28])\n", "Batch size: 100\n", "Input dimension: torch.Size([1, 28, 28])\n", "Batch size: 100\n", "Input dimension: torch.Size([1, 28, 28])\n", "Batch size: 100\n", "Input dimension: torch.Size([1, 28, 28])\n", "Batch size: 100\n", "Input dimension: torch.Size([1, 28, 28])\n", "Batch size: 100\n", "Input dimension: torch.Size([1, 28, 28])\n", "Batch size: 100\n", "Input dimension: torch.Size([1, 28, 28])\n", "Batch size: 100\n", "Input dimension: torch.Size([1, 28, 28])\n", "Batch size: 100\n", "Input dimension: torch.Size([1, 28, 28])\n", "Batch size: 100\n", "Input dimension: torch.Size([1, 28, 28])\n", "Batch size: 100\n", "Input dimension: torch.Size([1, 28, 28])\n", "Batch size: 100\n", "Input dimension: torch.Size([1, 28, 28])\n", "Batch size: 100\n", "Input dimension: torch.Size([1, 28, 28])\n", "Batch size: 100\n", "Input dimension: torch.Size([1, 28, 28])\n", "Batch size: 100\n", "Input dimension: torch.Size([1, 28, 28])\n", "Batch size: 100\n", "Input dimension: torch.Size([1, 28, 28])\n", "Batch size: 100\n", "Input dimension: torch.Size([1, 28, 28])\n", "Batch size: 100\n", "Input dimension: torch.Size([1, 28, 28])\n", "Batch size: 100\n", "Input dimension: torch.Size([1, 28, 28])\n", "Batch size: 100\n", "Input dimension: torch.Size([1, 28, 28])\n", "Batch size: 100\n", "Input dimension: torch.Size([1, 28, 28])\n", "Batch size: 100\n", "Input dimension: torch.Size([1, 28, 28])\n", "Batch size: 100\n", "Input dimension: torch.Size([1, 28, 28])\n", "Batch size: 100\n", "Input dimension: torch.Size([1, 28, 28])\n", "Batch size: 100\n", "Input dimension: torch.Size([1, 28, 28])\n", "Batch size: 100\n", "Input dimension: torch.Size([1, 28, 28])\n", "Batch size: 100\n", "Input dimension: torch.Size([1, 28, 28])\n", "Batch size: 100\n", "Input dimension: torch.Size([1, 28, 28])\n", "Batch size: 100\n", "Input dimension: torch.Size([1, 28, 28])\n", "Batch size: 100\n", "Input dimension: torch.Size([1, 28, 28])\n", "Batch size: 100\n", "Input dimension: torch.Size([1, 28, 28])\n", "Batch size: 100\n", "Input dimension: torch.Size([1, 28, 28])\n", "Batch size: 100\n", "Input dimension: torch.Size([1, 28, 28])\n", "Batch size: 100\n", "Input dimension: torch.Size([1, 28, 28])\n", "Batch size: 100\n", "Input dimension: torch.Size([1, 28, 28])\n", "Batch size: 100\n", "Input dimension: torch.Size([1, 28, 28])\n", "Batch size: 100\n", "Input dimension: torch.Size([1, 28, 28])\n", "Batch size: 100\n", "Input dimension: torch.Size([1, 28, 28])\n", "Batch size: 100\n", "Input dimension: torch.Size([1, 28, 28])\n", "Batch size: 100\n", "Input dimension: torch.Size([1, 28, 28])\n", "Batch size: 100\n", "Input dimension: torch.Size([1, 28, 28])\n", "Batch size: 100\n", "Input dimension: torch.Size([1, 28, 28])\n", "Batch size: 100\n", "Input dimension: torch.Size([1, 28, 28])\n", "Batch size: 100\n", "Input dimension: torch.Size([1, 28, 28])\n", "Batch size: 100\n", "Input dimension: torch.Size([1, 28, 28])\n", "Batch size: 100\n", "Input dimension: torch.Size([1, 28, 28])\n", "Batch size: 100\n", "Input dimension: torch.Size([1, 28, 28])\n", "Batch size: 100\n", "Input dimension: torch.Size([1, 28, 28])\n", "Batch size: 100\n", "Input dimension: torch.Size([1, 28, 28])\n", "Batch size: 100\n", "Input dimension: torch.Size([1, 28, 28])\n", "Batch size: 100\n", "Input dimension: torch.Size([1, 28, 28])\n", "Batch size: 100\n", "Input dimension: torch.Size([1, 28, 28])\n", "Batch size: 100\n", "Input dimension: torch.Size([1, 28, 28])\n", "Batch size: 100\n", "Input dimension: torch.Size([1, 28, 28])\n", "Batch size: 100\n", "Input dimension: torch.Size([1, 28, 28])\n", "Batch size: 100\n", "Input dimension: torch.Size([1, 28, 28])\n", "Batch size: 100\n", "Input dimension: torch.Size([1, 28, 28])\n", "Batch size: 100\n", "Input dimension: torch.Size([1, 28, 28])\n", "Batch size: 100\n", "Input dimension: torch.Size([1, 28, 28])\n", "Batch size: 100\n", "Input dimension: torch.Size([1, 28, 28])\n", "Batch size: 100\n", "Input dimension: torch.Size([1, 28, 28])\n", "Batch size: 100\n", "Input dimension: torch.Size([1, 28, 28])\n", "Batch size: 100\n", "Input dimension: torch.Size([1, 28, 28])\n", "Batch size: 100\n", "Input dimension: torch.Size([1, 28, 28])\n", "Batch size: 100\n", "Input dimension: torch.Size([1, 28, 28])\n", "Batch size: 100\n", "Input dimension: torch.Size([1, 28, 28])\n", "Batch size: 100\n", "Input dimension: torch.Size([1, 28, 28])\n", "Batch size: 100\n", "Input dimension: torch.Size([1, 28, 28])\n", "Batch size: 100\n", "Input dimension: torch.Size([1, 28, 28])\n", "Batch size: 100\n", "Input dimension: torch.Size([1, 28, 28])\n", "Batch size: 100\n", "Input dimension: torch.Size([1, 28, 28])\n", "Batch size: 100\n", "Input dimension: torch.Size([1, 28, 28])\n", "Batch size: 100\n", "Input dimension: torch.Size([1, 28, 28])\n", "Batch size: 100\n", "Input dimension: torch.Size([1, 28, 28])\n", "Batch size: 100\n", "Input dimension: torch.Size([1, 28, 28])\n", "Batch size: 100\n", "Input dimension: torch.Size([1, 28, 28])\n", "Batch size: 100\n", "Input dimension: torch.Size([1, 28, 28])\n", "Batch size: 100\n", "Input dimension: torch.Size([1, 28, 28])\n", "Batch size: 100\n", "Input dimension: torch.Size([1, 28, 28])\n", "Batch size: 100\n", "Input dimension: torch.Size([1, 28, 28])\n", "Batch size: 100\n", "Input dimension: torch.Size([1, 28, 28])\n", "Batch size: 100\n", "Input dimension: torch.Size([1, 28, 28])\n", "Batch size: 100\n", "Input dimension: torch.Size([1, 28, 28])\n", "Batch size: 100\n", "Input dimension: torch.Size([1, 28, 28])\n", "Batch size: 100\n", "Input dimension: torch.Size([1, 28, 28])\n", "Batch size: 100\n", "Input dimension: torch.Size([1, 28, 28])\n", "Batch size: 100\n", "Input dimension: torch.Size([1, 28, 28])\n", "Batch size: 100\n", "Input dimension: torch.Size([1, 28, 28])\n", "Batch size: 100\n", "Input dimension: torch.Size([1, 28, 28])\n", "Batch size: 100\n", "Input dimension: torch.Size([1, 28, 28])\n", "Batch size: 100\n", "Input dimension: torch.Size([1, 28, 28])\n", "Batch size: 100\n", "Input dimension: torch.Size([1, 28, 28])\n", "Batch size: 100\n", "Input dimension: torch.Size([1, 28, 28])\n", "Batch size: 100\n", "Input dimension: torch.Size([1, 28, 28])\n", "Batch size: 100\n", "Input dimension: torch.Size([1, 28, 28])\n", "Batch size: 100\n", "Input dimension: torch.Size([1, 28, 28])\n", "Batch size: 100\n", "Input dimension: torch.Size([1, 28, 28])\n", "Batch size: 100\n", "Input dimension: torch.Size([1, 28, 28])\n", "Batch size: 100\n", "Input dimension: torch.Size([1, 28, 28])\n", "Batch size: 100\n", "Input dimension: torch.Size([1, 28, 28])\n", "Batch size: 100\n", "Input dimension: torch.Size([1, 28, 28])\n", "Batch size: 100\n", "Input dimension: torch.Size([1, 28, 28])\n", "Batch size: 100\n", "Input dimension: torch.Size([1, 28, 28])\n", "Batch size: 100\n", "Input dimension: torch.Size([1, 28, 28])\n", "Batch size: 100\n", "Input dimension: torch.Size([1, 28, 28])\n", "Batch size: 100\n", "Input dimension: torch.Size([1, 28, 28])\n", "Batch size: 100\n", "Input dimension: torch.Size([1, 28, 28])\n", "Batch size: 100\n", "Input dimension: torch.Size([1, 28, 28])\n", "Batch size: 100\n", "Input dimension: torch.Size([1, 28, 28])\n", "Batch size: 100\n", "Input dimension: torch.Size([1, 28, 28])\n", "Batch size: 100\n", "Input dimension: torch.Size([1, 28, 28])\n", "Batch size: 100\n", "Input dimension: torch.Size([1, 28, 28])\n", "Batch size: 100\n", "Input dimension: torch.Size([1, 28, 28])\n", "Batch size: 100\n", "Input dimension: torch.Size([1, 28, 28])\n", "Batch size: 100\n", "Input dimension: torch.Size([1, 28, 28])\n", "Batch size: 100\n", "Input dimension: torch.Size([1, 28, 28])\n", "Batch size: 100\n", "Input dimension: torch.Size([1, 28, 28])\n", "Batch size: 100\n", "Input dimension: torch.Size([1, 28, 28])\n", "Batch size: 100\n", "Input dimension: torch.Size([1, 28, 28])\n", "Batch size: 100\n", "Input dimension: torch.Size([1, 28, 28])\n", "Batch size: 100\n", "Input dimension: torch.Size([1, 28, 28])\n", "Batch size: 100\n", "Input dimension: torch.Size([1, 28, 28])\n", "Batch size: 100\n", "Input dimension: torch.Size([1, 28, 28])\n", "Batch size: 100\n", "Input dimension: torch.Size([1, 28, 28])\n", "Batch size: 100\n", "Input dimension: torch.Size([1, 28, 28])\n", "Batch size: 100\n", "Input dimension: torch.Size([1, 28, 28])\n", "Batch size: 100\n", "Input dimension: torch.Size([1, 28, 28])\n", "Batch size: 100\n", "Input dimension: torch.Size([1, 28, 28])\n", "Batch size: 100\n", "Input dimension: torch.Size([1, 28, 28])\n", "Batch size: 100\n", "Input dimension: torch.Size([1, 28, 28])\n", "Batch size: 100\n", "Input dimension: torch.Size([1, 28, 28])\n", "Batch size: 100\n", "Input dimension: torch.Size([1, 28, 28])\n", "Batch size: 100\n", "Input dimension: torch.Size([1, 28, 28])\n", "Batch size: 100\n", "Input dimension: torch.Size([1, 28, 28])\n", "Batch size: 100\n", "Input dimension: torch.Size([1, 28, 28])\n", "Batch size: 100\n", "Input dimension: torch.Size([1, 28, 28])\n", "Batch size: 100\n", "Input dimension: torch.Size([1, 28, 28])\n", "Batch size: 100\n", "Input dimension: torch.Size([1, 28, 28])\n", "Batch size: 100\n", "Input dimension: torch.Size([1, 28, 28])\n", "Batch size: 100\n", "Input dimension: torch.Size([1, 28, 28])\n", "Batch size: 100\n", "Input dimension: torch.Size([1, 28, 28])\n", "Batch size: 100\n", "Input dimension: torch.Size([1, 28, 28])\n", "Batch size: 100\n", "Input dimension: torch.Size([1, 28, 28])\n", "Batch size: 100\n", "Input dimension: torch.Size([1, 28, 28])\n", "Batch size: 100\n", "Input dimension: torch.Size([1, 28, 28])\n", "Batch size: 100\n", "Input dimension: torch.Size([1, 28, 28])\n", "Batch size: 100\n", "Input dimension: torch.Size([1, 28, 28])\n", "Batch size: 100\n", "Input dimension: torch.Size([1, 28, 28])\n", "Batch size: 100\n", "Input dimension: torch.Size([1, 28, 28])\n", "Batch size: 100\n", "Input dimension: torch.Size([1, 28, 28])\n", "Batch size: 100\n", "Input dimension: torch.Size([1, 28, 28])\n", "Batch size: 100\n", "Input dimension: torch.Size([1, 28, 28])\n", "Batch size: 100\n", "Input dimension: torch.Size([1, 28, 28])\n", "Batch size: 100\n", "Input dimension: torch.Size([1, 28, 28])\n", "Batch size: 100\n", "Input dimension: torch.Size([1, 28, 28])\n", "Batch size: 100\n", "Input dimension: torch.Size([1, 28, 28])\n", "Batch size: 100\n", "Input dimension: torch.Size([1, 28, 28])\n", "Batch size: 100\n", "Input dimension: torch.Size([1, 28, 28])\n", "Batch size: 100\n", "Input dimension: torch.Size([1, 28, 28])\n", "Batch size: 100\n", "Input dimension: torch.Size([1, 28, 28])\n", "Batch size: 100\n", "Input dimension: torch.Size([1, 28, 28])\n", "Batch size: 100\n", "Input dimension: torch.Size([1, 28, 28])\n", "Batch size: 100\n", "Input dimension: torch.Size([1, 28, 28])\n", "Batch size: 100\n", "Input dimension: torch.Size([1, 28, 28])\n", "Batch size: 100\n", "Input dimension: torch.Size([1, 28, 28])\n", "Batch size: 100\n", "Input dimension: torch.Size([1, 28, 28])\n", "Batch size: 100\n", "Input dimension: torch.Size([1, 28, 28])\n", "Batch size: 100\n", "Input dimension: torch.Size([1, 28, 28])\n", "Batch size: 100\n", "Input dimension: torch.Size([1, 28, 28])\n", "Batch size: 100\n", "Input dimension: torch.Size([1, 28, 28])\n", "Batch size: 100\n", "Input dimension: torch.Size([1, 28, 28])\n", "Batch size: 100\n", "Input dimension: torch.Size([1, 28, 28])\n", "Batch size: 100\n", "Input dimension: torch.Size([1, 28, 28])\n", "Batch size: 100\n", "Input dimension: torch.Size([1, 28, 28])\n", "Batch size: 100\n", "Input dimension: torch.Size([1, 28, 28])\n", "Batch size: 100\n", "Input dimension: torch.Size([1, 28, 28])\n", "Batch size: 100\n", "Input dimension: torch.Size([1, 28, 28])\n", "Batch size: 100\n", "Input dimension: torch.Size([1, 28, 28])\n", "Batch size: 100\n", "Input dimension: torch.Size([1, 28, 28])\n", "Batch size: 100\n", "Input dimension: torch.Size([1, 28, 28])\n", "Batch size: 100\n", "Input dimension: torch.Size([1, 28, 28])\n", "Batch size: 100\n", "Input dimension: torch.Size([1, 28, 28])\n", "Batch size: 100\n", "Input dimension: torch.Size([1, 28, 28])\n", "Batch size: 100\n", "Input dimension: torch.Size([1, 28, 28])\n", "Batch size: 100\n", "Input dimension: torch.Size([1, 28, 28])\n", "Batch size: 100\n", "Input dimension: torch.Size([1, 28, 28])\n", "Batch size: 100\n", "Input dimension: torch.Size([1, 28, 28])\n", "Batch size: 100\n", "Input dimension: torch.Size([1, 28, 28])\n", "Batch size: 100\n", "Input dimension: torch.Size([1, 28, 28])\n", "Batch size: 100\n", "Input dimension: torch.Size([1, 28, 28])\n", "Batch size: 100\n", "Input dimension: torch.Size([1, 28, 28])\n", "Batch size: 100\n", "Input dimension: torch.Size([1, 28, 28])\n", "Batch size: 100\n", "Input dimension: torch.Size([1, 28, 28])\n", "Batch size: 100\n", "Input dimension: torch.Size([1, 28, 28])\n", "Batch size: 100\n", "Input dimension: torch.Size([1, 28, 28])\n", "Batch size: 100\n", "Input dimension: torch.Size([1, 28, 28])\n", "Batch size: 100\n", "Input dimension: torch.Size([1, 28, 28])\n", "Batch size: 100\n", "Input dimension: torch.Size([1, 28, 28])\n", "Batch size: 100\n", "Input dimension: torch.Size([1, 28, 28])\n", "Batch size: 100\n", "Input dimension: torch.Size([1, 28, 28])\n", "Batch size: 100\n", "Input dimension: torch.Size([1, 28, 28])\n", "Batch size: 100\n", "Input dimension: torch.Size([1, 28, 28])\n", "Batch size: 100\n", "Input dimension: torch.Size([1, 28, 28])\n", "Batch size: 100\n", "Input dimension: torch.Size([1, 28, 28])\n", "Batch size: 100\n", "Input dimension: torch.Size([1, 28, 28])\n", "Batch size: 100\n", "Input dimension: torch.Size([1, 28, 28])\n", "Batch size: 100\n", "Input dimension: torch.Size([1, 28, 28])\n", "Batch size: 100\n", "Input dimension: torch.Size([1, 28, 28])\n", "Batch size: 100\n", "Input dimension: torch.Size([1, 28, 28])\n", "Batch size: 100\n", "Input dimension: torch.Size([1, 28, 28])\n", "Batch size: 100\n", "Input dimension: torch.Size([1, 28, 28])\n", "Batch size: 100\n", "Input dimension: torch.Size([1, 28, 28])\n", "Batch size: 100\n", "Input dimension: torch.Size([1, 28, 28])\n", "Batch size: 100\n", "Input dimension: torch.Size([1, 28, 28])\n", "Batch size: 100\n", "Input dimension: torch.Size([1, 28, 28])\n", "Batch size: 100\n", "Input dimension: torch.Size([1, 28, 28])\n", "Batch size: 100\n", "Input dimension: torch.Size([1, 28, 28])\n", "Batch size: 100\n", "Input dimension: torch.Size([1, 28, 28])\n", "Batch size: 100\n", "Input dimension: torch.Size([1, 28, 28])\n", "Batch size: 100\n", "Input dimension: torch.Size([1, 28, 28])\n", "Batch size: 100\n", "Input dimension: torch.Size([1, 28, 28])\n", "Batch size: 100\n", "Input dimension: torch.Size([1, 28, 28])\n", "Batch size: 100\n", "Input dimension: torch.Size([1, 28, 28])\n", "Batch size: 100\n", "Input dimension: torch.Size([1, 28, 28])\n", "Batch size: 100\n", "Input dimension: torch.Size([1, 28, 28])\n", "Batch size: 100\n", "Input dimension: torch.Size([1, 28, 28])\n", "Batch size: 100\n", "Input dimension: torch.Size([1, 28, 28])\n", "Batch size: 100\n", "Input dimension: torch.Size([1, 28, 28])\n", "Batch size: 100\n", "Input dimension: torch.Size([1, 28, 28])\n", "Batch size: 100\n", "Input dimension: torch.Size([1, 28, 28])\n", "Batch size: 100\n", "Input dimension: torch.Size([1, 28, 28])\n", "Batch size: 100\n", "Input dimension: torch.Size([1, 28, 28])\n", "Batch size: 100\n", "Input dimension: torch.Size([1, 28, 28])\n", "Batch size: 100\n", "Input dimension: torch.Size([1, 28, 28])\n", "Batch size: 100\n", "Input dimension: torch.Size([1, 28, 28])\n", "Batch size: 100\n", "Input dimension: torch.Size([1, 28, 28])\n", "Batch size: 100\n", "Input dimension: torch.Size([1, 28, 28])\n", "Batch size: 100\n", "Input dimension: torch.Size([1, 28, 28])\n", "Batch size: 100\n", "Input dimension: torch.Size([1, 28, 28])\n", "Batch size: 100\n", "Input dimension: torch.Size([1, 28, 28])\n", "Batch size: 100\n", "Input dimension: torch.Size([1, 28, 28])\n", "Batch size: 100\n", "Input dimension: torch.Size([1, 28, 28])\n", "Batch size: 100\n", "Input dimension: torch.Size([1, 28, 28])\n", "Batch size: 100\n", "Input dimension: torch.Size([1, 28, 28])\n", "Batch size: 100\n", "Input dimension: torch.Size([1, 28, 28])\n", "Batch size: 100\n", "Input dimension: torch.Size([1, 28, 28])\n", "Batch size: 100\n", "Input dimension: torch.Size([1, 28, 28])\n", "Batch size: 100\n", "Input dimension: torch.Size([1, 28, 28])\n", "Batch size: 100\n", "Input dimension: torch.Size([1, 28, 28])\n", "Batch size: 100\n", "Input dimension: torch.Size([1, 28, 28])\n", "Batch size: 100\n", "Input dimension: torch.Size([1, 28, 28])\n", "Batch size: 100\n", "Input dimension: torch.Size([1, 28, 28])\n", "Batch size: 100\n", "Input dimension: torch.Size([1, 28, 28])\n", "Batch size: 100\n", "Input dimension: torch.Size([1, 28, 28])\n", "Batch size: 100\n", "Input dimension: torch.Size([1, 28, 28])\n", "Batch size: 100\n", "Input dimension: torch.Size([1, 28, 28])\n", "Batch size: 100\n", "Input dimension: torch.Size([1, 28, 28])\n", "Batch size: 100\n", "Input dimension: torch.Size([1, 28, 28])\n", "Batch size: 100\n", "Input dimension: torch.Size([1, 28, 28])\n", "Batch size: 100\n", "Input dimension: torch.Size([1, 28, 28])\n", "Batch size: 100\n", "Input dimension: torch.Size([1, 28, 28])\n", "Batch size: 100\n", "Input dimension: torch.Size([1, 28, 28])\n", "Batch size: 100\n", "Input dimension: torch.Size([1, 28, 28])\n", "Batch size: 100\n", "Input dimension: torch.Size([1, 28, 28])\n", "Batch size: 100\n", "Input dimension: torch.Size([1, 28, 28])\n", "Batch size: 100\n", "Input dimension: torch.Size([1, 28, 28])\n", "Batch size: 100\n", "Input dimension: torch.Size([1, 28, 28])\n", "Batch size: 100\n", "Input dimension: torch.Size([1, 28, 28])\n", "Batch size: 100\n", "Input dimension: torch.Size([1, 28, 28])\n", "Batch size: 100\n", "Input dimension: torch.Size([1, 28, 28])\n", "Batch size: 100\n", "Input dimension: torch.Size([1, 28, 28])\n", "Batch size: 100\n", "Input dimension: torch.Size([1, 28, 28])\n", "Batch size: 100\n", "Input dimension: torch.Size([1, 28, 28])\n", "Batch size: 100\n", "Input dimension: torch.Size([1, 28, 28])\n", "Batch size: 100\n", "Input dimension: torch.Size([1, 28, 28])\n", "Batch size: 100\n", "Input dimension: torch.Size([1, 28, 28])\n", "Batch size: 100\n", "Input dimension: torch.Size([1, 28, 28])\n", "Batch size: 100\n", "Input dimension: torch.Size([1, 28, 28])\n", "Batch size: 100\n", "Input dimension: torch.Size([1, 28, 28])\n", "Batch size: 100\n", "Input dimension: torch.Size([1, 28, 28])\n", "Batch size: 100\n", "Input dimension: torch.Size([1, 28, 28])\n", "Batch size: 100\n", "Input dimension: torch.Size([1, 28, 28])\n", "Batch size: 100\n", "Input dimension: torch.Size([1, 28, 28])\n", "Batch size: 100\n", "Input dimension: torch.Size([1, 28, 28])\n", "Batch size: 100\n", "Input dimension: torch.Size([1, 28, 28])\n", "Batch size: 100\n", "Input dimension: torch.Size([1, 28, 28])\n", "Batch size: 100\n", "Input dimension: torch.Size([1, 28, 28])\n", "Batch size: 100\n", "Input dimension: torch.Size([1, 28, 28])\n", "Batch size: 100\n", "Input dimension: torch.Size([1, 28, 28])\n", "Batch size: 100\n", "Input dimension: torch.Size([1, 28, 28])\n", "Batch size: 100\n", "Input dimension: torch.Size([1, 28, 28])\n", "Batch size: 100\n", "Input dimension: torch.Size([1, 28, 28])\n", "Batch size: 100\n", "Input dimension: torch.Size([1, 28, 28])\n", "Batch size: 100\n", "Input dimension: torch.Size([1, 28, 28])\n", "Batch size: 100\n", "Input dimension: torch.Size([1, 28, 28])\n", "Batch size: 100\n", "Input dimension: torch.Size([1, 28, 28])\n", "Batch size: 100\n", "Input dimension: torch.Size([1, 28, 28])\n", "Batch size: 100\n", "Input dimension: torch.Size([1, 28, 28])\n", "Batch size: 100\n", "Input dimension: torch.Size([1, 28, 28])\n", "Batch size: 100\n", "Input dimension: torch.Size([1, 28, 28])\n", "Batch size: 100\n", "Input dimension: torch.Size([1, 28, 28])\n", "Batch size: 100\n", "Input dimension: torch.Size([1, 28, 28])\n", "Batch size: 100\n", "Input dimension: torch.Size([1, 28, 28])\n", "Batch size: 100\n", "Input dimension: torch.Size([1, 28, 28])\n", "Batch size: 100\n", "Input dimension: torch.Size([1, 28, 28])\n", "Batch size: 100\n", "Input dimension: torch.Size([1, 28, 28])\n", "Batch size: 100\n", "Input dimension: torch.Size([1, 28, 28])\n", "Batch size: 100\n", "Input dimension: torch.Size([1, 28, 28])\n", "Batch size: 100\n", "Input dimension: torch.Size([1, 28, 28])\n", "Batch size: 100\n", "Input dimension: torch.Size([1, 28, 28])\n", "Batch size: 100\n", "Input dimension: torch.Size([1, 28, 28])\n", "Batch size: 100\n", "Input dimension: torch.Size([1, 28, 28])\n", "Batch size: 100\n", "Input dimension: torch.Size([1, 28, 28])\n", "Batch size: 100\n", "Input dimension: torch.Size([1, 28, 28])\n", "Batch size: 100\n", "Input dimension: torch.Size([1, 28, 28])\n", "Batch size: 100\n", "Input dimension: torch.Size([1, 28, 28])\n", "Batch size: 100\n", "Input dimension: torch.Size([1, 28, 28])\n", "Batch size: 100\n", "Input dimension: torch.Size([1, 28, 28])\n", "Batch size: 100\n", "Input dimension: torch.Size([1, 28, 28])\n", "Batch size: 100\n", "Input dimension: torch.Size([1, 28, 28])\n", "Batch size: 100\n", "Input dimension: torch.Size([1, 28, 28])\n", "Batch size: 100\n", "Input dimension: torch.Size([1, 28, 28])\n", "Batch size: 100\n", "Input dimension: torch.Size([1, 28, 28])\n", "Batch size: 100\n", "Input dimension: torch.Size([1, 28, 28])\n", "Batch size: 100\n", "Input dimension: torch.Size([1, 28, 28])\n", "Batch size: 100\n", "Input dimension: torch.Size([1, 28, 28])\n", "Batch size: 100\n", "Input dimension: torch.Size([1, 28, 28])\n", "Batch size: 100\n", "Input dimension: torch.Size([1, 28, 28])\n", "Batch size: 100\n", "Input dimension: torch.Size([1, 28, 28])\n", "Batch size: 100\n", "Input dimension: torch.Size([1, 28, 28])\n", "Batch size: 100\n", "Input dimension: torch.Size([1, 28, 28])\n", "Batch size: 100\n", "Input dimension: torch.Size([1, 28, 28])\n", "Batch size: 100\n", "Input dimension: torch.Size([1, 28, 28])\n", "Batch size: 100\n", "Input dimension: torch.Size([1, 28, 28])\n", "Batch size: 100\n", "Input dimension: torch.Size([1, 28, 28])\n", "Batch size: 100\n", "Input dimension: torch.Size([1, 28, 28])\n", "Batch size: 100\n", "Input dimension: torch.Size([1, 28, 28])\n", "Batch size: 100\n", "Input dimension: torch.Size([1, 28, 28])\n", "Batch size: 100\n", "Input dimension: torch.Size([1, 28, 28])\n", "Batch size: 100\n", "Input dimension: torch.Size([1, 28, 28])\n", "Batch size: 100\n", "Input dimension: torch.Size([1, 28, 28])\n", "Batch size: 100\n", "Input dimension: torch.Size([1, 28, 28])\n", "Batch size: 100\n", "Input dimension: torch.Size([1, 28, 28])\n", "Batch size: 100\n", "Input dimension: torch.Size([1, 28, 28])\n", "Batch size: 100\n", "Input dimension: torch.Size([1, 28, 28])\n", "Batch size: 100\n", "Input dimension: torch.Size([1, 28, 28])\n", "Batch size: 100\n", "Input dimension: torch.Size([1, 28, 28])\n", "Batch size: 100\n", "Input dimension: torch.Size([1, 28, 28])\n", "Batch size: 100\n", "Input dimension: torch.Size([1, 28, 28])\n", "Batch size: 100\n", "Input dimension: torch.Size([1, 28, 28])\n", "Batch size: 100\n", "Input dimension: torch.Size([1, 28, 28])\n", "Batch size: 100\n", "Input dimension: torch.Size([1, 28, 28])\n", "Batch size: 100\n", "Input dimension: torch.Size([1, 28, 28])\n", "Batch size: 100\n", "Input dimension: torch.Size([1, 28, 28])\n", "Batch size: 100\n", "Input dimension: torch.Size([1, 28, 28])\n", "Batch size: 100\n", "Input dimension: torch.Size([1, 28, 28])\n", "Batch size: 100\n", "Input dimension: torch.Size([1, 28, 28])\n", "Batch size: 100\n", "Input dimension: torch.Size([1, 28, 28])\n", "Batch size: 100\n", "Input dimension: torch.Size([1, 28, 28])\n", "Batch size: 100\n", "Input dimension: torch.Size([1, 28, 28])\n", "Batch size: 100\n", "Input dimension: torch.Size([1, 28, 28])\n", "Batch size: 100\n", "Input dimension: torch.Size([1, 28, 28])\n", "Batch size: 100\n", "Input dimension: torch.Size([1, 28, 28])\n", "Batch size: 100\n", "Input dimension: torch.Size([1, 28, 28])\n", "Batch size: 100\n", "Input dimension: torch.Size([1, 28, 28])\n", "Batch size: 100\n", "Input dimension: torch.Size([1, 28, 28])\n", "Batch size: 100\n", "Input dimension: torch.Size([1, 28, 28])\n", "Batch size: 100\n", "Input dimension: torch.Size([1, 28, 28])\n", "Batch size: 100\n", "Input dimension: torch.Size([1, 28, 28])\n", "Batch size: 100\n", "Input dimension: torch.Size([1, 28, 28])\n", "Batch size: 100\n", "Input dimension: torch.Size([1, 28, 28])\n", "Batch size: 100\n", "Input dimension: torch.Size([1, 28, 28])\n", "Batch size: 100\n", "Input dimension: torch.Size([1, 28, 28])\n", "Batch size: 100\n", "Input dimension: torch.Size([1, 28, 28])\n", "Batch size: 100\n", "Input dimension: torch.Size([1, 28, 28])\n", "Batch size: 100\n", "Input dimension: torch.Size([1, 28, 28])\n", "Batch size: 100\n", "Input dimension: torch.Size([1, 28, 28])\n", "Batch size: 100\n", "Input dimension: torch.Size([1, 28, 28])\n", "Batch size: 100\n", "Input dimension: torch.Size([1, 28, 28])\n", "Batch size: 100\n", "Input dimension: torch.Size([1, 28, 28])\n", "Batch size: 100\n", "Input dimension: torch.Size([1, 28, 28])\n", "Batch size: 100\n", "Input dimension: torch.Size([1, 28, 28])\n", "Batch size: 100\n", "Input dimension: torch.Size([1, 28, 28])\n", "Batch size: 100\n", "Input dimension: torch.Size([1, 28, 28])\n", "Batch size: 100\n", "Input dimension: torch.Size([1, 28, 28])\n", "Batch size: 100\n", "Input dimension: torch.Size([1, 28, 28])\n", "Batch size: 100\n", "Input dimension: torch.Size([1, 28, 28])\n", "Batch size: 100\n", "Input dimension: torch.Size([1, 28, 28])\n", "Batch size: 100\n", "Input dimension: torch.Size([1, 28, 28])\n", "Batch size: 100\n", "Input dimension: torch.Size([1, 28, 28])\n", "Batch size: 100\n", "Input dimension: torch.Size([1, 28, 28])\n", "Batch size: 100\n", "Input dimension: torch.Size([1, 28, 28])\n", "Batch size: 100\n", "Input dimension: torch.Size([1, 28, 28])\n", "Batch size: 100\n", "Input dimension: torch.Size([1, 28, 28])\n", "Batch size: 100\n", "Input dimension: torch.Size([1, 28, 28])\n", "Batch size: 100\n", "Input dimension: torch.Size([1, 28, 28])\n", "Batch size: 100\n", "Input dimension: torch.Size([1, 28, 28])\n", "Batch size: 100\n", "Input dimension: torch.Size([1, 28, 28])\n", "Batch size: 100\n", "Input dimension: torch.Size([1, 28, 28])\n", "Batch size: 100\n", "Input dimension: torch.Size([1, 28, 28])\n", "Batch size: 100\n", "Input dimension: torch.Size([1, 28, 28])\n", "Batch size: 100\n", "Input dimension: torch.Size([1, 28, 28])\n", "Batch size: 100\n", "Input dimension: torch.Size([1, 28, 28])\n", "Batch size: 100\n", "Input dimension: torch.Size([1, 28, 28])\n", "Batch size: 100\n", "Input dimension: torch.Size([1, 28, 28])\n", "Batch size: 100\n", "Input dimension: torch.Size([1, 28, 28])\n", "Batch size: 100\n", "Input dimension: torch.Size([1, 28, 28])\n", "Batch size: 100\n", "Input dimension: torch.Size([1, 28, 28])\n", "Batch size: 100\n", "Input dimension: torch.Size([1, 28, 28])\n", "Batch size: 100\n", "Input dimension: torch.Size([1, 28, 28])\n", "Batch size: 100\n", "Input dimension: torch.Size([1, 28, 28])\n", "Batch size: 100\n", "Input dimension: torch.Size([1, 28, 28])\n", "Batch size: 100\n", "Input dimension: torch.Size([1, 28, 28])\n", "Batch size: 100\n", "Input dimension: torch.Size([1, 28, 28])\n", "Batch size: 100\n", "Input dimension: torch.Size([1, 28, 28])\n", "Batch size: 100\n", "Input dimension: torch.Size([1, 28, 28])\n", "Batch size: 100\n", "Input dimension: torch.Size([1, 28, 28])\n", "Batch size: 100\n", "Input dimension: torch.Size([1, 28, 28])\n", "Batch size: 100\n", "Input dimension: torch.Size([1, 28, 28])\n", "Batch size: 100\n", "Input dimension: torch.Size([1, 28, 28])\n", "Batch size: 100\n", "Input dimension: torch.Size([1, 28, 28])\n", "Batch size: 100\n", "Input dimension: torch.Size([1, 28, 28])\n", "Batch size: 100\n", "Input dimension: torch.Size([1, 28, 28])\n" ] } ], "source": [ "import torch, torchvision\n", "from torchvision import transforms\n", "\n", "train_dataset = torchvision.datasets.MNIST(\n", " root='.', # Replace with your desired data path\n", " train=True,\n", " transform=transforms.ToTensor(),\n", " download=True)\n", "\n", "train_loader = torch.utils.data.DataLoader(\n", " dataset=train_dataset,\n", " batch_size=100,\n", " shuffle=True)\n", "\n", "for a, b in train_loader:\n", " print('Batch size:', a.shape[0])\n", " print('Input dimension:', a[0].shape)" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "Note that the batch of data is the tensor `a` which has dimensions batch_size × input_dimension. In our particular example we have chosen a batch size of 100 and the MNIST images have dimension 28×28. The network we defined in section 2 has an input dimension of 784 = 28 x 28. In order to reshape the batch tensor `a` as a tensor of dimensions batch_size × 784 we can use the following:\n", "`a = a.view(a.shape[0], -1)`.\n", "With this shape we can feed the tensor to our network.\n", "We now visualize some examples from the MNIST dataset.\n" ] }, { "cell_type": "code", "execution_count": 5, "metadata": {}, "outputs": [ { "data": { "image/png": "iVBORw0KGgoAAAANSUhEUgAAAxoAAAEOCAYAAAAOmGH2AAAAOXRFWHRTb2Z0d2FyZQBNYXRwbG90bGliIHZlcnNpb24zLjkuMiwgaHR0cHM6Ly9tYXRwbG90bGliLm9yZy8hTgPZAAAACXBIWXMAAA9hAAAPYQGoP6dpAAAWzklEQVR4nO3df6zVdf0H8NcBEn9QYjIcifijNIRZswzvZWYYTsZ0DRdcpzbm2io1pgLmKH+gK1duci+WpVbij6AUSB0lUyulP/JekKFt/iCV+QOoEFBHaIFyz/cP92Uq+P6cy+d97zkHHo/NPzzPz/l8Xvde7pvzvB/ueVeq1Wo1AAAAMupX7wEAAIC9j6IBAABkp2gAAADZKRoAAEB2igYAAJCdogEAAGSnaAAAANkpGgAAQHaKBgAAkJ2i0cRefvnlqFQqceONN2Y757Jly6JSqcSyZcuynRNoTNYQoCzrCCmKRh+78847o1KpxMqVK+s9Sq846qijolKp7Pa/Y489tt7jQdPb29eQiIg///nPcdppp8WQIUNi8ODBMWbMmPjNb35T77Fgr7G3ryNeizSOAfUegL3L3LlzY+vWrR947JVXXomrrroqzjjjjDpNBTSLJUuWxKRJk6K1tTWuvfbaqFQqsXDhwpg6dWps2rQppk+fXu8RgQbntUjjUDTIatKkSbs89qMf/SgiIs4///w+ngZoNjfffHMMGzYsHn300Rg4cGBERHznO9+JkSNHxp133qloAIW8Fmkc/ulUA9q+fXtcc8018cUvfjEOPvjgOOigg+LLX/5yPPbYYx/5nI6OjjjyyCPjgAMOiK985Svx9NNP73LM6tWrY/LkyfHJT34y9t9//zjppJNiyZIlhfO8/fbbsXr16ti0adMefTy//e1v4+ijj46xY8fu0fOBnmnmNWTLli1xyCGH7CwZEREDBgyIIUOGxAEHHFD4fCCPZl5HdsdrkfpQNBrQli1b4te//nWMGzcubrjhhrj22mtj48aNMWHChHjqqad2Of7uu++On/70p/Hd7343vv/978fTTz8dX/3qV2PDhg07j3nmmWeipaUlnnvuuZg1a1bMmTMnDjrooJg0aVLcf//9yXlWrFgRxx9/fNx88809/liefPLJeO655+K8887r8XOBPdPMa8i4cePimWeeiauvvjpefPHFWLNmTfzwhz+MlStXxhVXXNHjzwWwZ5p5Hfkwr0XqqEqfuuOOO6oRUX3iiSc+8ph33323um3btg889sYbb1QPO+yw6je/+c2dj7300kvViKgecMAB1XXr1u18fPny5dWIqE6fPn3nY+PHj6+ecMIJ1f/97387H+vu7q6OHTu2euyxx+587LHHHqtGRPWxxx7b5bHZs2f3+OOdOXNmNSKqzz77bI+fC+xqb19Dtm7dWm1ra6tWKpVqRFQjonrggQdWH3jggcLnArXZ29eRD/NapH7c0WhA/fv3j/322y8iIrq7u+P111+Pd999N0466aRYtWrVLsdPmjQpDj/88J3/P2bMmDj55JNj6dKlERHx+uuvx6OPPhptbW3xn//8JzZt2hSbNm2KzZs3x4QJE+KFF16I9evXf+Q848aNi2q1Gtdee22PPo7u7u6455574sQTT4zjjz++R88F9lwzryEDBw6M4447LiZPnhy/+93vYv78+XHSSSfFN77xjejq6urhZwLYU828jryf1yL15ZfBG9Rdd90Vc+bMidWrV8c777yz8/Gjjz56l2N391Ztxx13XCxcuDAiIl588cWoVqtx9dVXx9VXX73b67322msfWCBy+Otf/xrr16/3y5tQB826hkybNi26urpi1apV0a/fez8La2tri9GjR8ell14ay5cvL30NoDbNuo68n9ci9aVoNKD58+fHBRdcEJMmTYrvfe97MXTo0Ojfv3/8+Mc/jjVr1vT4fN3d3RERcfnll8eECRN2e8xnPvOZUjPvzoIFC6Jfv35x7rnnZj838NGadQ3Zvn173H777XHFFVfsLBkRER/72Mdi4sSJcfPNN8f27dt3/pQV6D3Nuo58mNci9aVoNKDFixfHMcccE/fdd19UKpWdj8+ePXu3x7/wwgu7PPb888/HUUcdFRERxxxzTES895f16aefnn/g3di2bVv8/ve/j3HjxsWnPvWpPrkm8J5mXUM2b94c7777buzYsWOX7J133onu7u7dZkB+zbqOvJ/XIvXndzQaUP/+/SMiolqt7nxs+fLl0dnZudvjH3jggQ/8u8YVK1bE8uXLY+LEiRERMXTo0Bg3blzcdttt8a9//WuX52/cuDE5z568pdzSpUvjzTff9H7VUAfNuoYMHTo0Bg8eHPfff39s37595+Nbt26NP/zhDzFy5EhvcQt9pFnXkffzWqT+3NGok3nz5sVDDz20y+OXXnppnHXWWXHffffF2WefHWeeeWa89NJLceutt8aoUaN22eky4r1bjaecckpcdNFFsW3btpg7d24ceuihH3gryJ///OdxyimnxAknnBDf+ta34phjjokNGzZEZ2dnrFu3Lv7+979/5KwrVqyI0047LWbPnl3zL2EtWLAgBg4cGF//+tdrOh7omb1xDenfv39cfvnlcdVVV0VLS0tMnTo1duzYEbfffnusW7cu5s+f37NPEpC0N64j7+e1SP0pGnVyyy237PbxCy64IC644IL497//Hbfddls8/PDDMWrUqJg/f34sWrQoli1btstzpk6dGv369Yu5c+fGa6+9FmPGjNm5u+7/GzVqVKxcuTKuu+66uPPOO2Pz5s0xdOjQOPHEE+Oaa67J+rFt2bIlHnzwwTjzzDPj4IMPznpu4D176xpy5ZVXxtFHHx033XRTXHfddbFt27b43Oc+F4sXL/ZiATLbW9eRCK9FGkWl+v57YgAAABn4HQ0AACA7RQMAAMhO0QAAALJTNAAAgOwUDQAAIDtFAwAAyE7RAAAAsqt5w75KpdKbcwA90Kzb31hHoHE04zpiDYHGUcsa4o4GAACQnaIBAABkp2gAAADZKRoAAEB2igYAAJCdogEAAGSnaAAAANkpGgAAQHaKBgAAkJ2iAQAAZKdoAAAA2SkaAABAdooGAACQnaIBAABkp2gAAADZKRoAAEB2igYAAJCdogEAAGSnaAAAANkpGgAAQHaKBgAAkJ2iAQAAZKdoAAAA2SkaAABAdooGAACQnaIBAABkp2gAAADZKRoAAEB2igYAAJCdogEAAGSnaAAAANkpGgAAQHYD6j0AAM2lpaUlmc+YMSOZDx8+PJm3trb2eKaeKpqxo6Oj12cA2Nu5owEAAGSnaAAAANkpGgAAQHaKBgAAkJ2iAQAAZKdoAAAA2SkaAABAdpVqtVqt6cBKpbdnAWpU47dtw7GONIe2trZkfu+99/bRJPXT3t6ezGfOnNlHk/SeZlxHrCHQOGpZQ9zRAAAAslM0AACA7BQNAAAgO0UDAADITtEAAACyUzQAAIDsFA0AACA7+2g0mCFDhiTzhQsXJvM//elPpa7f2dmZzJctW1bq/OTRjO9/H2EdaRRHHHFEMn/11Vf7aJI9t3bt2mTe0dGRzKdPn57Miz5Hra2tybyrqyuZN4JmXEesIdA47KMBAADUhaIBAABkp2gAAADZKRoAAEB2igYAAJCdogEAAGSnaAAAANkNqPcAfNDPfvazZH7qqaeWyou88847ybxoH40//vGPpa4fEbF48eJkvmHDhtLXgH3ZvffeW+8RSiva56JI0T4XRecfMWJEqfPTvK644opk/vnPfz6Zn3vuuaWuX7SXyKpVq5J50R40ERFnnXVWMv/lL39ZeI7etGLFimT+5JNPFp7j2WefTeZFr4eojTsaAABAdooGAACQnaIBAABkp2gAAADZKRoAAEB2igYAAJCdogEAAGRXqVar1ZoOLHjfZooNGTKk8JilS5cm8y984Qu5xtmtoq9zjX9cSrn44ouTeb3fv7sR9MXXoTdYR8qrZf+Ion0yWltbc42zW4sWLUrm7e3tyTzHHhRtbW3J/MYbb0zmRZ/nc845J5kvXLgwmTeCZlxHGmEN+dvf/pbMW1pa+mgSyijab+TBBx9M5kXr2JYtW3o8U7OpZQ1xRwMAAMhO0QAAALJTNAAAgOwUDQAAIDtFAwAAyE7RAAAAslM0AACA7OyjkdHQoUOT+YIFCwrPcdppp5Wa4a233krmY8aMSebjxo1L5jfddFMyHzBgQDKvxdtvv53MzzjjjGSe4z34G10zvv99hHUkh1r2Z5gyZUofTPLRZsyYkcw7Ojr6aJKP9uqrrybzon00RowYkczXrl3b45n6WjOuI42whsyePTuZX3LJJcl88ODBGaehXrZu3ZrMx48fn8xXrlyZc5y6sI8GAABQF4oGAACQnaIBAABkp2gAAADZKRoAAEB2igYAAJCdogEAAGRXftMDdjrhhBOSedk9MiIitm3blsxnzZqVzP/xj3+Uyp944olkvnTp0mQeETFkyJBkfuCBBybzmTNnJvN67yEAZTz++OPJvLW1tfQ12tvbk3nRHhJFe0QsXry4xzPlNn369GRe9DF2dnYm82bYJ4Pecd111yXzoj2zjjzyyJzj9IrRo0cn86K/x0eOHJnMn3322WQ+atSoZF5k2LBhhceMHTu21DUGDRqUzI8//vhkvjfso1ELdzQAAIDsFA0AACA7RQMAAMhO0QAAALJTNAAAgOwUDQAAIDtFAwAAyE7RAAAAsrNhX0ZFG0Dl8NRTTyXzW265pVevv379+l49fy0eeeSReo8Ae6wvNuQrOkdXV1fpa9Rb0XpbtGFfkblz55Z6PvuuF198sVTeCP7yl7/Ue4RSBg4cWHjMSy+9lMwPO+ywXOPs09zRAAAAslM0AACA7BQNAAAgO0UDAADITtEAAACyUzQAAIDsFA0AACA7+2hkdOGFF5Y+x7Zt25L5DTfcUPoaZdx///3JfMiQIaWvUfT+3Q888EDpa0BvaWlpSebDhw8vdf61a9cWHjNixIhkXrQfTi3XqLfLLrssmRfts7Fo0aJk3tnZ2dORgAYxePDgwmP222+/UtfYsWNHMv/nP/9Z6vx7C3c0AACA7BQNAAAgO0UDAADITtEAAACyUzQAAIDsFA0AACA7RQMAAMjOPhoZLViwIJkPGzas8By33HJLMl+yZEmPZuqpj3/848l89OjRvXr9iIgrr7wymW/cuLHXZ4A9VbRHRdH+DkVqef69995b6hpFivaYWLduXannT58+vXCGos9D0V4g7e3tpZ4P1E+/fumfk0+bNq3wHIccckipGbq6upJ50Z5g+wp3NAAAgOwUDQAAIDtFAwAAyE7RAAAAslM0AACA7BQNAAAgO0UDAADIrlKtVqs1HVip9PYs9IH9998/mRe97/PJJ59ceoYVK1Yk80mTJiXz1157rfQMza7Gb9uGYx2JaGlpSeZTpkxJ5rXsozF8+PAezfRhra2tpZ7fCIr26pgxY0YyL3qP/L1BM64j1hAiIo466qhkvmbNml6f4fLLL0/mHR0dvT5DvdWyhrijAQAAZKdoAAAA2SkaAABAdooGAACQnaIBAABkp2gAAADZKRoAAEB29tHYx1x//fXJfNasWcm86I/LqlWrCmeYOHFiMt+8eXPhOfZ1zfj+9xHWkb1Fs/7564kRI0Yk87Vr1/bRJL2nGb+O1pB9Q79+6Z+D33XXXcn8vPPOKz1D0ff4l770pWS+cePG0jM0OvtoAAAAdaFoAAAA2SkaAABAdooGAACQnaIBAABkp2gAAADZKRoAAEB2A+o9AHl94hOfSOZF+2QUvXd1d3d3Mn/mmWeSeYR9MqDeWlpaknl7e3uvXn/RokWlr9/V1ZXMH3/88WTe2tqazCdPnpzMOzo6kjmw56ZNm5bMc+yTUeRXv/pVMt8X9snIwR0NAAAgO0UDAADITtEAAACyUzQAAIDsFA0AACA7RQMAAMhO0QAAALKzj0aTGTRoUDIven/6arWazIv2ySh6ftF72wO9b86cOcl8xowZpc7f2dmZzOfOnZvMFy5cWOr6tVi3bl2p569fvz7TJMCHFe3lc/HFF/f6DEV77fziF7/o9Rn2Be5oAAAA2SkaAABAdooGAACQnaIBAABkp2gAAADZKRoAAEB2igYAAJCdogEAAGRXqRbtwPb/B1YqvT0LNTj88MOT+SuvvFLq/EVf50suuSSZ33rrrYXX2LFjR49mYlc1fts2HOtIsSOOOCKZF23GFxExZcqUUjO0t7cn85kzZ5Y6f18o+z2yL/xZbcZ1ZF/4uuwLbrrppmQ+bdq0Xp9h4sSJyfyRRx7p9RmaXS1riDsaAABAdooGAACQnaIBAABkp2gAAADZKRoAAEB2igYAAJCdogEAAGQ3oN4D0DOXXXZZr57/+eefT+bz5s1L5vbIYF/X1taWzCdPnpzMi/bAWLt2beEMnZ2dyfycc84pfY16q2U/kZRFixZlmgT4sAkTJiTzs88+u1evv379+sJjnnjiiV6dgfe4owEAAGSnaAAAANkpGgAAQHaKBgAAkJ2iAQAAZKdoAAAA2SkaAABAdvbRaDCnn356Mp86dWqp869ZsyaZ33jjjcn8v//9b6nrQ7ObPn16qfyII45I5kV7WHR0dCTziIjFixeXuka9FX2OIor3GylStNcI8NFGjhyZzO+4445kfthhh5W6/ptvvpnMzzjjjMJzvPHGG6VmoDbuaAAAANkpGgAAQHaKBgAAkJ2iAQAAZKdoAAAA2SkaAABAdooGAACQnX00+tCgQYMKj/nBD36QzA899NBSM8ybNy+Z33777aXOD82upaUlmbe3t5c6f9EeFm1tbcm8q6ur1PVzKJqxyOGHH57Mi/YiiSjea6Non4yivUZgX9avX/rn0D/96U+Tedl9Mor85Cc/SearV6/u1etTO3c0AACA7BQNAAAgO0UDAADITtEAAACyUzQAAIDsFA0AACA7RQMAAMjOPhp96Prrry885tRTTy11je7u7mS+efPmUueHvd2UKVN69fxl938oyiMihg8fXmqGZlD0eRg7dmwfTQLNZb/99is85qKLLkrm48ePzzXObt1zzz3JfM6cOb16ffJxRwMAAMhO0QAAALJTNAAAgOwUDQAAIDtFAwAAyE7RAAAAslM0AACA7CrVarVa04GVSm/P0vRGjx6dzB966KHCcwwbNqzUDPPmzUvm3/72t0udn8ZQ47dtw2mGdaRoj4nLLrssmZfdh6MZ9rhYtGhRqecX7YGxePHiwnOsXbu21Aw05zrSDGtIozv//PMLj7n77rt7dYbly5cn86997WvJfNOmTTnHYQ/Vsoa4owEAAGSnaAAAANkpGgAAQHaKBgAAkJ2iAQAAZKdoAAAA2SkaAABAdvbRyGj8+PHJ/OGHHy59jQ0bNiTzCRMmJPOnn3669AzUXzO+/32EdQQaSTOuI9aQYhdeeGEyv+GGGwrPMWjQoFIzPPXUU8n89NNPT+ZvvPFGqevTN+yjAQAA1IWiAQAAZKdoAAAA2SkaAABAdooGAACQnaIBAABkp2gAAADZKRoAAEB2A+o9wN5kzZo1pfKIiE9/+tPJfNasWcnchnwAsO8aPHhwMi+7GV9ExFtvvZXMzz///GRuQ759hzsaAABAdooGAACQnaIBAABkp2gAAADZKRoAAEB2igYAAJCdogEAAGRnH42MXn755WT+2c9+tm8GAQD2SU8++WQyX7JkSeE5zjrrrGRetE/G6tWrC6/BvsEdDQAAIDtFAwAAyE7RAAAAslM0AACA7BQNAAAgO0UDAADITtEAAACyq1Sr1WpNB1YqvT0LUKMav20bjnUEGkczriPWEGgctawh7mgAAADZKRoAAEB2igYAAJCdogEAAGSnaAAAANkpGgAAQHaKBgAAkF3N+2gAAADUyh0NAAAgO0UDAADITtEAAACyUzQAAIDsFA0AACA7RQMAAMhO0QAAALJTNAAAgOwUDQAAILv/Azv5Up3jd2fjAAAAAElFTkSuQmCC", "text/plain": [ "
" ] }, "metadata": {}, "output_type": "display_data" } ], "source": [ "import matplotlib.pyplot as plt\n", "\n", "# Get 3 examples from the dataset\n", "examples = iter(train_loader)\n", "images, labels = next(examples)\n", "\n", "# Plot the images\n", "fig, axes = plt.subplots(1, 3, figsize=(10, 3))\n", "for i in range(3):\n", " axes[i].imshow(images[i].numpy().squeeze(), cmap='gray')\n", " axes[i].set_title(f'Label: {labels[i].item()}')\n", " axes[i].axis('off')\n", "\n", "plt.show()" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "\n", "## Defining a loss function and obtaining its gradient with respect to the network's parameters\n", "\n", "Now that we have a neural network architecture and some labeled data, we turn to the task of choosing a set of parameters that make the network perform well on the image classification task. One popular loss function for image classification is the cross-entropy loss, and PyTorch makes it easy to obtain the gradient of such a function with respect to each of the network's parameters, in order to use first-order optimization methods. In the following example we will obtain the gradient on each mini-batch of the training data:\n" ] }, { "cell_type": "code", "execution_count": 6, "metadata": {}, "outputs": [], "source": [ "for a, b in train_loader:\n", " loss_fn = nn.CrossEntropyLoss()\n", " a = a.view(a.shape[0], -1)\n", " net.zero_grad() # set the gradients to 0\n", " output = net(a)\n", " loss = loss_fn(output, b)\n", " loss.backward() # backpropagation\n", " for p in net.parameters():\n", " gradient = p.grad" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "Notice that the gradient is stored as an attribute `p.grad` on each of the network's parameters. Also, we have to set all the gradients to zero with the line `net.zero_grad()`. This is because everytime the method `loss.backward()` is called, the gradients are accumulated rather than replaced.\n", "\n", "\n", "\n", "## Using Pytorch's Optimizer class\n", "\n", "In order to provide useful abstractions for an optimization algorithm, PyTorch provides the Optimizer interface as well as different classes implementing well known methods like SGD or ADAM.\n", "\n", "An optimizer takes as input a list of variables for an optimization problem (usually the parameters of a PyTorch module), and the hyperparameters of the optimization algorithm. \n", "\n", "The Optimizer class works by calling two different methods `optimizer.zero_grad()` and `optimizer.step()`. The first one sets as zero all the gradients corresponding to the variables that were assigned at initialization (in this case, the network's parameters), and the second one updates the variables according to some rule. In the case of SGD this just means subtracting a multiple of the gradient from each variable. For example if we want to train the weights of our neural network with SGD and learning rate (step size) 0.1 we would initialize an optimizer and write a typical training loop as follows:\n" ] }, { "cell_type": "code", "execution_count": 7, "metadata": {}, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "Loss: 0.5223714760442575\n", "Loss: 0.2764257527887821\n", "Loss: 0.22273141620059808\n", "Loss: 0.1863413620988528\n", "Loss: 0.16030733563626806\n", "Loss: 0.1402264552252988\n", "Loss: 0.12440013649562995\n", "Loss: 0.11180179509955147\n", "Loss: 0.10145463613482814\n", "Loss: 0.09289616520516575\n" ] }, { "data": { "image/png": "iVBORw0KGgoAAAANSUhEUgAAAjcAAAGwCAYAAABVdURTAAAAOXRFWHRTb2Z0d2FyZQBNYXRwbG90bGliIHZlcnNpb24zLjkuMiwgaHR0cHM6Ly9tYXRwbG90bGliLm9yZy8hTgPZAAAACXBIWXMAAA9hAAAPYQGoP6dpAAA9VklEQVR4nO3deXiU5aH//8/MJJnsO0kIBEKICgHZSQipSyuCttXiUqnFQmmrp25Hm9PzrRyP4I7WpV4tCELFasWKXaz+bAtojiuERTbZEQJJCGQn+z4zvz+SDKbACGGSZ5b367rmQp55ZvKZk+p8zv3cz32bHA6HQwAAAD7CbHQAAAAAd6LcAAAAn0K5AQAAPoVyAwAAfArlBgAA+BTKDQAA8CmUGwAA4FMCjA7Q3+x2u44fP66IiAiZTCaj4wAAgHPgcDhUX1+v5ORkmc2ux2b8rtwcP35cKSkpRscAAAC9UFxcrMGDB7s8x+/KTUREhKTO/+NERkYanAYAAJyLuro6paSkOL/HXfG7ctN9KSoyMpJyAwCAlzmXKSVMKAYAAD6FcgMAAHwK5QYAAPgUyg0AAPAplBsAAOBTKDcAAMCnUG4AAIBPodwAAACfQrkBAAA+hXIDAAB8CuUGAAD4FMoNAADwKZQbN6pqaNX+0jqjYwAA4NcoN26ydk+pJj7+gX71ly+MjgIAgF+j3LjJmMFRkqRdJbWqbW43OA0AAP6LcuMmA6NClBYfJrtD2lhQZXQcAAD8FuXGjaamx0mS8g9TbgAAMArlxo2mDo+XJK0/VGlwEgAA/Bflxo2y0+JkMklfljeovK7F6DgAAPglyo0bxYQFKWNgpCQpn3k3AAAYgnLjZlOHd8674dIUAADGoNy42dT07nk3VXI4HAanAQDA/1Bu3CwzNVYBZpNKappVXN1sdBwAAPwO5cbNwqwBGpcSLUlaf5hLUwAA9DfKTR84dWmKcgMAQH+j3PSBnOGnFvNj3g0AAP2LctMHxg2JVnCgWVWNbTpQVm90HAAA/Arlpg9YAyyanBorSdpwiPVuAADoTx5RbpYsWaLU1FQFBwcrKytLmzdvPuu5f/jDH2QymXo8goOD+zHtucnpmnezgUnFAAD0K8PLzerVq5Wbm6uFCxdq27ZtGjt2rGbMmKHy8vKzviYyMlInTpxwPgoLC/sx8bnpXsxvU0G1Omx2g9MAAOA/DC83zz//vG6//XbNmzdPGRkZWrZsmUJDQ7Vy5cqzvsZkMikpKcn5SExMPOu5ra2tqqur6/HoD6OSoxQZHKD61g7tKqntl58JAAAMLjdtbW3aunWrpk2b5jxmNps1bdo05efnn/V1DQ0NGjp0qFJSUvS9731Pe/bsOeu5ixYtUlRUlPORkpLi1s9wNhazSdldozcbDjPvBgCA/mJouamsrJTNZjtt5CUxMVGlpaVnfM0ll1yilStX6p133tHrr78uu92uqVOn6tixY2c8f/78+aqtrXU+iouL3f45zmbqcNa7AQCgvwUYHeB8ZWdnKzs72/n3qVOnauTIkXrppZf02GOPnXa+1WqV1Wrtz4hOOemdIzefF55US7tNwYEWQ3IAAOBPDB25iY+Pl8ViUVlZWY/jZWVlSkpKOqf3CAwM1Pjx43Xo0KG+iHhBhg8IV0KEVW0ddm0rPGl0HAAA/IKh5SYoKEgTJ05UXl6e85jdbldeXl6P0RlXbDabdu3apYEDB/ZVzF4zmUzOu6bYZwoAgP5h+N1Subm5WrFihV599VXt27dPd955pxobGzVv3jxJ0pw5czR//nzn+Y8++qjWrVungoICbdu2TbfddpsKCwv1s5/9zKiP4NJU53o3TCoGAKA/GD7nZtasWaqoqNCCBQtUWlqqcePGac2aNc5JxkVFRTKbT3WwkydP6vbbb1dpaaliYmI0ceJEbdiwQRkZGUZ9BJe6R26+OFar+pZ2RQQHGpwIAADfZnL42c6OdXV1ioqKUm1trSIjI/vlZ17xzIcqrGrS7+dM0rSMs6/JAwAAzux8vr8NvyzlD7pvCefSFAAAfY9y0w+6bwlnnykAAPoe5aYfZKd1lpv9pfWqbGg1OA0AAL6NctMP4sKtGpEUIUnK59IUAAB9inLTT3Kct4RzaQoAgL5EueknzsX8DjFyAwBAX6Lc9JPMYbGymE0qqm5ScXWT0XEAAPBZlJt+EhEcqLGDoyQx7wYAgL5EuelH3evdsM8UAAB9h3LTj6Y617upkp8tDA0AQL+h3PSjCUNiZA0wq6K+VYfKG4yOAwCAT6Lc9KPgQIsmpcZIktYf4tIUAAB9gXLTz9hnCgCAvkW56Wfdi/ltLKiSzc68GwAA3I1y089GJ0cqwhqgupYO7Tlea3QcAAB8DuWmnwVYzMpKY7ViAAD6CuXGADnOW8KZVAwAgLtRbgzQPal4y9FqtXbYDE4DAIBvodwY4OLEcMWHW9XSbtf2ohqj4wAA4FMoNwYwmUzOXcI3sN4NAABuRbkxiLPcsN4NAABuRbkxSPd6NzuKa9TY2mFwGgAAfAflxiApsaFKiQ1Rh92hzUeqjY4DAIDPoNwYaGpa91YMzLsBAMBdKDcGmprOYn4AALgb5cZA3evd7D1Rp+rGNoPTAADgGyg3BhoQYdXFieGSOjfSBAAAF45yY7Du0Zv1rHcDAIBbUG4Mxno3AAC4F+XGYFlpcTKbpCOVjTpe02x0HAAAvB7lxmBRIYG6dHC0JEZvAABwB8qNB2CfKQAA3Idy4wFyhncv5lclh8NhcBoAALwb5cYDTEqNUZDFrNK6FhVUNhodBwAAr0a58QDBgRZNGBotiUtTAABcKMqNh/jqpSkAANB7lBsPMTW9s9zkF1TJbmfeDQAAvUW58RBjBkcpLMiimqZ27T1RZ3QcAAC8FuXGQwRazMpK616tmHk3AAD0FuXGg3Svd7P+EPNuAADoLcqNB+neRHPL0Wq1ddgNTgMAgHei3HiQEUkRig0LUlObTTuP1RgdBwAAr0S58SBms0nZzktTzLsBAKA3KDcexrnPFOvdAADQK5QbD9O9mN/2opNqauswOA0AAN6HcuNhhsaFalB0iNptDm05etLoOAAAeB3KjYcxmU7Nu2G9GwAAzh/lxgPlpHeVG9a7AQDgvFFuPFD3eje7j9eqpqnN4DQAAHgXyo0HSowM1vABYXI4pI0F1UbHAQDAq1BuPFRO1y7hzLsBAOD8UG48VPelKRbzAwDg/FBuPNSUtFiZTNLhikaV1bUYHQcAAK9BufFQ0aFBGp0cJYlLUwAAnA/KjQebmt69zxS3hAMAcK4oNx6se95N/uEqORwOg9MAAOAdKDcebHJqjAItJpXUNKuwqsnoOAAAeAXKjQcLDQrQ+CExkqT1zLsBAOCcUG483FTnPlPMuwEA4FxQbjxc92J++YerZLcz7wYAgK9DufFwYwdHKzTIourGNu0vrTc6DgAAHo9y4+GCAsyanBorifVuAAA4F5QbL5CTzrwbAADOFeXGC3Svd7OpoErtNrvBaQAA8GyUGy+QMTBS0aGBamyz6YtjtUbHAQDAo1FuvIDZbFJ2WtelKXYJBwDAJcqNl5jadUs4824AAHDNI8rNkiVLlJqaquDgYGVlZWnz5s3n9Lo333xTJpNJM2fO7NuAHqB7Mb+tRSfV0m4zOA0AAJ7L8HKzevVq5ebmauHChdq2bZvGjh2rGTNmqLy83OXrjh49ql/+8pe67LLL+impsdLiw5QUGay2Drs+P3rS6DgAAHgsw8vN888/r9tvv13z5s1TRkaGli1bptDQUK1cufKsr7HZbJo9e7YeeeQRpaWl9WNa45hMJk113hLOvBsAAM7G0HLT1tamrVu3atq0ac5jZrNZ06ZNU35+/llf9+ijjyohIUE//elPv/ZntLa2qq6ursfDW3XfEr6eeTcAAJyVoeWmsrJSNptNiYmJPY4nJiaqtLT0jK/57LPP9PLLL2vFihXn9DMWLVqkqKgo5yMlJeWCcxulezG/XcdqVNvcbnAaAAA8k+GXpc5HfX29fvSjH2nFihWKj48/p9fMnz9ftbW1zkdxcXEfp+w7A6NClBYfJrtD2nyk2ug4AAB4pAAjf3h8fLwsFovKysp6HC8rK1NSUtJp5x8+fFhHjx7Vdddd5zxmt3eu2BsQEKADBw5o+PDhPV5jtVpltVr7IL0xsofHqaCyUesPVerqjMSvfwEAAH7G0JGboKAgTZw4UXl5ec5jdrtdeXl5ys7OPu38ESNGaNeuXdqxY4fzcf311+ub3/ymduzY4dWXnM5VjnO9GyYVAwBwJoaO3EhSbm6u5s6dq0mTJikzM1MvvPCCGhsbNW/ePEnSnDlzNGjQIC1atEjBwcEaPXp0j9dHR0dL0mnHfVX3SsUHyxpUUd+qARG+MyoFAIA7GF5uZs2apYqKCi1YsEClpaUaN26c1qxZ45xkXFRUJLPZq6YG9amYsCBlDIzU3hN12nC4Ut8bN8joSAAAeBSTw+FwGB2iP9XV1SkqKkq1tbWKjIw0Ok6vPPGPvVrx6RHNmpSip28eY3QcAAD63Pl8fzMk4oWc+0wVMO8GAIB/R7nxQpmpsQowm1Rc3azi6iaj4wAA4FEoN14ozBqgcSnRkqT1hxi9AQDgqyg3Xsp5aYqtGAAA6IFy46WmDu/eRLNKfjYnHAAAlyg3Xmr8kGgFB5pV2dCqg2UNRscBAMBjUG68lDXAosmpsZJYrRgAgK+i3HixqcM7592sP8S8GwAAulFuvFhOeue8m00FVeqw2Q1OAwCAZ6DceLFRyVGKDA5QfWuHdh+vMzoOAAAegXLjxSxmk6Z0baTJejcAAHSi3Hi5HOd6N5QbAAAkyo3X617v5vOjJ9XSbjM4DQAAxqPceLn0hHAlRFjV2mHXtqKTRscBAMBwlBsvZzKZTq1WzC3hAABQbnxB93o3zLsBAIBy4xOmdq13s/NYrepb2g1OAwCAsSg3PmBwTKiGxoXKZndoy9Fqo+MAAGAoyo2P6J53w1YMAAB/R7nxEaf2mWLeDQDAv1FufER218jN/tJ6VTW0GpwGAADjUG58RHy4VSOSIiRJ+QVcmgIA+C/KjQ85dWmKcgMA8F+UGx+S03VLeD7r3QAA/BjlxodkDouVxWzS0aomldQ0Gx0HAABDUG58SERwoMYMjpLEXVMAAP9FufExOV3zbvIPM+8GAOCfKDc+5tRifpVyOBwGpwEAoP9RbnzMhKExsgaYVV7fqsMVDUbHAQCg31FufExwoEWTUmMkSRu4NAUA8EOUGx/EVgwAAH9GufFB3fNu8g9XyWZn3g0AwL9QbnzQpYOiFGENUF1Lh/YerzM6DgAA/Ypy44MCLGZlpcVKktazWjEAwM9QbnwU824AAP6KcuOjctI7y82Wo9Vq67AbnAYAgP5DufFRFyeGKz48SC3tdm0vOml0HAAA+g3lxkeZTCZld1+aYr0bAIAfodz4sBznLeHMuwEA+A/KjQ/rnlS8vahGja0dBqcBAKB/UG582JC4UA2OCVGH3aHNR6uNjgMAQL+g3Pi4nK7Rm3zm3QAA/ATlxsdNTe+cd8N6NwAAf0G58XHZXZOK956o08nGNoPTAADQ9yg3Pi4hIlgXJ4bL4ZA2FnBpCgDg+yg3fsC5FQO3hAMA/ADlxg9M7bo0tYFJxQAAP0C58QNZaXEym6SCikaV1rYYHQcAgD5FufEDUSGBunRQlCTumgIA+D7KjZ+Y2rVLOJemAAC+jnLjJ7oX89twuFIOh8PgNAAA9B3KjZ+YODRGQRazTtS26Ehlo9FxAADoM5QbPxESZNGEodGSuDQFAPBtlBs/8tVLUwAA+KpelZvi4mIdO3bM+ffNmzfr/vvv1/Lly90WDO7Xvc9U/uEq2e3MuwEA+KZelZsf/vCH+vDDDyVJpaWluvrqq7V582Y9+OCDevTRR90aEO4zZnC0woIsOtnUrn2ldUbHAQCgT/Sq3OzevVuZmZmSpLfeekujR4/Whg0btGrVKv3hD39wZz64UaDFrKy0rtWKDzHvBgDgm3pVbtrb22W1WiVJH3zwga6//npJ0ogRI3TixAn3pYPbdW/FwD5TAABf1atyM2rUKC1btkyffvqp3n//fV1zzTWSpOPHjysuLs6tAeFe3Ztobj5SrXab3eA0AAC4X6/KzdNPP62XXnpJV155pW699VaNHTtWkvTuu+86L1fBM41IilBsWJCa2mzaWVxjdBwAANwuoDcvuvLKK1VZWam6ujrFxMQ4j99xxx0KDQ11Wzi4n9lsUnZanP6x64TWH6rSpNRYoyMBAOBWvRq5aW5uVmtrq7PYFBYW6oUXXtCBAweUkJDg1oBwv+5bwlnvBgDgi3pVbr73ve/ptddekyTV1NQoKytLzz33nGbOnKmlS5e6NSDcr3sxv+1FNWpusxmcBgAA9+pVudm2bZsuu+wySdJf/vIXJSYmqrCwUK+99pp++9vfujUg3G9oXKiSo4LVZrNry9Fqo+MAAOBWvSo3TU1NioiIkCStW7dON954o8xms6ZMmaLCwkK3BoT7mUwmTU3v3oqB9W4AAL6lV+UmPT1df//731VcXKy1a9dq+vTpkqTy8nJFRka6NSD6Rg7zbgAAPqpX5WbBggX65S9/qdTUVGVmZio7O1tS5yjO+PHj3RoQfaN7vZtdJbWqbWo3OA0AAO7Tq3Jz8803q6ioSJ9//rnWrl3rPH7VVVfpN7/5zXm/35IlS5Samqrg4GBlZWVp8+bNZz33b3/7myZNmqTo6GiFhYVp3Lhx+uMf/9ibj+HXEiODNXxAmBwOaeMRLk0BAHxHr8qNJCUlJWn8+PE6fvy4c4fwzMxMjRgx4rzeZ/Xq1crNzdXChQu1bds2jR07VjNmzFB5efkZz4+NjdWDDz6o/Px8ffHFF5o3b57mzZvXo2Th3OR0z7s5xKUpAIDv6FW5sdvtevTRRxUVFaWhQ4dq6NChio6O1mOPPSa7/fyW9H/++ed1++23a968ecrIyNCyZcsUGhqqlStXnvH8K6+8UjfccINGjhyp4cOH67777tOYMWP02Wef9eaj+LXufaaYVAwA8CW9KjcPPvigFi9erKeeekrbt2/X9u3b9eSTT+p3v/udHnrooXN+n7a2Nm3dulXTpk07Fchs1rRp05Sfn/+1r3c4HMrLy9OBAwd0+eWXn/Gc1tZW1dXV9Xig05S0OJlM0pflDSqvazE6DgAAbtGr7RdeffVV/f73v3fuBi5JY8aM0aBBg3TXXXfpiSeeOKf3qayslM1mU2JiYo/jiYmJ2r9//1lfV1tbq0GDBqm1tVUWi0Uvvviirr766jOeu2jRIj3yyCPnlMffRIcGaVRypHaX1GnD4SrNHD/I6EgAAFywXo3cVFdXn3FuzYgRI1Rd3feLwkVERGjHjh3asmWLnnjiCeXm5uqjjz4647nz589XbW2t81FcXNzn+bxJ92rF3BIOAPAVvSo3Y8eO1eLFi087vnjxYo0ZM+ac3yc+Pl4Wi0VlZWU9jpeVlSkpKemsrzObzUpPT9e4ceP0X//1X7r55pu1aNGiM55rtVoVGRnZ44FTuhfzW3+oSg6Hw+A0AABcuF5dlvr1r3+t73znO/rggw+ca9zk5+eruLhY//znP8/5fYKCgjRx4kTl5eVp5syZkjonK+fl5emee+455/ex2+1qbW09r8+ATpNTYxRoMamkpllF1U0aGhdmdCQAAC5Ir0ZurrjiCh08eFA33HCDampqVFNToxtvvFF79uw57zVncnNztWLFCr366qvat2+f7rzzTjU2NmrevHmSpDlz5mj+/PnO8xctWqT3339fBQUF2rdvn5577jn98Y9/1G233dabj+L3QoMCND6lc3d37poCAPiCXo3cSFJycvJpE4d37typl19+WcuXLz/n95k1a5YqKiq0YMEClZaWaty4cVqzZo1zknFRUZHM5lMdrLGxUXfddZeOHTumkJAQjRgxQq+//rpmzZrV24/i96amx2nz0WqtP1SpWzOHGB0HAIALYnK4caLFzp07NWHCBNlsNne9pdvV1dUpKipKtbW1zL/psvlItW55KV9xYUHa8uA0mc0moyMBANDD+Xx/93qFYviOcSnRCgm0qKqxTQfL642OAwDABaHcQEEBZmUOi5XUedcUAADe7Lzm3Nx4440un6+pqbmQLDDQ1OFx+vhghTYcqtRPvzHM6DgAAPTaeZWbqKior31+zpw5FxQIxujeRHPTkWp12OwKsDCoBwDwTudVbl555ZW+ygGDZQyMVFRIoGqb2/VFSa0mDIkxOhIAAL3C/3sOSZLZbFJ2Wtcu4YfYigEA4L0oN3DKSe8qNyzmBwDwYpQbOHXvM/V54Um1tHvuWkUAALhCuYFTWnyYEiOtauuwa2vhSaPjAADQK5QbOJlMJuUM7xy92XCYeTcAAO9EuUEP3ZemPj5YIZvdbTtzAADQbyg36OEb6fEKMJu0u6RO//HHrWps7TA6EgAA54Vygx6SooL1/KxxCgow64N9Zbpp6QaV1DQbHQsAgHNGucFprh+brDfvmKL4cKv2l9bre4vXa1sRE4wBAN6BcoMzmjAkRu/ck6ORAyNV2dCqHyzfqHd2lBgdCwCAr0W5wVkNig7RX36erWkjE9XWYdd9b+7Q8+sOyM5EYwCAB6PcwKUwa4Be+tFE/cflaZKk3/7fId37p+1qbmORPwCAZ6Lc4GtZzCbN//ZI/frmMQq0mPSPXSc0a3m+yupajI4GAMBpKDc4Z7dMStGqn01RTGigvjhWq+sXf6bdJbVGxwIAoAfKDc5L5rBYvXP3N5SeEK6yulbdvGyD1uw+YXQsAACcKDc4b0PiQvW3u6bq8osHqKXdrp+/vk1LPjwkh4OJxgAA41Fu0CuRwYFaOXeSfjw1VZL0zNoDyn1rJ7uJAwAMR7lBrwVYzHr4+lF6fOZoWcwmvb29RLN/v0mVDa1GRwMA+DHKDS7YbVOG6tV5mYoMDtDWwpP63uL12l9aZ3QsAICfotzALb5xUbzevjtHw+LDVFLTrJte3KC8fWVGxwIA+CHKDdxm+IBwvX3XVGWnxamxzaafvfa5fv9pARONAQD9inIDt4oODdJrP83UrZkpcjikx/+xT/P/tkttHXajowEA/ATlBm4XaDHryRsu1UPfzZDZJL25pVg/enmTTja2GR0NAOAHKDfoEyaTST/9xjC9PHeywq0B2nSkWjNfXK9D5Q1GRwMA+DjKDfrUN0ck6K93TtXgmBAVVjXphhfX69MvK4yOBQDwYZQb9LlLkiL0zt05mjQ0RvUtHfrxK1v0Wv5Ro2MBAHwU5Qb9Ii7cqlW3Z+nGCYNkszu04J09WvDObnXYmGgMAHAvyg36jTXAoue+P1a/umaETCbptfxCzfvDFtU2txsdDQDgQyg36Fcmk0l3Xjlcy26bqJBAiz79slI3vrheRysbjY4GAPARlBsYYsaoJP3559kaGBWswxWNmvniem0sqDI6FgDAB1BuYJjRg6L0zt05GpsSrZqmdv3o5U16a0ux0bEAAF6OcgNDJUQGa/UdU/TdMQPVbnPo//31Cz35z32y2dmyAQDQO5QbGC440KLf3Tpe9111kSRp+ScFuuO1z9XQ2mFwMgCAN6LcwCOYTCb94uqL9dtbx8saYFbe/nLdvHSDjp1sMjoaAMDLUG7gUa4fm6zV/5GtARFW7S+t18wl67W18KTRsQAAXoRyA48zLiVa79ydo4yBkapsaNOtyzfq79tLjI4FAPASlBt4pOToEP3559manpGoNptd96/eoWfXHpCdicYAgK9BuYHHCrMGaNltE3XnlcMlSYs/PKS739impjYmGgMAzo5yA49mNpv0q2tG6Nnvj1WgxaR/7S7VLS/lq7S2xehoAAAPRbmBV7h54mC9cfsUxYYFaXdJnb635DPtOlZrdCwAgAei3MBrTE6N1Tt35+jixHCV1bXq+y9t0D93nTA6FgDAw1Bu4FVSYkP11zun6spLBqil3a67Vm3T7/K+lMPBRGMAQCfKDbxORHCgfj9nkn6SM0yS9Nz7B3X/6h1qabcZnAwA4AkoN/BKARazFlyXoSduGK0As0nv7DiuW1dsVEV9q9HRAAAGo9zAq83OGqrXfpKpqJBAbS+q0cwl67XvRJ3RsQAABqLcwOtNTY/X23dNVVp8mEpqmnXT0g36YG+Z0bEAAAah3MAnpA0I19t35SgnPU5NbTbd/sfPtfyTw0w0BgA/RLmBz4gKDdQf5mVqdtYQORzSk//cr/96a6eqG9uMjgYA6EeUG/iUQItZj88crYXXZchskv62vURXPPOhfv9pgdo67EbHAwD0A8oNfI7JZNK8nGFa/R/ZGpUcqfqWDj3+j32a/puPtW5PKZeqAMDHmRx+9l/6uro6RUVFqba2VpGRkUbHQR+z2R3667ZjembtAedt4tlpcXrouxnKSOb3DwDe4ny+vyk38AsNrR1a9tFhLe+6PGUySbMmpei/pl+iARFWo+MBAL4G5cYFyo1/O3aySU+vOaD/b+dxSVK4NUB3fXO4fpIzTMGBFoPTAQDOhnLjAuUGkrS1sFqPvrdPO4trJEmDY0I0/9qR+valSTKZTMaGAwCchnLjAuUG3ex2h97ZWaKn/3VApXUtkqTJqTF66LsZGjM42thwAIAeKDcuUG7w75raOrT8kwK99HGBmrs237xxwiD9vxkjlBQVbHA6AIBEuXGJcoOzOVHbrGfWHNDftpdIkkICLfr5FcN1x+VpCgliPg4AGIly4wLlBl9nZ3GNHn1vr7YWnpQkDYwK1q+uGaHrxybLbGY+DgAYgXLjAuUG58LhcOgfu05o0T/3q6SmWZI0LiVaD303QxOHxhicDgD8D+XGBcoNzkdLu00vf3ZEL354SI1tnfNxrhubrF9dc4kGx4QanA4A/AflxgXKDXqjvL5Fz609qLe2FsvhkKwBZt1+WZruvHK4wqwBRscDAJ9HuXGBcoMLsbukVo//Y682FlRLkhIirPrljEt084TBzMcBgD5EuXGBcoML5XA4tG5vmZ785z4VVjVJkkYPitRD38lQVlqcwekAwDedz/e3R+wKvmTJEqWmpio4OFhZWVnavHnzWc9dsWKFLrvsMsXExCgmJkbTpk1zeT7gbiaTSTNGJWndLy7X/3x7hCKsAdpdUqdZyzfqzte3qqir8AAAjGF4uVm9erVyc3O1cOFCbdu2TWPHjtWMGTNUXl5+xvM/+ugj3Xrrrfrwww+Vn5+vlJQUTZ8+XSUlJf2cHP7OGmDRHZcP10f/faVumzJEZpP0r92lmvb8x1r0z32qa2k3OiIA+CXDL0tlZWVp8uTJWrx4sSTJbrcrJSVF9957rx544IGvfb3NZlNMTIwWL16sOXPmnPZ8a2urWltbnX+vq6tTSkoKl6XgdgdK6/X4P/bq0y8rJUlxYUHKnX6xfjB5iCzMxwGAC+I1l6Xa2tq0detWTZs2zXnMbDZr2rRpys/PP6f3aGpqUnt7u2JjY8/4/KJFixQVFeV8pKSkuCU78O8uSYrQaz/J1MofT1LagDBVNbbpwbd36zu//VSfdRUeAEDfM7TcVFZWymazKTExscfxxMRElZaWntN7/OpXv1JycnKPgvRV8+fPV21trfNRXFx8wbmBszGZTPrWiEStvf9yLbwuQ1EhgdpfWq/bXt6kn726RQUVDUZHBACf59ULdDz11FN688039dFHHyk4+MwbHFqtVlmt1n5OBn8XaDFrXs4w3TB+kF744Eu9vrFQH+wr10cHKjQnO1X3XXWRokIDjY4JAD7J0JGb+Ph4WSwWlZWV9TheVlampKQkl6999tln9dRTT2ndunUaM2ZMX8YEei06NEgPXz9Ka+6/XN8akaAOu0Mr1x/RFc9+qFc3HFW7zW50RADwOYaWm6CgIE2cOFF5eXnOY3a7XXl5ecrOzj7r637961/rscce05o1azRp0qT+iApckPSEcK388WS99pNMXZwYrpqmdi18d4+ueeETfXjgzHcGAgB6x/BbwXNzc7VixQq9+uqr2rdvn+688041NjZq3rx5kqQ5c+Zo/vz5zvOffvppPfTQQ1q5cqVSU1NVWlqq0tJSNTQwlwGe7/KLB+if/3mZHp85WrFhQTpc0ah5r2zRnJWbdbCs3uh4AOATDJ9zM2vWLFVUVGjBggUqLS3VuHHjtGbNGuck46KiIpnNpzrY0qVL1dbWpptvvrnH+yxcuFAPP/xwf0YHeiXAYtZtU4bqurHJWvLhIb2y/og+OVihaw9V6oeZQ/SLqy9WbFiQ0TEBwGsZvs5Nf2P7BXiao5WNWvSvfVq7p3PuWURwgO676iLNyU5VUIDhg6sA4BHYW8oFyg08Vf7hKj323l7tPVEnSUqNC9X/fHukrs5IlMnEIoAA/BvlxgXKDTyZze7QX7YW65m1B1XZ0LmydnZanB78zkiNHhRlcDoAMA7lxgXKDbxBQ2uHXvzwkH7/2RG1dXTeLp45LFazs4bomtFJsgZYDE4IAP2LcuMC5QbepLi6Sc+sPaD3vjgue9e/qbFhQfr+pMGanTlUQ+JCjQ0IAP2EcuMC5Qbe6ERts97cXKw3txSprO7URrCXXzxAs7OG6KoRCQqwMPkYgO+i3LhAuYE367DZlbe/XKs2FemTgxXO44mRVv1g8hD9IDNFA6NCDEwIAH2DcuMC5Qa+oqiqSW9sLtKfPy9WVWObJMlskq4amajZWUN0+UUDZDZzlxUA30C5cYFyA1/T2mHT2j1lWrWxUJuOVDuPp8SG6NbMIbplUoriw9k8FoB3o9y4QLmBLztUXq9Vm4r0163HVNfSIUkKtJg0Y1SSbpsyVFnDYlkzB4BXoty4QLmBP2hus+m9L45r1aYi7SiucR4fPiBMs7OG6qYJgxUVGmhcQAA4T5QbFyg38De7S2q1alOR3tlRoqY2myTJGmDWdWOTNTtriMalRDOaA8DjUW5coNzAX9W3tOvvO45r1cZC7S89tQN5xsBIzZ4yRN8bN0jhVsP30gWAM6LcuEC5gb9zOBzaVlSjVZsK9d4XJ5wrIIdbAzRzfLJ+mDlUGcn8uwHAs1BuXKDcAKfUNLXpL1uP6Y1NRSqobHQenzAkWrOzhuo7YwYqOJCtHgAYj3LjAuUGOJ3D4VB+QZVWbSrS2t2l6uja6yEqJFA3TxysH2YN0fAB4QanBODPKDcuUG4A18rrW/TnzztHc0pqmp3Hs9PiNHvKEE3PSFJQAFs9AOhflBsXKDfAubHZHfrkywqt2lik/9tf5ty4Mz48SLdMStGtmUOUEsvGnQD6B+XGBcoNcP5Kapq1enOR3txSrPL6zo07TSbpiosHaHbWUH1rRIIsbPUAoA9Rblyg3AC9126zK29fmVZtKtKnX1Y6jydHBesHmUM0a3KKEiODDUwIwFdRblyg3ADucbSyUX/aXKS3Pi/WyaZ2SZLFbNLVIxM1e8oQ5QyPZ+NOAG5DuXGBcgO4V0u7TWv3lGrVxiJtPnpq486hcaH6YeYQ3TxxsOLYuBPABaLcuEC5AfrOwbJ6vdG1cWd9a+fGnUEWs669NEmzs4ZqcmoMWz0A6BXKjQuUG6DvNbV16L2dJ/T6pkJ9cazWeXz4gDB9+9KBmjEqSaOSIyk6AM4Z5cYFyg3Qv744VqM3NhXpnR3H1dxucx5PjgrW9FFJmj4qUZmpsQqwsHYOgLOj3LhAuQGMUdfSrrx9ZVq3p0wfHajoUXSiQwP1rREJmp6RpCsuHqCQILZ8ANAT5cYFyg1gvJZ2mz77slLr9pbqg33lqm5scz4XHGjWZRcN0PSMRE0bmaiYsCADkwLwFJQbFyg3gGex2R36/Gi11u0t09o9pTp28tSWD2aTlDksVtMzknR1RiIrIgN+jHLjAuUG8FwOh0P7TtRr3d5SrdtTpr0n6no8Pyo5UtMzOufpjEiKYEIy4EcoNy5QbgDvUVzdpHV7y7RuT6m2HK127m8lSUNiQzU9I1HTRyVp4tAYtn8AfBzlxgXKDeCdqhvb9EHXhORPv6xQa4fd+VxcWJCuGpmgGaOSlJMer+BAJiQDvoZy4wLlBvB+TW0d+uRgpdbtKVXe/nLVNrc7nwsNsuiKiwdoxqgkffOSBEWFBhqYFIC7UG5coNwAvqXdZteWI9XOy1fHa1uczwWYTZqSFqfpoxJ1dUaiBkaFGJgUwIWg3LhAuQF8l8Ph0O6SOq3bW6q1e0p1sKyhx/NjB0d1LhyYkaj0hHAmJANehHLjAuUG8B9HKhv1ftedV1uLTuqr/7VLiw/T1aMSNT0jSeNTotnBHPBwlBsXKDeAfyqvb1HevnKt21Oq9Yeq1GY7NSF5QIRVV2ckanpGorKHx8kawIRkwNNQblyg3ACob2nXxwcrtG5PmT7cX+7cwVySIqwBunJEgqZnJOrKSwYoIpgJyYAnoNy4QLkB8FVtHXblF1Rp3Z5Svb+3TOX1rc7nAi0mTR0erxmjkjQtI0EJEcEGJgX8G+XGBcoNgLOx2x3acaxG6/Z03nlVUNnofM5kksanRGv6qM6tINLiw5iQDPQjyo0LlBsA5+pQeYPW7inVur1l2llc0+O5pMhgZaXFKmtYnLLSYik7QB+j3LhAuQHQG6W1LZ13Xu0t08aCKrXbev6nc0CEVVnDYpWVFqcpw2K51RxwM8qNC5QbABequc2m7UUntfFItTYVVGl7cY3avrIdhCTFhwcpc9ipkZ2LEyK43Ry4AJQbFyg3ANytpd2mHcU12lRQrU1HqrS18GSPva8kKSY0sEfZGZkUSdkBzgPlxgXKDYC+1tph0xfHarWpoEqbjlTr86Mn1dxu63FOVEigJqfGakrXvJ2M5Eh2NgdcoNy4QLkB0N/aOuzaVVKrTUeqtLGgWluPVquxrWfZiQgO0OTUWOe8ndHJkQqwmA1KDHgeyo0LlBsARuuw2bX7eJ02FVRpY0GVPj96ssdCgpIUbg3QxKExykqL1ZS0OF06KEqBlB34McqNC5QbAJ7GZndo7/E6bSyo0qYjVdp8pFp1LT3LTmiQpbPsDOssO2MGRysogLID/0G5cYFyA8DT2ewO7S+t08aCzruxNh+tVk1Te49zggPNmjAkRlnD4jQlLVZjU6IVHMieWPBdlBsXKDcAvI3d7tDB8nptPNw5QXnTkWpVN7b1OCcowKzxKdGaktZ5N9aEITGUHfgUyo0LlBsA3s7hcOhQeYM2FlR1rbVTrcqG1h7nBFnMGpsS1Vl2hsVpwtBohQYFGJQYuHCUGxcoNwB8jcPhUEFlY+ecna61dsrqepadALNJYwZ3lZ20OE0aGqMwK2UH3oNy4wLlBoCvczgcKqxq6pqg3Dlv53htS49zLGaTRg+K0tjBURqdHKXRg6J0UWI4d2TBY1FuXKDcAPA3DodDx042K/8rIzvHTjafdl5QgFkjkiI0elB34YnUJUkRsgYwdwfGo9y4QLkBAOnYySZtLTyp3SW12l1Sp93Ha1X/b7efS52Xsy5OjNDoQZG6dFCURg2KUsbASCYro99Rblyg3ADA6ex2h4pPNml3SZ12ldRqz/Fa7SqpPe0WdKnzklb6gHCN6io8o7sKD3N40JcoNy5QbgDg3DgcDpXUNHeO7JTUavfxWu0uqVVlQ9tp55pMUlp82FcuaUVp1KBIRQYHGpAcvohy4wLlBgB6z+FwqKyutUfZ2V1Sp9K6ljOenxoXqlFdhefSQVEalRypmLCgfk4NX0C5cYFyAwDuV1Hfqt3Ha7Wnq+zsKqlVSc3pk5YlaVB0SNflrEiNGtRZeuLDrf2cGN6GcuMC5QYA+sfJxrau0Z065yhPYVXTGc9NigzuvKQ1KLJzlGdwlBIirDKZTP2cGp6KcuMC5QYAjFPb3K69x3vO4SmobNSZvoniw62n7tLqujV9UHQIhcdPUW5coNwAgGdpaO3QvhOdhWdXSa32lNTpy/J62c/w7RQTGtg1wnNqLZ4hsaEUHj9AuXGBcgMAnq+5zaZ9pXXa01V4dpfU6WBZvTrO0HgiggN0cWKE0geEKz0hXOmJ4UofEK5B0SEymyk9voJy4wLlBgC8U2uHTQdK63vM4dl/ol5tNvsZzw8JtGh4QpjSB4TrosQIDe8qP0PjQtlmwgtRblyg3ACA72i32XWovEFfljfoUHmDDpc36Mvyeh2pbFS77cxfb4EWk1LjwpSeEK6LEsI1PKGz9AwfEM7Kyx6McuMC5QYAfF+Hza6i6iZn6el+HK5oUFOb7YyvMZmklJhQXdRddrrKT3pCuCJYjNBwlBsXKDcA4L/sdoeO1zb3KDzdIz+1zadvNdEtMdKqixIiTis9cWFBTGbuJ5QbFyg3AIB/53A4VNnQ1lV26jv/rGjQl2UNKq9vPevrYkIDOycxJ4Qrvav8pCeEKzkqmNLjZpQbFyg3AIDzUdvcrsMVDTpU1l146nWookHHTjafcX0eSQoLsnTO5Rlw6u6tixIjlBITogAmM/eKV5WbJUuW6JlnnlFpaanGjh2r3/3ud8rMzDzjuXv27NGCBQu0detWFRYW6je/+Y3uv//+8/p5lBsAgDs0t9l0uKJzHs+h8s5RnkMVDTpa2XjGW9YlKchi1rD4MGfhSU8I10WJ4RoWHyZrAJOZXTmf729D96dfvXq1cnNztWzZMmVlZemFF17QjBkzdODAASUkJJx2flNTk9LS0vT9739fv/jFLwxIDABAp5Agi3NBwa9qt9lVWNXYo/B0T2ZuabfrQFm9DpTV93iN2SQNiQ1VanyYhsSGnnrEhSolJlRhVkO/rr2OoSM3WVlZmjx5shYvXixJstvtSklJ0b333qsHHnjA5WtTU1N1//33f+3ITWtrq1pbT10vraurU0pKCiM3AIB+Zbc7VFLT3DWBub7HLez1LR0uXxsfHqSU2FAN7So9KV1/Do0LU0KE1S8WK/SKkZu2tjZt3bpV8+fPdx4zm82aNm2a8vPz3fZzFi1apEceecRt7wcAQG+YzSaldBWTb444dXXC4XCoor5Vh8obVFjdpKKuR3HXnzVN7apsaFNlQ5u2F9Wc9r5BAWalxIR8ZbTn1OhPSmyIQoP8b9THsE9cWVkpm82mxMTEHscTExO1f/9+t/2c+fPnKzc31/n37pEbAAA8gclkUkJksBIigzX1DM/XNrc7i05RdZMKq04Vn5KaZrV12HW4olGHKxrP+P7x4VYNjes54tM56hOqAeG+Oerj83XOarXKarUaHQMAgF6JCglU1Bnm9kidixWeqG1RYdXpIz6FVY2qa+lQZUOrKhtatbXw5GmvtwaYexSef5/rExLknZOcDSs38fHxslgsKisr63G8rKxMSUlJBqUCAMB7BFjMzktdZ1Lb1H5qxKe6sccI0PGaFrV22J0LGZ5JQoT1K5e4Qp0jQENiQzUgwuqxa/kYVm6CgoI0ceJE5eXlaebMmZI6JxTn5eXpnnvuMSoWAAA+Iyo0UJeGRunSwaeP+rTb7Dpe0+wsO0XVTSqqOvVnfWuHyutbVV7fqs/PMOoTHGhWSkxn4fn30Z+U2FBD9+ky9LJUbm6u5s6dq0mTJikzM1MvvPCCGhsbNW/ePEnSnDlzNGjQIC1atEhS5yTkvXv3Ov+5pKREO3bsUHh4uNLT0w37HAAAeJtAi1lD48I0NC7stOccDodqm9udc3y+ermrc9SnWS3tdn3ZdcfXv0tPCNcHuVf0x8c4I0PLzaxZs1RRUaEFCxaotLRU48aN05o1a5yTjIuKimQ2n1rJ8fjx4xo/frzz788++6yeffZZXXHFFfroo4/6Oz4AAD7JZDIpOjRI0aFBGjM4+rTn2zpcjPpUN2nIWS6T9RfDVyjub6xQDABA33E4HGrtsLv9stT5fH+zwQUAAHAbk8lk6HwbiXIDAAB8DOUGAAD4FMoNAADwKZQbAADgUyg3AADAp1BuAACAT6HcAAAAn0K5AQAAPoVyAwAAfArlBgAA+BTKDQAA8CmUGwAA4FMoNwAAwKcEGB2gvzkcDkmdW6cDAADv0P293f097orflZv6+npJUkpKisFJAADA+aqvr1dUVJTLc0yOc6lAPsRut+v48eOKiIiQyWRy63vX1dUpJSVFxcXFioyMdOt74/zx+/As/D48C78Pz8PvxDWHw6H6+nolJyfLbHY9q8bvRm7MZrMGDx7cpz8jMjKS/2F6EH4fnoXfh2fh9+F5+J2c3deN2HRjQjEAAPAplBsAAOBTKDduZLVatXDhQlmtVqOjQPw+PA2/D8/C78Pz8DtxH7+bUAwAAHwbIzcAAMCnUG4AAIBPodwAAACfQrkBAAA+hXLjJkuWLFFqaqqCg4OVlZWlzZs3Gx3Jby1atEiTJ09WRESEEhISNHPmTB04cMDoWOjy1FNPyWQy6f777zc6it8qKSnRbbfdpri4OIWEhOjSSy/V559/bnQsv2Sz2fTQQw9p2LBhCgkJ0fDhw/XYY4+d0/5JODvKjRusXr1aubm5WrhwobZt26axY8dqxowZKi8vNzqaX/r444919913a+PGjXr//ffV3t6u6dOnq7Gx0ehofm/Lli166aWXNGbMGKOj+K2TJ08qJydHgYGB+te//qW9e/fqueeeU0xMjNHR/NLTTz+tpUuXavHixdq3b5+efvpp/frXv9bvfvc7o6N5NW4Fd4OsrCxNnjxZixcvltS5f1VKSoruvfdePfDAAwanQ0VFhRISEvTxxx/r8ssvNzqO32poaNCECRP04osv6vHHH9e4ceP0wgsvGB3L7zzwwANav369Pv30U6OjQNJ3v/tdJSYm6uWXX3Yeu+mmmxQSEqLXX3/dwGTejZGbC9TW1qatW7dq2rRpzmNms1nTpk1Tfn6+gcnQrba2VpIUGxtrcBL/dvfdd+s73/lOj39X0P/effddTZo0Sd///veVkJCg8ePHa8WKFUbH8ltTp05VXl6eDh48KEnauXOnPvvsM1177bUGJ/NufrdxprtVVlbKZrMpMTGxx/HExETt37/foFToZrfbdf/99ysnJ0ejR482Oo7fevPNN7Vt2zZt2bLF6Ch+r6CgQEuXLlVubq7+53/+R1u2bNF//ud/KigoSHPnzjU6nt954IEHVFdXpxEjRshischms+mJJ57Q7NmzjY7m1Sg38Gl33323du/erc8++8zoKH6ruLhY9913n95//30FBwcbHcfv2e12TZo0SU8++aQkafz48dq9e7eWLVtGuTHAW2+9pVWrVumNN97QqFGjtGPHDt1///1KTk7m93EBKDcXKD4+XhaLRWVlZT2Ol5WVKSkpyaBUkKR77rlH7733nj755BMNHjzY6Dh+a+vWrSovL9eECROcx2w2mz755BMtXrxYra2tslgsBib0LwMHDlRGRkaPYyNHjtRf//pXgxL5t//+7//WAw88oB/84AeSpEsvvVSFhYVatGgR5eYCMOfmAgUFBWnixInKy8tzHrPb7crLy1N2draByfyXw+HQPffco7ffflv/93//p2HDhhkdya9dddVV2rVrl3bs2OF8TJo0SbNnz9aOHTsoNv0sJyfntKURDh48qKFDhxqUyL81NTXJbO75VWyxWGS32w1K5BsYuXGD3NxczZ07V5MmTVJmZqZeeOEFNTY2at68eUZH80t333233njjDb3zzjuKiIhQaWmpJCkqKkohISEGp/M/ERERp813CgsLU1xcHPOgDPCLX/xCU6dO1ZNPPqlbbrlFmzdv1vLly7V8+XKjo/ml6667Tk888YSGDBmiUaNGafv27Xr++ef1k5/8xOhoXo1bwd1k8eLFeuaZZ1RaWqpx48bpt7/9rbKysoyO5ZdMJtMZj7/yyiv68Y9/3L9hcEZXXnklt4Ib6L333tP8+fP15ZdfatiwYcrNzdXtt99udCy/VF9fr4ceekhvv/22ysvLlZycrFtvvVULFixQUFCQ0fG8FuUGAAD4FObcAAAAn0K5AQAAPoVyAwAAfArlBgAA+BTKDQAA8CmUGwAA4FMoNwAAwKdQbgAAgE+h3ADweyaTSX//+9+NjgHATSg3AAz14x//WCaT6bTHNddcY3Q0AF6KjTMBGO6aa67RK6+80uOY1Wo1KA0Ab8fIDQDDWa1WJSUl9XjExMRI6rxktHTpUl177bUKCQlRWlqa/vKXv/R4/a5du/Stb31LISEhiouL0x133KGGhoYe56xcuVKjRo2S1WrVwIEDdc899/R4vrKyUjfccINCQ0N10UUX6d133+3bDw2gz1BuAHi8hx56SDfddJN27typ2bNn6wc/+IH27dsnSWpsbNSMGTMUExOjLVu26M9//rM++OCDHuVl6dKluvvuu3XHHXdo165devfdd5Went7jZzzyyCO65ZZb9MUXX+jb3/62Zs+ererq6n79nADcxAEABpo7d67DYrE4wsLCejyeeOIJh8PhcEhy/PznP+/xmqysLMedd97pcDgcjuXLlztiYmIcDQ0Nzuf/8Y9/OMxms6O0tNThcDgcycnJjgcffPCsGSQ5/vd//9f594aGBockx7/+9S+3fU4A/Yc5NwAM981vflNLly7tcSw2Ntb5z9nZ2T2ey87O1o4dOyRJ+/bt09ixYxUWFuZ8PicnR3a7XQcOHJDJZNLx48d11VVXucwwZswY5z+HhYUpMjJS5eXlvf1IAAxEuQFguLCwsNMuE7lLSEjIOZ0XGBjY4+8mk0l2u70vIgHoY8y5AeDxNm7ceNrfR44cKUkaOXKkdu7cqcbGRufz69evl9ls1iWXXKKIiAilpqYqLy+vXzMDMA4jNwAM19raqtLS0h7HAgICFB8fL0n685//rEmTJukb3/iGVq1apc2bN+vll1+WJM2ePVsLFy7U3Llz9fDDD6uiokL33nuvfvSjHykxMVGS9PDDD+vnP/+5EhISdO2116q+vl7r16/Xvffe278fFEC/oNwAMNyaNWs0cODAHscuueQS7d+/X1LnnUxvvvmm7rrrLg0cOFB/+tOflJGRIUkKDQ3V2rVrdd9992ny5MkKDQ3VTTfdpOeff975XnPnzlVLS4t+85vf6Je//KXi4+N18803998HBNCvTA6Hw2F0CAA4G5PJpLffflszZ840OgoAL8GcGwAA4FMoNwAAwKcw5waAR+PKOYDzxcgNAADwKZQbAADgUyg3AADAp1BuAACAT6HcAAAAn0K5AQAAPoVyAwAAfArlBgAA+JT/H7O02dG6CYJ9AAAAAElFTkSuQmCC", "text/plain": [ "
" ] }, "metadata": {}, "output_type": "display_data" } ], "source": [ "from torch import optim\n", "import numpy as np\n", "\n", "optimizer = optim.SGD(net.parameters(), lr=0.1)\n", "loss_history = []\n", "\n", "for epoch in range(10):\n", " loss_avg = []\n", " for a, b in train_loader:\n", " loss_fn = nn.CrossEntropyLoss()\n", " a = a.view(a.shape[0], -1) # reshape 100x28x28 batch of images to a 100x784 tensor\n", " optimizer.zero_grad() # set the gradients to 0\n", " output = net(a)\n", " loss = loss_fn(output, b) # compute loss function\n", " loss.backward() # backpropagation\n", " optimizer.step() # update\n", " loss_avg.append(loss.item())\n", "\n", " # Logging loss\n", " loss_history.append(np.mean(loss_avg))\n", " print(f'Loss: {loss_history[-1]}')\n", "\n", "# Plotting the loss\n", "import matplotlib.pyplot as plt\n", "plt.plot(loss_history)\n", "plt.xlabel('Epoch')\n", "plt.ylabel('Loss')\n", "plt.show()" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "## Some useful low-level functions\n", "\n", "* `torch.zeros(*sizes)`: returns a tensor of zeros with dimensions specified by a list of integers.\n", "* `torch.ones(*sizes)`: similar to torch.zeros\n", "* `torch.sqrt(x)`: returns the element-wise square root of the tensor x.\n", "* `(*, **, /, +, -)`: python's math operators work element-wise with tensors.\n", "* see [https://pytorch.org/docs/stable/torch.html#math-operations](https://pytorch.org/docs/stable/torch.html#math-operations) for a complete review of PyTorch low-level functions." ] } ], "metadata": { "kernelspec": { "display_name": ".venv", "language": "python", "name": "python3" }, "language_info": { "codemirror_mode": { "name": "ipython", "version": 3 }, "file_extension": ".py", "mimetype": "text/x-python", "name": "python", "nbconvert_exporter": "python", "pygments_lexer": "ipython3", "version": "3.9.6" } }, "nbformat": 4, "nbformat_minor": 2 }