{
  "cells": [
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "T1cOtLH2E3ll"
      },
      "source": [
        "# CS490/590 Tutorial 4: Autograd & PyTorch"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "oW6E9VvcmYpd"
      },
      "source": [
        "## Part 2.1: Automatic differentiation\n",
        "\n",
        "- In class, we have seen how the backpropagation algorithm could be used to\n",
        "compute gradients for basically any neural net architecture, as long as all the individual pieces of the computation are differentiable.\n",
        "\n",
        "- However, implementing backprop manually is like writing assembly language:\n",
        "you need to apply chain rule to each node in your computation graph. \n",
        "\n",
        "- You may notice that the entire procedure is mechanical. Can we write a program to build the computation graph and apply the backprop updates? Yes, it is exactly the autodiff engine does.\n",
        "\n"
      ]
    },
    {
      "cell_type": "markdown",
      "source": [
        "### Confusing concepts\n",
        "\n",
        "- **Backpropagation**: the mathematical algorithm we use to compute the gradient.\n",
        "\n",
        "- **Automatic differentiation** (AutoDiff): any software that implements backpropagation.\n",
        "  - Examples: **Autograd**, TensorFlow, PyTorch, Jax, etc."
      ],
      "metadata": {
        "id": "G4hx7YavysiW"
      }
    },
    {
      "cell_type": "markdown",
      "source": [
        "### Approaches for Computing Derivatives\n",
        "\n",
        "- To understand why we really need AutoDiff, we may want to contrast it with several different approaches for computing derivatives.\n",
        "\n",
        "- **Symbolic differentiation:** manipulate the mathematical expressions to get derivatives.\n",
        "    - Takes a math expression and returns a math expression: $f(x) = x^2 \\rightarrow \\frac{df(x)}{dx} = 2x$.\n",
        "    - Used in Mathematica, Maple, SymPy, etc.\n",
        "\n",
        "- **Numeric differentiation:** Approximating derivatives by finite differences:\n",
        "$$\n",
        "\\frac{\\partial}{\\partial x_i} f(x_1, \\dots, x_N) = \\lim_{h \\to 0} \\frac{f(x_1, \\dots, x_i + h, \\dots, x_N) - f(x_1, \\dots, x_i - h, \\dots, x_N)}{2h}\n",
        "$$\n",
        "\n",
        "- **Automatic differentiation:** Write a program that efficiently computes the derivatives.\n",
        "    - Reverse Mode AD: A method to get exact derivatives efficiently, by storing information as you go forward that you can reuse as you go backwards\n",
        "    - This is efficient for graphs with large fan-in, like most loss functions in ML. In machine learning, we have functions that have large fan-in, e.g. a neural net can have millions of parameters, that all squeeze down to one scalar that tells you how well it predicts something."
      ],
      "metadata": {
        "id": "by7nKutvyhY7"
      }
    },
    {
      "cell_type": "markdown",
      "source": [
        "### General idea for Implementation\n",
        "* Create a \"tape\" data structure that tracks the operations performed in computing a function.\n",
        "* Overload primitives to:\n",
        "    - Add themselves to the tape when called.\n",
        "    - Compute gradients with respect to their local inputs.\n",
        "* _Forward pass_ computes the function, and adds operations to the tape.\n",
        "* _Backward pass_ accumulates the local gradients using the chain rule."
      ],
      "metadata": {
        "id": "llvmUF-pyhce"
      }
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "0JxunitHmYpf"
      },
      "source": [
        "### Autograd\n",
        "\n",
        "* [Autograd](https://github.com/HIPS/autograd) is a Python package for automatic differentiation.\n",
        "* To install Autograd:\n",
        "                pip install autograd\n",
        "* There are a lot of great [examples](https://github.com/HIPS/autograd/tree/master/examples) provided with the source code.\n",
        "\n",
        "### What can Autograd do?\n",
        "\n",
        "From the Autograd Github repository:\n",
        "\n",
        "* Autograd can automatically differentiate native Python and Numpy code.\n",
        "* It can handle a large subset of Python's features, including loops, conditional statements (if/else), recursion and closures.\n",
        "* It can also compute higher-order derivatives.\n",
        "* It uses reverse-mode differentiation (a.k.a. backpropagation) so it can efficiently take gradients of scalar-valued functions with respect to array-valued arguments.\n",
        "* It constructs a computation graph _implicitly_, by tracking the sequence of operations that have been performed during the execution of a program.\n",
        "\n",
        "### Learn more about Automatic differentiation:\n",
        "\n",
        "- Roger's Notes on Automatic Differentiation.\n",
        "- Ryan Adams' talk: https://www.youtube.com/watch?v=sq2gPzlrM0g\n",
        "- Backpropagation notes from Stanford's CS231n: http://cs231n.github.io/optimization-2/\n",
        "- Autograd Github Repository (contains a tutorial and examples): https://github.com/HIPS/autograd"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "1kDjpkTqmYph"
      },
      "source": [
        "### Autograd basic usage"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 1,
      "metadata": {
        "id": "0F3euyjuE3ls"
      },
      "outputs": [],
      "source": [
        "import autograd.numpy as jnp  # Import thinly-wrapped numpy\n",
        "from autograd import grad  # Basicallly the only autograd function you need"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 2,
      "metadata": {
        "id": "szDksM6wmYpn",
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "outputId": "f78ef1ec-9ede-40e7-f935-b3d34f39b7c1"
      },
      "outputs": [
        {
          "output_type": "stream",
          "name": "stdout",
          "text": [
            "0.39322386648296376\n",
            "0.39322386636453377\n"
          ]
        }
      ],
      "source": [
        "# Define a function like normal, using Python and (autograd's) NumPy\n",
        "def tanh(x):\n",
        "    y = jnp.exp(-x)\n",
        "    return (1.0 - y) / (1.0 + y)\n",
        "\n",
        "\n",
        "# Create a *function* that computes the gradient of tanh\n",
        "grad_tanh = grad(tanh)\n",
        "\n",
        "# Evaluate the gradient at x = 1.0\n",
        "print(grad_tanh(1.0))\n",
        "\n",
        "# Compare to numeric gradient computed using finite differences\n",
        "print((tanh(1.0001) - tanh(0.9999)) / 0.0002)"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "SDGKdlknJMuK"
      },
      "source": [
        "## Part 2.2: PyTorch\n",
        "\n",
        "- Makes it possible to work with arrays and tensors efficiently in Python (wraps NumPy for CPU tensors).\n",
        "\n",
        "- Adds GPU support.\n",
        "\n",
        "- Adds automatic differentiation.\n",
        "\n",
        "- Provides a high-level abstractions for working with neural networks."
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 3,
      "metadata": {
        "id": "xHx0nbsrE3lv"
      },
      "outputs": [],
      "source": [
        "import matplotlib.pyplot as plt\n",
        "import numpy as np\n",
        "import numpy.random as npr\n",
        "import torch\n",
        "\n",
        "%matplotlib inline"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "iYkp3o3zE3lw"
      },
      "source": [
        "### PyTorch — API\n",
        "\n",
        "See: https://pytorch.org/docs/stable/index.html (especially `torch`, `torch.nn`, `torch.nn.functional`, and `torch.Tensor`)."
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 4,
      "metadata": {
        "id": "8JrqCxaFE3lw",
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "outputId": "78bf60b7-7be9-4381-e835-832b8487d9be"
      },
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "tensor([1., 2., 3., 4.])"
            ]
          },
          "metadata": {},
          "execution_count": 4
        }
      ],
      "source": [
        "x = torch.tensor([1.0, 2.0, 3.0, 4.0])\n",
        "\n",
        "x"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 5,
      "metadata": {
        "id": "ReldVOHaE3lw",
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "outputId": "531cd469-322c-4a6e-fc08-1fcaf3432b96"
      },
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "tensor([100., 200., 300.,   4.])"
            ]
          },
          "metadata": {},
          "execution_count": 5
        }
      ],
      "source": [
        "x_np = np.array([1.0, 2.0, 3.0, 4.0], dtype=np.float32)\n",
        "x = torch.from_numpy(x_np)\n",
        "\n",
        "# Torch abstracts over NumPy but uses a NumPy-compatible representation under the hood\n",
        "x_np[0] = 100.0\n",
        "x[1] = 200.0\n",
        "x.data.numpy()[2] = 300\n",
        "\n",
        "x"
      ]
    },
    {
      "cell_type": "code",
      "source": [
        "x.shape"
      ],
      "metadata": {
        "id": "A8ntNuxDEwNN",
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "outputId": "26ae7d0b-92c4-4e11-bebd-aa2d7ca01ddc"
      },
      "execution_count": 6,
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "torch.Size([4])"
            ]
          },
          "metadata": {},
          "execution_count": 6
        }
      ]
    },
    {
      "cell_type": "code",
      "source": [
        "x.reshape(-1, 1)"
      ],
      "metadata": {
        "id": "WTTvOw9sENQ1",
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "outputId": "1b56b1ae-9fbe-4d06-e7d9-82c7a1e684fa"
      },
      "execution_count": 7,
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "tensor([[100.],\n",
              "        [200.],\n",
              "        [300.],\n",
              "        [  4.]])"
            ]
          },
          "metadata": {},
          "execution_count": 7
        }
      ]
    },
    {
      "cell_type": "code",
      "source": [
        "x.reshape(-1, 1).shape"
      ],
      "metadata": {
        "id": "qzDfcieJE1pU",
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "outputId": "ff396677-df94-4fa7-f1df-c1231acc4c7e"
      },
      "execution_count": 8,
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "torch.Size([4, 1])"
            ]
          },
          "metadata": {},
          "execution_count": 8
        }
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 9,
      "metadata": {
        "id": "Xr-RVmWlE3lx",
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "outputId": "789c48d8-6aa1-49eb-fc79-4fd542c3c19a"
      },
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "tensor([[200., 300., 400., 104.],\n",
              "        [300., 400., 500., 204.],\n",
              "        [400., 500., 600., 304.],\n",
              "        [104., 204., 304.,   8.]])"
            ]
          },
          "metadata": {},
          "execution_count": 9
        }
      ],
      "source": [
        "# Broadcasting\n",
        "x.reshape(-1, 1) + x"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 10,
      "metadata": {
        "id": "67T-w0KkE3lx",
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "outputId": "420340dc-e6ff-42fb-a0ee-df096a73bb07"
      },
      "outputs": [
        {
          "output_type": "stream",
          "name": "stderr",
          "text": [
            "<ipython-input-10-2efe49c6cd98>:2: UserWarning: The use of `x.T` on tensors of dimension other than 2 to reverse their shape is deprecated and it will throw an error in a future release. Consider `x.mT` to transpose batches of matrices or `x.permute(*torch.arange(x.ndim - 1, -1, -1))` to reverse the dimensions of a tensor. (Triggered internally at ../aten/src/ATen/native/TensorShape.cpp:3277.)\n",
            "  x @ x.T\n"
          ]
        },
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "tensor(140016.)"
            ]
          },
          "metadata": {},
          "execution_count": 10
        }
      ],
      "source": [
        "# Dot product\n",
        "x @ x.T"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "Co-aXekIE3ly"
      },
      "source": [
        "### PyTorch — GPU support\n",
        "\n",
        "We can move PyTorch tensors to the GPU, which allows us to perform some computations much faster."
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 11,
      "metadata": {
        "id": "Fo0IHCd0E3ly",
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "outputId": "f6bc73eb-545b-43b3-fc00-2a334dad6673"
      },
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "device(type='cuda')"
            ]
          },
          "metadata": {},
          "execution_count": 11
        }
      ],
      "source": [
        "device = torch.device(\"cuda\" if torch.cuda.is_available() else \"cpu\")\n",
        "\n",
        "device"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 12,
      "metadata": {
        "id": "KS4O12elE3ly",
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "outputId": "b504247a-95b2-46e8-f675-1485b76be8b8"
      },
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "tensor([100., 200., 300.,   4.], device='cuda:0')"
            ]
          },
          "metadata": {},
          "execution_count": 12
        }
      ],
      "source": [
        "x = x.to(device=device)\n",
        "\n",
        "x"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 13,
      "metadata": {
        "id": "sa2rZfoNE3lz",
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "outputId": "6e8049d6-bff0-496f-fc4f-f0397b2e4aeb"
      },
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "tensor(140016., device='cuda:0')"
            ]
          },
          "metadata": {},
          "execution_count": 13
        }
      ],
      "source": [
        "x @ x.T"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "lxSzk_bWE3lz"
      },
      "source": [
        "### PyTorch — Automatic differentiation\n",
        "\n",
        "PyTorch allows us to dynamically define computational graphs that can be evaluated efficently on GPUs."
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 14,
      "metadata": {
        "id": "5v19hIquE3lz",
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "outputId": "9347e6bf-65c8-46ce-e459-13ab55f31735"
      },
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "tensor([1.])"
            ]
          },
          "metadata": {},
          "execution_count": 14
        }
      ],
      "source": [
        "data = torch.tensor([1.0])\n",
        "data"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 15,
      "metadata": {
        "id": "UgI2RgjlE3lz",
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "outputId": "0dc14746-fc94-4048-ff65-2adb1469c7ac"
      },
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "tensor([1.], requires_grad=True)"
            ]
          },
          "metadata": {},
          "execution_count": 15
        }
      ],
      "source": [
        "param = torch.tensor([1.0], requires_grad=True)\n",
        "param"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 16,
      "metadata": {
        "id": "3jOTeqZAE3l0",
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "outputId": "fdb45ce2-05e8-4aec-f17c-3e3181314d8a"
      },
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "tensor([2.], grad_fn=<AddBackward0>)"
            ]
          },
          "metadata": {},
          "execution_count": 16
        }
      ],
      "source": [
        "(data + param)"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "MGHorbqkE3l0"
      },
      "source": [
        "For a more concrete example, let's work with the function:\n",
        "\n",
        "$$f(x) = x^2 + 2x + 6$$"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 17,
      "metadata": {
        "id": "lKC3Lzwqs07E",
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "outputId": "fbbc75a0-9960-485c-e783-9fcc0269c068"
      },
      "outputs": [
        {
          "output_type": "stream",
          "name": "stdout",
          "text": [
            "tensor([30.], dtype=torch.float64, grad_fn=<AddBackward0>)\n"
          ]
        }
      ],
      "source": [
        "def f(x):\n",
        "    return x ** 2 + 2 * x + 6\n",
        "\n",
        "\n",
        "np_x = np.array([4.0])\n",
        "x = torch.from_numpy(np_x).requires_grad_(True)\n",
        "y = f(x)\n",
        "print(y)"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 18,
      "metadata": {
        "id": "kWQJmBlytIOV",
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "outputId": "b0a6651b-93b4-472a-8fb4-67bcba99b19e"
      },
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "tensor([10.], dtype=torch.float64)"
            ]
          },
          "metadata": {},
          "execution_count": 18
        }
      ],
      "source": [
        "y.backward()\n",
        "x.grad"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 19,
      "metadata": {
        "id": "nCdN3mtXtX-N"
      },
      "outputs": [],
      "source": [
        "np_x = np.array([5.0])\n",
        "x = torch.from_numpy(np_x).requires_grad_(True)\n",
        "y = f(x)"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 20,
      "metadata": {
        "id": "2TWXc5gmtdrA",
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "outputId": "a8744d5c-964a-4acf-9534-69deb24566f5"
      },
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "tensor([12.], dtype=torch.float64)"
            ]
          },
          "metadata": {},
          "execution_count": 20
        }
      ],
      "source": [
        "y.backward()\n",
        "x.grad"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "v60TtmvumYps"
      },
      "source": [
        "### PyTorch autodiff vs. manual gradients via staged computation\n",
        "\n",
        "In this example, we will see how a complicated computation can be written as a composition of simpler functions, and how autodiff provides a scalable strategy for computing gradients using the chain rule.\n",
        "\n",
        "Say we want to write a function to compute the gradient of the *sigmoid function*:\n",
        "$$\n",
        "\\sigma(x) = \\frac{1}{1 + e^{-x}}\n",
        "$$\n",
        "We can write $\\sigma(x)$ as a composition of several elementary functions, as $\\sigma(x) = s(c(b(a(x))))$, where:\n",
        "\n",
        "$$\n",
        "a(x) = -x\n",
        "$$\n",
        "\n",
        "$$\n",
        "b(a) = e^a\n",
        "$$\n",
        "\n",
        "$$\n",
        "c(b) = 1 + b\n",
        "$$\n",
        "\n",
        "$$\n",
        "s(c) = \\frac{1}{c}\n",
        "$$\n",
        "\n",
        "Here, we have \"staged\" the computation such that it contains several intermediate variables, each of which are basic expressions that we can easily compute the local gradients.\n",
        "\n",
        "The computation graph for this expression is shown in the figure below. \n",
        " \n",
        "![Gradient Computation Image](https://drive.google.com/uc?export=view&id=1bvdPv0MI2eM3GeobsHFsFjLrLsibuhJa)\n",
        "\n",
        "The input to this function is $x$, and the output is represented by node $s$. We wish compute the gradient of $s$ with respect to $x$, $\\frac{\\partial s}{\\partial x}$. In order to make use of our intermediate computations, we can use the chain rule as follows:\n",
        "$$\n",
        "\\frac{\\partial s}{\\partial x} = \\frac{\\partial s}{\\partial c} \\frac{\\partial c}{\\partial b} \\frac{\\partial b}{\\partial a} \\frac{\\partial a}{\\partial x}\n",
        "$$\n",
        "\n",
        "<!--\n",
        "Given a vector-to-scalar function, $\\mathbb{R}^D \\to \\mathbb{R}$, composed of a set of primitive functions\n",
        "$\\mathbb{R}^M \\to \\mathbb{R}^N$ (for various $M$, $N$), the gradient of the composition is given by the product of the gradients of the primitive functions, according to the chain rule. But the chain rule doesn’t prescribe the order in which to multiply the gradients. From the perspective of computational complexity, the order makes all the\n",
        "difference.\n",
        "-->"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 21,
      "metadata": {
        "id": "D5RUMyRsmYpt",
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "outputId": "318e73c2-8876-401d-d48a-99f6bd094940"
      },
      "outputs": [
        {
          "output_type": "stream",
          "name": "stdout",
          "text": [
            "autograd: 0.10499356687068939\n",
            "manual: 0.10499357432126999\n",
            "symbolic: 0.10499362647533417\n"
          ]
        }
      ],
      "source": [
        "def sigmoid(x):\n",
        "    \"\"\"Sigmoid function reimplemented for clarity. Use `torch.sigmoid` in real life!\"\"\"\n",
        "    y = 1.0 / (1.0 + torch.exp(-x))\n",
        "    return y\n",
        "\n",
        "\n",
        "def grad_sigmoid_pytorch(x):\n",
        "    x = x.clone().requires_grad_(True)\n",
        "    y = sigmoid(x)\n",
        "    y.backward()\n",
        "    return x.grad\n",
        "\n",
        "\n",
        "def grad_sigmoid_manual(x):\n",
        "    \"\"\"Implements the gradient of the logistic sigmoid function\n",
        "    $\\sigma(x) = 1 / (1 + e^{-x})$ using staged computation\n",
        "    \"\"\"\n",
        "    # Forward pass, keeping track of intermediate values for use in the\n",
        "    # backward pass\n",
        "    a = -x  # -x in denominator\n",
        "    b = np.exp(a)  # e^{-x} in denominator\n",
        "    c = 1 + b  # 1 + e^{-x} in denominator\n",
        "    s = 1.0 / c  # Final result, 1.0 / (1 + e^{-x})\n",
        "\n",
        "    # Backward pass\n",
        "    dsdc = -1.0 / (c ** 2)\n",
        "    dsdb = dsdc * 1\n",
        "    dsda = dsdb * torch.exp(a)\n",
        "    dsdx = dsda * (-1)\n",
        "\n",
        "    return dsdx\n",
        "\n",
        "\n",
        "def grad_sigmoid_symbolic(x):\n",
        "    # Since d sigmoid(x) / dx = sigmoid(x) * (1 - sigmoid(x))\n",
        "    s = sigmoid(x)\n",
        "    dsdx = s * (1 - s)\n",
        "    return dsdx\n",
        "\n",
        "\n",
        "input_x = torch.tensor([2.0])\n",
        "\n",
        "\n",
        "# Compare the results of manual and automatic gradient functions:\n",
        "print(\"autograd:\", grad_sigmoid_pytorch(input_x).item())\n",
        "print(\"manual:\", grad_sigmoid_manual(input_x).item())\n",
        "print(\"symbolic:\", grad_sigmoid_symbolic(input_x).item())"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "YTHQYM2ImYq4"
      },
      "source": [
        "### Implementing custom gradients\n",
        "\n",
        "One thing you can do is define custom gradients for your own functions. There are several reasons you might want to do this, including:\n",
        "\n",
        "1. **Speed:** You may know a faster way to compute the gradient for a specific function.\n",
        "2. **Numerical Stability**.\n",
        "3. When your code depends on **external library calls**.\n"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 22,
      "metadata": {
        "id": "OUC_rGWwE3l2"
      },
      "outputs": [],
      "source": [
        "class MySigmoid(torch.autograd.Function):\n",
        "    \"\"\"\n",
        "    We can implement our own custom autograd Functions by subclassing\n",
        "    torch.autograd.Function and implementing the forward and backward passes\n",
        "    which operate on Tensors.\n",
        "    \"\"\"\n",
        "\n",
        "    @staticmethod\n",
        "    def forward(ctx, input):\n",
        "        \"\"\"\n",
        "        In the forward pass we receive a Tensor containing the input and return\n",
        "        a Tensor containing the output. ctx is a context object that can be used\n",
        "        to stash information for backward computation. You can cache arbitrary\n",
        "        objects for use in the backward pass using the ctx.save_for_backward method.\n",
        "        \"\"\"\n",
        "        ans = 1.0 / (1.0 + torch.exp(-input))\n",
        "        ctx.save_for_backward(input, ans)\n",
        "        return ans\n",
        "\n",
        "    @staticmethod\n",
        "    def backward(ctx, grad_output):\n",
        "        \"\"\"\n",
        "        In the backward pass we receive a Tensor containing the gradient of the loss\n",
        "        with respect to the output, and we need to compute the gradient of the loss\n",
        "        with respect to the input.\n",
        "        \"\"\"\n",
        "        input, ans = ctx.saved_tensors\n",
        "        return grad_output * ans * (1 - ans)"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 24,
      "metadata": {
        "id": "ydX0HQOJE3l2",
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "outputId": "a3138238-e3bf-4422-86fc-c58e60f4c8fb"
      },
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "0.10499362647533417"
            ]
          },
          "metadata": {},
          "execution_count": 24
        }
      ],
      "source": [
        "my_sigmoid = MySigmoid.apply\n",
        "\n",
        "x = input_x.clone().requires_grad_(True)\n",
        "y = my_sigmoid(x)\n",
        "y.backward()\n",
        "x.grad.item()"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "I-ePOja_mYrA"
      },
      "source": [
        "## Part 3.1 Basic models\n",
        "\n",
        "The next three sections of the notebook show examples of using pytorch in the context of three problems:\n",
        "\n",
        "1. **1-D linear regression**, where we try to fit a model to a function $y = wx + b$\n",
        "2. **Linear regression using a polynomial feature map**, to fit a function of the form $y = w_0 + w_1 x + w_2 x^2 + \\dots + w_M x^M$\n",
        "3. **Nonlinear regression using a neural network**"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "JBs8UkXfmYrC"
      },
      "source": [
        "### Linear Regression"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "cLHB3U0BmYrD"
      },
      "source": [
        "#### Review\n",
        "\n",
        "We are given a set of data points $\\{ (x_1, t_1), (x_2, t_2), \\dots, (x_N, t_N) \\}$, where each point $(x_i, t_i)$ consists of an *input value* $x_i$ and a *target value* $t_i$.\n",
        "\n",
        "The **model** we use is:\n",
        "$$\n",
        "y_i = wx_i + b\n",
        "$$\n",
        "\n",
        "We want each predicted value $y_i$ to be close to the ground truth value $t_i$. In linear regression, we use squared error to quantify the disagreement between $y_i$ and $t_i$. The **loss function** for a single example is:\n",
        "$$\n",
        "L(y_i,t_i) = \\frac{1}{2} (y_i - t_i)^2\n",
        "$$\n",
        "\n",
        "The **cost function** is the loss averaged over all the training examples:\n",
        "$$\n",
        "E(w,b) = \\frac{1}{N} \\sum_{i=1}^N L(y_i, t_i) = \\frac{1}{N} \\sum_{i=1}^N \\frac{1}{2} \\left(wx_i + b - t_i \\right)^2\n",
        "$$"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "b-BhEYZOmYrJ"
      },
      "source": [
        "#### Data generation\n",
        "\n",
        "We generate a synthetic dataset $\\{ (x_i, t_i) \\}$ by first taking the $x_i$ to be linearly spaced in the range $[0, 10]$ and generating the corresponding value of $t_i$ using the following equation (where $w = 4$ and $b=10$):\n",
        "$$\n",
        "t_i = 4 x_i + 10 + \\epsilon\n",
        "$$\n",
        "\n",
        "Here, $\\epsilon \\sim N(0, 2)$ (that is, $\\epsilon$ is drawn from a Gaussian distribution with mean 0 and variance 2). This introduces some random fluctuation in the data, to mimic real data that has an underlying regularity, but for which individual observations are corrupted by random noise."
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 25,
      "metadata": {
        "id": "mOLDubBYmYrK",
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 265
        },
        "outputId": "b75279f9-4c96-46a5-e391-f59accb1bb98"
      },
      "outputs": [
        {
          "output_type": "display_data",
          "data": {
            "text/plain": [
              "<Figure size 432x288 with 1 Axes>"
            ],
            "image/png": "iVBORw0KGgoAAAANSUhEUgAAAXAAAAD4CAYAAAD1jb0+AAAAOXRFWHRTb2Z0d2FyZQBNYXRwbG90bGliIHZlcnNpb24zLjUuMywgaHR0cHM6Ly9tYXRwbG90bGliLm9yZy/NK7nSAAAACXBIWXMAAAsTAAALEwEAmpwYAAAUxklEQVR4nO3db6ykZXnH8e+1Z6H+a4sc6GYLni5JiYZolOaEuNU0W9DGViK+MKS2lY0h7Bttpdooa0JsuprVpNH1hdFu1LqmVqSIgZjWarZubZON8aw09Q82WgoIXVhcocW+EJa9+mLmyDicmXlm5vk78/0km5kzZ8557hPY37n3eq77viMzkSR1z7amByBJmo0BLkkdZYBLUkcZ4JLUUQa4JHXU9jovdsEFF+SuXbvqvKQkdd6JEyd+lJkXDr9ea4Dv2rWLjY2NOi8pSZ0XEfdt9bolFEnqKANckjrKAJekjjLAJamjDHBJ6igDXJI6ygCXpDIcPw4HD/Yea1KoDzwi7gUeB54CzmTmekScD3wO2AXcC1ybmY9WM0xJarHjx+Gqq+CJJ+Dcc+HoUdi9u/LLTjMD/+3MfFlmrvc/vgk4mpmXAkf7H0vS8ticdX/6073wfuqp3uOxY7Vcfp6VmNcAe/rPjwDHgHfNOR5J6obBWffKCmzvx+m558KePbUMoWiAJ/DliEjgrzLzMLAjM0/2P/8QsGOrL4yIfcA+gLW1tTmHK0ktcezY07NugBtugLW1XnjXUD6B4gH+ysx8MCJ+BfhKRHxv8JOZmf1wf4Z+2B8GWF9f9/w2SYthz57ebHuz7n3ddbUF96ZCAZ6ZD/YfT0XEF4ArgIcjYmdmnoyIncCpCscpSe2ye3fvZuWxY7XOugdNDPCIeC6wLTMf7z//HeAvgDuBvcD7+493VDlQSWqd3bsbCe5NRWbgO4AvRMTm+/82M78UEd8Abo2I64H7gGurG6Ykdcjx40/PzKGyWfrEAM/Me4CXbvH6aeCqUkcjSV033J0SAWfOVNIf7kpMSSrTYHfKk09W2h9ugEtaPlUue9/sTllZgXPOefp5Bf3htR6pJkmNG172fugQnD5dXo16uDsFmquBS9JCGSxx/PSn8Na3wtmz5daoh7tTKupUsYQiabkMlji2besFec17mJTFGbikbhts2Ssy0x0scayuwo03Pl1OqWkPk7IY4JK6a9ZtXAdLHC95SaOrKedhgEvqrsF69mYJZNoQbng15TwMcEndNbyh1HAJpKYVkU0xwCV117gNpWpcEdkUA1xSt40qgQyWV86e7b2WWbzU0oHZuwEuqV2m7SoZZbC8MjwDn9Rt0pHZuwEuqT3KPBx4nhWR887ea2KAS2qPMrpKBs26InKe2XuNDHBJ7TGpq6Rqg+WbmvYzmYcBLqk9ih5TVladfPh7Dpdv9u//+bFVde0ZGeCS2mXSwppJdfJZA7ZI+abMGn0J3MxKUrdsFbSbNgP25pt7j9Ps9z24ydWo8s24azfAGbikbhlXJ5/nJmiR8k3TNfohBrikbhkXtPMG7KTyTdEafU0iM2u72Pr6em5sbNR2PUlLqEU3GcsSEScyc334dWfgkppRVdB2eHfBaRngkurXsm6OrrILRVL9WtbN0VUGuKT6FWnZ00SWUCTVZ9RSdcsnMzHAJZVvqxuUk5aqz/I9l5wBLqlco25QzrPIxpueW7IGLqlco25QzlP39qbnlpyBSyrXqNWQ86xibNkS9rZwJaak8lVxnuQS18BdiSmpPpurIeetXQ+H9pIF9yQGuKTqeOOyUt7ElDSd48fh4MFie21747JSzsAlFTftrNgbl5UywCUVN64kMuomY5Ha9VZf27K9t9vIAJdU3PCseHW1V05ZXYUbb5ytXj1uVu+Ny7EKB3hErAAbwIOZeXVEXALcAqwCJ4A3ZeYT1QxTUmmmbccbfv/mrHgwtCPg7Nnen2lvVs5zo3PJTTMDfxtwN/BL/Y8/AHwoM2+JiI8B1wMfLXl8kso0brZbdP+SzVnxwYNPB++2bb0blRHT16utdc+sUIBHxMXAa4H3AW+PiACuBP6g/5YjwJ9jgEvtNmq2O8v+JcPBe+gQnD49fb3aWvfMis7ADwHvBH6x//Eq8Fhmnul//ABw0VZfGBH7gH0Aa2trMw9UUglGzXZHBfW42XGZwWuteyYTAzwirgZOZeaJiNgz7QUy8zBwGHpL6af9ekklGhW6s+5fYvA2qsgM/BXA6yLi94Bn0auBfxg4LyK292fhFwMPVjdMSaXZKnSHgxp6NW6XsLfaxADPzP3AfoD+DPzPMvMPI+LvgDfQ60TZC9xR3TAlVWKrvUZcwt4Z8/SBvwu4JSLeC9wFfKKcIUmqRRUHL6hWUwV4Zh4DjvWf3wNcUf6QJNVilhuXahVXYkqLZJpFOlUcvKBaGeDSoph2kc64oPbGZScY4FLXbYbz/fdPt0gHDOqOM8ClLhsM55UV2N7/K11kkY46zwCXumwwnAFuuAHW1oot0lHnGeBSlw2H83XXTV6k4+x7YRjgUpeNC2cPBF54BrjUdVuFs6spl4KHGkttNs0BwoM8EHgpOAOXmlBkwc08s2hvXC4FA1yqW9Fgnqf9zxuXS8EAl+pWNJjHzaKLzOC9cbnwDHCpbkXLG6P26J7nBHgtFANcqts05Y2t9uie5wR4LRQDXGrCtOWNwbLLPCfAa6EY4FKVptnedZyyToDXQjHApaqUuZjGrhJtwQCXqlL2LoB2lWiIAS6VbbNssrrqYhpVygCXyjRcNrFWrQoZ4FKZhssmp0/D/v1Nj0oLys2spDJtdousrFg2UeWcgUtlmmZ/7lHKaj3UwjPApbLNsj/34I1Pl8mrIANcqsO4lkKXyWtG1sClOoyrjQ+G+9mzvfdYQ1cBzsClOoyrjbtMXjMywKWtVHEjcdRKSpfJa0YGuDSsiQOBXSavGVgDl4bNciDwrIcPS3NwBi5tGrWHyepqL5xHlTeamLFLGOBSz6g9TIr0ZZe966BUkCUULYdJJY5Re5icPj25nOLyeTXEGbgWX5ESx6iDhoscQGwXiRpigGvxFSlxjArhouFsF4kaYIBr8RWZRcP4Pu1RNy+ddatBBrgWXxUlDjtP1AITAzwingV8DfiF/vtvy8z3RMQlwC3AKnACeFNmPlHlYLXE5p3tllXi2BzH/ffbeaLGFZmB/xS4MjN/EhHnAP8aEf8AvB34UGbeEhEfA64HPlrhWLWs2jLbHRzHygps7//1sfNEDZnYRpg9P+l/eE7/TwJXArf1Xz8CvL6KAUozrYysehxPPQVvfjMcOGD5RI0pVAOPiBV6ZZJfBz4C/CfwWGae6b/lAeCiEV+7D9gHsLa2Nu94tYyK3oSsexzXXWdwq1GFAjwznwJeFhHnAV8AXlT0Apl5GDgMsL6+njOMUcuuLX3WbRmH1DdVF0pmPhYRXwV2A+dFxPb+LPxi4MEqBigB7Wnls99bLVKkC+VC4Ml+eD8beDXwAeCrwBvodaLsBe6ocqDSM4y7uWmPtpZAkRn4TuBIvw6+Dbg1M78YEd8FbomI9wJ3AZ+ocJxaRpNCeNQKy2m6Vgx6ddjEAM/Mfwcu3+L1e4ArqhiUNNf+JUV3B2xLe6I0I3cjVDsVaR3cvKk43MpXdHfAtrQnSjNyKb3aaZ79S4p2i7SlPVGaUWTW19m3vr6eGxsbtV1PHVdVfXrw+4I1cLVeRJzIzPXh152Bqz7TBnIVLXtb1b337y/3GlJNDHBVa/CcyUlHk9XB48+0QAxwVWdwthsBZ8/2/kzqDKmypGHdWwvEAFd1Bme727b1ukIiRgdnHW19LofXAjHAVZ3h2e7mSe/DwVn3Htsuh9eCMMBVnSKzXffYlmZmgKtak2a7g2UWgBtugLU1yxtSAQa4muUe29LMDHA1Y7DbZNqbiqM6VdyYSkvGAFf95llMM6pTxY2ptITczEr1m2cTqVFf68ZUWkIGuOpXdLfAab52nu8pdZQlFNVvnsU0o77WBTpaQu5GqPJ5M1EqlbsRqh7eTJRqYw1c5fJmolQbA1zl8maiVBtLKCrHPAtzJM3EANf8POVGaoQllEV1/DgcPNh7rJp1b6kRzsAXUd2dIJ5yIzXCAF9EdZ/76CIaqREG+CKqakY8boGOp9xItTPAF1EVM2IX6EitY4AvqrJnxHWXZSRNZBeKinGBjtQ6BrjG22xHhF7Z5MCB3iPU16YoaUuWUDTaqAU61sOlVnAGrtE8/UZqNQN8mU1arenpN1KrWUJZVkXKIJ5+I7WaAb6sirYFjmpHdOGO1DhLKMvKMojUec7Al9W4MohnWkqdMDHAI+IFwKeBHUAChzPzwxFxPvA5YBdwL3BtZj5a3VBVuq3KILYISp1RpIRyBnhHZl4GvBx4S0RcBtwEHM3MS4Gj/Y/VdbYISp0xMcAz82RmfrP//HHgbuAi4BrgSP9tR4DXVzRG1cnauNQZU9XAI2IXcDnwdWBHZp7sf+oheiWWrb5mH7APYG1tbeaBqoBRtetpatq2CEqdEZlZ7I0RzwP+GXhfZt4eEY9l5nkDn380M58/7nusr6/nxsbGPOPVKKNq19a0pc6LiBOZuT78eqE2wog4B/g88JnMvL3/8sMRsbP/+Z3AqbIGqxm47F1aOhMDPCIC+ARwd2Z+cOBTdwJ7+8/3AneUPzwV5rJ3aelMLKFExCuBfwG+BZztv/xuenXwW4E14D56bYQ/Hve9LKFUYLC+DZNr4KPeI6m1RpVQCtfAy2CAl2za+va497t4R2qtuWrgaqlp69uj3r8Z7Dff3Hv0kAapEwzwpk3a0nWcaevbo97vjU6pk9wLpUnztvhN27M96v2bwb45Dm90Sp1ggDepjJPep93Wdav3u3hH6iQDvEltmvm6v7fUOQZ4k5z5SpqDAd40Z76SZmQXiiR1lAFepXlaBCVpAksoValyF0BXTUpi2QO8yiAso0VwK24PK6lveQO86iCsqkWwql8MkjpneWvgRZePz1rH3mwRPHCg3F8Obg8rqW95Z+BFZshlLHUve3Zs77ikvuUN8CJB2NZyhb3jkljmAIfJQdj0Une7TSSNsdwBPkmT5Qq7TSRNYIBP0lS5oq3lG0mtsbxdKOO0YQWl3SaSJnAGPmzecyPnqVsPf63dJpLGMMCHjSpdFKlJz1O3HvW1BrekESyhDJvn3Mh5zpb0XEpJU3IGPmyecyPnaTtsumVRUudEZtZ2sfX19dzY2KjteqWruwYuSUBEnMjM9We8boBLUruNCnBr4JLUUQa4JHXUYgb4qIU4bVigI0klWbwulFH91F3fW8QbnJKGLN4MfFQ/9aQ+6zpn59Nea/OXz8039x79F4QkFnEGPqqfelyfdR2z880Z9Ooq3HjjdNdyYytJW1i8AB+1EGfc3iJVB+TgL4gIOHu296fotVzkI2kLixfgMHoPkVGvzxuQk+rTg78gtm3rLdOPKH4tN7aStIXFDPBpzRKQ05REhn9BHDoEp09PF8ZubCVpiAG+aZqAnLYk4gxaUgUM8FnMUhJxBi2pZAb4LMooiUjSnCYGeER8ErgaOJWZL+6/dj7wOWAXcC9wbWY+Wt0wW8aSiKQWKLKQ51PAa4Zeuwk4mpmXAkf7H7dTVQt0du+G/fsNb0mNmTgDz8yvRcSuoZevAfb0nx8BjgHvKnNgpej68nlJGmPWpfQ7MvNk//lDwI5Rb4yIfRGxEREbjzzyyIyXm5HHlElaYHPvhZK9EyFGngqRmYczcz0z1y+88MJ5LzedUedbStICmLUL5eGI2JmZJyNiJ3CqzEGNNc2ufN5slLTAZg3wO4G9wPv7j3eUNqJxZqlp238taUFNLKFExGeB48ALI+KBiLieXnC/OiK+D7yq/3H1rGlL0s8U6UJ544hPXVXyWCZzVz5J+pluHeiwWdM+cKD3CB6RJmlpdW8p/WZN2x5vSUuuWzPwQdbDJS257gb4cI/36qrlFElLpRsllK16vwd7vGc5Z1KSOq79AT6u1r1ZDz940EN/JS2d9pdQitS6XTIvaQm1fwZepPfbJfOSllD7A7xoOLtkXtKSaX+Ag+EsSVtofw1ckrQlA1ySOsoAl6SOMsAlqaMMcEnqKANckjoqemcS13SxiEeA+2b88guAH5U4nC7wZ14O/syLb96f99cy8xmnwtca4POIiI3MXG96HHXyZ14O/syLr6qf1xKKJHWUAS5JHdWlAD/c9AAa4M+8HPyZF18lP29nauCSpJ/XpRm4JGmAAS5JHdWJAI+I10TEf0TEDyLipqbHU6WIeEFEfDUivhsR34mItzU9prpExEpE3BURX2x6LHWIiPMi4raI+F5E3B0RC79nckT8af//629HxGcj4llNj6lsEfHJiDgVEd8eeO38iPhKRHy///j8Mq7V+gCPiBXgI8DvApcBb4yIy5odVaXOAO/IzMuAlwNvWfCfd9DbgLubHkSNPgx8KTNfBLyUBf/ZI+Ii4E+A9cx8MbAC/H6zo6rEp4DXDL12E3A0My8FjvY/nlvrAxy4AvhBZt6TmU8AtwDXNDymymTmycz8Zv/54/T+Ul/U7KiqFxEXA68FPt70WOoQEb8M/BbwCYDMfCIzH2t0UPXYDjw7IrYDzwH+u+HxlC4zvwb8eOjla4Aj/edHgNeXca0uBPhFwA8HPn6AJQg0gIjYBVwOfL3hodThEPBO4GzD46jLJcAjwF/3y0Yfj4jnNj2oKmXmg8BfAvcDJ4H/ycwvNzuq2uzIzJP95w8BO8r4pl0I8KUUEc8DPg/cmJn/2/R4qhQRVwOnMvNE02Op0XbgN4CPZublwP9R0j+r26pf972G3i+vXwWeGxF/1Oyo6pe93u1S+re7EOAPAi8Y+Pji/msLKyLOoRfen8nM25seTw1eAbwuIu6lVyK7MiL+ptkhVe4B4IHM3PzX1W30An2RvQr4r8x8JDOfBG4HfrPhMdXl4YjYCdB/PFXGN+1CgH8DuDQiLomIc+nd9Liz4TFVJiKCXl307sz8YNPjqUNm7s/MizNzF73/vv+UmQs9M8vMh4AfRsQL+y9dBXy3wSHV4X7g5RHxnP7/51ex4DduB9wJ7O0/3wvcUcY3bf2p9Jl5JiLeCvwjvbvWn8zM7zQ8rCq9AngT8K2I+Lf+a+/OzL9vbkiqyB8Dn+lPTO4B3tzweCqVmV+PiNuAb9LrtrqLBVxSHxGfBfYAF0TEA8B7gPcDt0bE9fS21L62lGu5lF6SuqkLJRRJ0hYMcEnqKANckjrKAJekjjLAJamjDHBJ6igDXJI66v8BoSKqarKki04AAAAASUVORK5CYII=\n"
          },
          "metadata": {
            "needs_background": "light"
          }
        }
      ],
      "source": [
        "# In our synthetic data, we have w = 4 and b = 10\n",
        "N = 100  # Number of training data points\n",
        "x = np.linspace(0, 10, N)\n",
        "\n",
        "t = 4 * x + 10 + npr.normal(0, 2, x.shape[0])\n",
        "plt.plot(x, t, \"r.\")\n",
        "\n",
        "x = torch.from_numpy(x)\n",
        "t = torch.from_numpy(t)"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 26,
      "metadata": {
        "id": "DWKVaOrimYrO",
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "outputId": "65690384-81cd-46bc-dcf3-054054ba5ba4"
      },
      "outputs": [
        {
          "output_type": "stream",
          "name": "stdout",
          "text": [
            "i: 0     loss: 469.1813\n",
            "i: 100   loss: 5.9902\n",
            "i: 200   loss: 4.3283\n",
            "i: 300   loss: 3.3172\n",
            "i: 400   loss: 2.7022\n",
            "i: 500   loss: 2.3280\n",
            "i: 600   loss: 2.1004\n",
            "i: 700   loss: 1.9619\n",
            "i: 800   loss: 1.8777\n",
            "i: 900   loss: 1.8264\n",
            "{'w': tensor([4.2818], requires_grad=True), 'b': tensor([8.3327], requires_grad=True)}\n"
          ]
        }
      ],
      "source": [
        "# Initialize random parameters\n",
        "params = {\n",
        "    \"w\": torch.randn(1).requires_grad_(True),\n",
        "    \"b\": torch.randn(1).requires_grad_(True),\n",
        "}\n",
        "\n",
        "\n",
        "def cost(params):\n",
        "    y = params[\"w\"] * x + params[\"b\"]\n",
        "    return (1 / N) * torch.sum(0.5 * (y - t) ** 2)\n",
        "\n",
        "\n",
        "# Find the gradient of the cost function using pytorch\n",
        "num_epochs = 1000  # Number of epochs of training\n",
        "alpha = 0.01  # Learning rate\n",
        "\n",
        "for i in range(num_epochs):\n",
        "    # Evaluate the gradient of the current parameters stored in params\n",
        "    loss = cost(params)\n",
        "    loss.backward()\n",
        "\n",
        "    if i % 100 == 0:\n",
        "        print(f\"i: {i:<5d} loss: {loss.item():.4f}\")\n",
        "\n",
        "    # Update parameters w and b\n",
        "    with torch.no_grad():\n",
        "        params[\"w\"].data = params[\"w\"] - alpha * params[\"w\"].grad\n",
        "        params[\"b\"].data = params[\"b\"] - alpha * params[\"b\"].grad\n",
        "        params[\"w\"].grad.zero_()\n",
        "        params[\"b\"].grad.zero_()\n",
        "\n",
        "print(params)"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "Q6JKtls8mYrY"
      },
      "outputs": [],
      "source": [
        "# Plot the training data again, together with the line defined by y = wx + b\n",
        "# where w and b are our final learned parameters\n",
        "plt.plot(x, t, \"r.\")\n",
        "plt.plot([0, 10], [params[\"b\"].detach().numpy(), params[\"w\"].detach().numpy() * 10 + params[\"b\"].detach().numpy()], \"b-\")"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "UMf3GbOhmYrc"
      },
      "source": [
        "### Linear regression with a feature mapping"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "I6sur0MDmYrd"
      },
      "source": [
        "In this example, we will fit a polynomial using linear regression with a polynomial feature mapping.\n",
        "The target function is:\n",
        "\n",
        "$$\n",
        "t = x^4 - 10 x^2 + 10 x + \\epsilon\n",
        "$$\n",
        "\n",
        "where $\\epsilon \\sim N(0, 4)$. \n",
        "\n",
        "This is an example of a _generalized linear model_, in which we perform a fixed nonlinear transformation of the inputs $\\mathbf{x} = (x_1, x_2, \\dots, x_D)$, and the model is still linear in the _parameters_. We can define a set of _feature mappings_ (also called feature functions or basis functions) $\\phi$ to implement the fixed transformations.\n",
        "\n",
        "In this case, we have $x \\in \\mathbb{R}$, and we define the feature mapping:\n",
        "$$\n",
        "\\mathbf{\\phi}(x) = \\begin{pmatrix}\\phi_1(x) \\\\ \\phi_2(x) \\\\ \\phi_3(x) \\\\ \\phi_4(x) \\\\ \\phi_5(x) \\end{pmatrix} = \\begin{pmatrix}1\\\\x\\\\x^2\\\\x^3\\\\x^4\\end{pmatrix}\n",
        "$$"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 27,
      "metadata": {
        "id": "D-XKvKC4mYre",
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 265
        },
        "outputId": "d89acc03-f314-4717-9d32-fdbd91e5fdd4"
      },
      "outputs": [
        {
          "output_type": "display_data",
          "data": {
            "text/plain": [
              "<Figure size 432x288 with 1 Axes>"
            ],
            "image/png": "iVBORw0KGgoAAAANSUhEUgAAAXkAAAD4CAYAAAAJmJb0AAAAOXRFWHRTb2Z0d2FyZQBNYXRwbG90bGliIHZlcnNpb24zLjUuMywgaHR0cHM6Ly9tYXRwbG90bGliLm9yZy/NK7nSAAAACXBIWXMAAAsTAAALEwEAmpwYAAAXlElEQVR4nO3df4xlZX3H8feHUWmjNtRhC+vuTpc/VlM0Rs0NdtOmIQEFieloWw3WiIhxSgKpm9goI1Habshibf3RHzEOFcomVCRB48bSKksl1mRWmCWo/KwbFdgtyoq/Q6IMfPvHvSOHy71z77nnnHvOee7nlUxm7rl373nO3J3Pec73POc5igjMzCxNJ9TdADMzq45D3swsYQ55M7OEOeTNzBLmkDczS9hz6m5A1sknnxw7d+6suxlmZq1y+PDhH0bElkHPNSrkd+7cydraWt3NMDNrFUkPDnvO5Rozs4Q55M3MEuaQNzNLmEPezCxhDnkzs4Q55M3MElY45CXtkPQVSfdKukfSe3rLXyTpFknf7n3/7eLNNTNLxOoq7NvX/V6hMsbJrwPvjYg7Jb0QOCzpFuBC4NaIuErSZcBlwPtLWJ+ZWbutrsJZZ8GvfgXPex7ceivs3l3Jqgr35CPikYi4s/fzz4H7gG3AInBd72XXAW8sui4zsyTcdls34J98svv9ttsqW1WpNXlJO4FXAV8HTomIR3pPfR84Zci/WZK0Jmnt+PHjZTbHzKyZzjyz24Ofm+t+P/PMylZV2rQGkl4A3ATsiYifSfr1cxERkgbegioiVoAVgE6n49tUmVn6du/ulmhuu60b8BWVaqCkkJf0XLoBf31EfK63+AeStkbEI5K2Ao+WsS4zsyTs3l1puG8oY3SNgE8D90XERzNPHQDe0fv5HcAXiq7LzMzyKaMn/wfA24FvSbqrt+wDwFXAjZLeBTwIvKWEdZmZWQ6FQz4ivgZoyNNnFX1/MzObnK94NTNLmEPezCxhDnkzs4Q55M3MEuaQNzNLmEPezKxuFc5IWdq0BmZmNoGKZ6R0T97MrE4Vz0jpkDczq1PFM1K6XGNmVqeKZ6R0yJuZ1a3CGSldrjEzS5hD3sxsWqZ08+4sl2vMzKZhijfvznJP3sxsGqZ48+4sh7yZ2TRM8ebdWS7XmJlVaXX16eGRU7p5d1ZZN/K+BngD8GhEvLy37EXAZ4GdwPeAt0TEj8tYn5lZKwyqwy8vT7UJZZVr/g04t2/ZZcCtEbELuLX32MxsdtRUh88qJeQj4qvAj/oWLwLX9X6+DnhjGesyM2uNmurwWVXW5E+JiEd6P38fOGXQiyQtAUsACwsLFTbHzKygbH0dRtfXK56yYBxTOfEaESEphjy3AqwAdDqdga8xM6tdtr4+NwcSrK+PHvNe4ZQF46hyCOUPJG0F6H1/tMJ1mbVLDVc+WkHZ+voTTwyvtTfss62yJ38AeAdwVe/7Fypcl1l7THLlY7ZMUGOvcKZt1NcH9eQ3yjc1XdW6mbKGUH4GOBM4WdJR4Aq64X6jpHcBDwJvKWNdZq03aMTFZkHQwOCYSf31dXj2jjfvZzsFpYR8RLx1yFNnlfH+ZknJ9gjHGXHRwOCYWf319f7PIe9nOwW+4tVs2vKOuBg3OFzSqV8DRtP0U0RzBrR0Op1YW1uruxlmzTMqwF3SmWmSDkdEZ9Bz7smbVaXMnvWoYXgu6dgQDnmzKky7Z93AWrA1g6caNqvCtOcs2agF791bf6mmYePEp6LB2+yevFkVBvWsqz4xWvOVlcBsnhto+DY75M2qMGhM9ThB0PYRMrN4bqDh2+yQN6tKtme9b9/oIGh4j3Ass3huoOHb7JA3m4ZxgqDhPcKxNHCc+NgmPYpq+DY75M2mYZwgaHiPcGxNODeQV9GjqAZvs0PerEyb9QZHBUFTe4RtP08wjhSOooZwyJuVpYyaetN6hCmcJxhHKkdRA3icvFlZGnA/z9KluE2D9F9nAI0d956Xe/JmZUmxN9i/TfPz3fBLsXSzcRSV2NGLQ96sLE2tqReR3ab5edizJ5nwGyqx+rxD3qxMTaupl2Fjm8YZ65+CxI7IXJM3s/FshN/cXBLhN1Ri9fnKe/KSzgU+AcwB/xoRV1W9TjOrQIrlqGESqs9XGvKS5oB/AV4LHAXukHQgIu6tcr1mSWji+PQ2laPK+P0lUJ+vuid/BnAkIr4DIOkGYBFwyFs6qgjjtvUgm7ZDKuv3l0B9vuqQ3wY8nHl8FHhN9gWSloAlgIWFhYqbY1ayMsJkUEBW1YNMcYdU5e8vgRJV7aNrImIFWIHuPV5rbo7ZcFWEybCA3KwHmTeoN15f1RDIcX8H09zBlNkDb1OJaoCqQ/4YsCPzeHtvmVm7VBUmwwJyWA8yb685+3oJnnqq+1XG0UF25zHqd1BVbz/v728GVR3ydwC7JJ1GN9zPB/684nWalWcjyB56qJow2WwnMagHmffIIfv6E07oDn+Uivdu+0P74x+Hxx4b/juoqvyU9/c3gyoN+YhYl3Qp8CW6QyiviYh7qlyn2abylAyyQTY3B8/p/bmUGSZ5dxLjHjkM62WPCuNx9Yf2Y4/B8nJ57R63fe6xj1R5TT4ibgZurno9ZiNtVjIYVW8HePe7YWGh/DDJs5MYJ9Ty9rInkbdMNUm7xy3p5N3JNm0kUMVqP/FqNjXDSgbj1tsvuKAZoTAq1MbtZW8WdqOCcNwedP/75Gn3ZiWdSYO67pFANXDI2+wY1vtM7eTdOL3sUUc14wThqNDOG6h5SjqTBnUCFzfl5ZC3dilyqD0stNt88m7Q72OcndNmYVdWEOZ9n3F3qkXal8DFTXk55K09qrrzUlt77Jv9PkbtnDYLu7KCcJL3GWenWqR9bf2sC3DIW3tUeajd9B77IEV+H/1hB8+8GUgZQVhVoBZ93zZ+1gU45K09mnSo3YQRGkV/HxthN+yIoIztqipQy3rfJnyOFXPIW3vk6cFV+cfblBEaZfWUZ/BkJNCcz7FiDnlrl2wPbliQV/3H26RQLKNH26QjpGlq0udYIYe8TV8ZvezNgrz/j3f//nJ79amFYh0nI5tQJkntcxzCIW/TVVYve7NeWPaPd24Orr0W1tfHu8p1HCmO0Jjmycg6yiSTDjVNgEPepqusQ+RRY9s3/ngfegiuvnr8q1zHNWMjNEo17TJJkaGmCfCNvG26yroZ9EaQb9xsedDY9+Xl7lQEg9Y3KGhsOvr/D8zPV3uj7Bn/rN2Tt+kq8xC5vxeW55B8RuqxjZT9TKq6kUnWjH/WimjOzZg6nU6sra3V3Qxro0nKL004+Tfr9u2DD36w28uem+semW02ZfGkEv+sJR2OiM6g59yTtzRMUuedgXps45Xdyx4W5jP8WTvkLQ1lhkXivb5GKbN8NyMXN+XlkLc0lBUWDorpK6uXPSMXN+VVaHSNpDdLukfSU5I6fc8tSzoi6QFJ5xRrptkYNkbUFPnDnvGRGK1W1sitxBTtyd8N/AnwqexCSafTvWn3y4AXAwclvSQiniy4PrNqzfhIjMYap4Q2Ixc35VUo5CPiPgBJ/U8tAjdExC+B70o6ApwBVDQQ1qwkDormyVNCm+ETrMNUVZPfBhzKPD7aW/YskpaAJYCFhYWKmmOWg4OiWVxrL2RkTV7SQUl3D/haLKMBEbESEZ2I6GzZsqWMt7TUrK5We0WkNZtr7YWM7MlHxNkTvO8xYEfm8fbeMrN8PNrFXEIrpKpyzQHg3yV9lO6J113A7RWty1LmQ3UDl9AKKDqE8k2SjgK7gf+Q9CWAiLgHuBG4F/gv4BKPrLGJ+FDdrBDPXWPN5ytQzTbluWus3XyobjYxzydvZs3kUVWlcE/ezJrHo6pK4568mTWP5xAqjUPezJrHo6pK43KNmTWPL4AqjUPezJrJo6pK4XKNmVnCHPJmZglzyJuZJcwhb2aWMIe8TYevXjSrhUfXWPV89aJZbdyTt+r56kWz2jjkrXqTXL3o8o5ZKVyuseJGzfee9+pFl3fMSuOQt2LGDeQ8Vy/6ln9mpSl6+7+PSLpf0jclfV7SSZnnliUdkfSApHMKt9SaqWi9fVBZxpNTmZWmaE/+FmA5ItYlfRhYBt4v6XTgfOBldG/kfVDSS3yf1wRtBPJGTz5PIA87CvDkVGalKRTyEfHlzMNDwJ/1fl4EboiIXwLflXQEOAPwWbTUFAnkzcoynpzKrBRl1uQvAj7b+3kb3dDfcLS3zFI0aSD3HwXMz3dLN+69m5VmZMhLOgicOuCpyyPiC73XXA6sA9fnbYCkJWAJYGFhIe8/tzbLHgXMz8OePR5RY1aykSEfEWdv9rykC4E3AGdFRPQWHwN2ZF62vbds0PuvACsAnU4nBr3GErZxFLBvn0fUmFWg6Oiac4H3AX8cEY9nnjoAnC/pREmnAbuA24usyxLnETVmlShak/9n4ETgFkkAhyLi4oi4R9KNwL10yziXeGSNbcojaswqoacrLPXrdDqxtrZWdzOsiOzVr+DQNpsCSYcjojPoOV/xauXJjnufmwMJ1td9ItWsRp6gzMqTHff+xBOeedKsAdyTt/Jkx7339+R9ItWsFg55Gz2L5Lj6T56Ca/JmNXPIz7qyp/Xtv/rV4W5WK9fkZ53v2mSWNIf8rPNFSGZJc7lm1vkiJLOkOeRtslkkyzpZa2aVcshbfr4Hq1lruCZv+flkrVlrOOQtP5+sNWsNl2ssP5+sNWsNh7xNxvdgNWsFl2vMzBLmkDczS5hD3swsYQ5529zqavcm26urdbfEzCZQ6MSrpL3AIvAU8ChwYUT8n7o3fP0EcB7weG/5nUUba1Pmi57MWq9oT/4jEfGKiHgl8EXgQ73lrwd29b6WgE8WXI9NqkhPvP+ip/373as3a5lCPfmI+Fnm4fOBjbuCLwL7o3uX8EOSTpK0NSIeKbI+y6loT7z/Tk/XXut7tpq1TOGavKQrJT0MvI2ne/LbgIczLzvaWzbo3y9JWpO0dvz48aLNsayi0w9sXPS0dy9cdFE34D2VgVmrjAx5SQcl3T3gaxEgIi6PiB3A9cCleRsQESsR0YmIzpYtW/JvgQ1XxvQDu3fD8jJccIGnMjBroZHlmog4e8z3uh64GbgCOAbsyDy3vbfMpqnM6Qc8lYFZKxUdXbMrIr7de7gI3N/7+QBwqaQbgNcAP3U9viZlTj/gqQzMWqfo3DVXSXop3SGUDwIX95bfTHf45BG6QyjfWXA9Ni2+GYhZUoqOrvnTIcsDuKTIe1sNPC7eLDm+4jVFw8bGjxoz75uBmCXHUw2nZlhvfJxeenZcvEfQmCUhjZ6851d52rDe+Di99Oy4eJdqzJLQ/p6868jPNKw3Pm4v3SNozJLS/pAf1EOd5ZAaNp69fzl0j348isYsae0PedeRn21Yb3xjuY9+zGZG+0PeV2Lm56Mfs5nR/pAH15Hz8tGP2cxII+QtHx/9mM0Mh/ys8tGP2UxIY5y8mZkN5JBvKl/gZWYlcLmmiTzE0cxK4p58E3miMDMriUO+icq4bZ+ZGS7XNJOHOJpZSRzyTeUhjmZWglLKNZLeKykkndx7LEn/KOmIpG9KenUZ6zEzs3wKh7ykHcDrgIcyi18P7Op9LQGfLLoeMzPLr4ye/MeA9wGRWbYI7I+uQ8BJkraWsC4zM8uhUMhLWgSORcQ3+p7aBjyceXy0t2zQeyxJWpO0dvz48SLNKUcTL0JqYpvMrBVGnniVdBA4dcBTlwMfoFuqmVhErAArAJ1OJ0a8vFpNvAipiW0ys9YY2ZOPiLMj4uX9X8B3gNOAb0j6HrAduFPSqcAxYEfmbbb3ljVbEy9CamKbzKw1Ji7XRMS3IuJ3ImJnROykW5J5dUR8HzgAXNAbZfP7wE8j4pFymlyhJl6E1MQ2mVlrVDVO/mbgPOAI8DjwzorW82yrq5NfRNTEi5Ca2CYzaw1F1FsGz+p0OrG2tjb5G8xa/brIDs3MkiHpcER0Bj2X1hWvw+rXKQbhrO3QzGwiaYV8/71L5+frC8Jxe9mT9sb7d2j796e5MzOzQtIK+f769aCe/TQCcNxedpHeeHaHNjcH114L6+vu1ZvZM6QV8vDsib2yPftpjUwZd+dSpDee3aE99BBcffX0d2Zm1njphXxWXSNT+stG2Z1LtjxTtDe+sUNbXYXrrpv+zszMGi/tkId6puwdtnMZVJ4pozfuYZZmNkT6IV+XQTuXQWWc5eV8vfFhJ2o9/7yZDeCQn6bNyjjj9MY9bNLMcnLIT9OoIB/VG69rtJCZtZZDflxlXV1apKyy2ZGAmdkADvlxNKVM4hOsZpaTQ34cTSqT+ASrmeVQyo28k+fpfs2spdyTH4fLJGbWUrMV8kXnmh/1b4a9v6cENrOazE7IV33ydNj7N+WkrZnNpNmpyVd9r9TN5rL3PVrNrCaFQl7SX0s6Jumu3td5meeWJR2R9ICkc4o3taCqT54Oe3+ftDWzGpVRrvlYRPx9doGk04HzgZcBLwYOSnpJRDxZwvomU/XJ02Hv75O2Zlajqmryi8ANEfFL4LuSjgBnAKsVrW88eceY5z1hOuz9PbbdzGpSRshfKukCYA14b0T8GNgGHMq85mhv2bNIWgKWABYWFkpozgQGhflmJ0w9WsbMWmJkyEs6CJw64KnLgU8Ce4Hoff8H4KI8DYiIFWAFoNPpRJ5/W4phYT7sKlePljGzFhkZ8hFx9jhvJOlq4Iu9h8eAHZmnt/eWNc+wMB82GViTpjgwMxuh6OiarZmHbwLu7v18ADhf0omSTgN2AbcXWVdlho1+2ThhunfvM3vr/a+fn4d9+7o9fDOzhilak/87Sa+kW675HvAXABFxj6QbgXuBdeCSWkfWbGaz0S+DTphmXz8/D3v2uHRjZo1VKOQj4u2bPHclcGWR95+avKNfNl6/b59LN2bWaLNzxWsVfKGTmTXc7MxdUwVf6GRmDeeQL8oXOplZg7lcY2aWMIe8mVnCHPJmZgmb3ZBfXfVFTGaWvNk88er5Z8xsRsxmT953azKzGTGbIe+LmMxsRsxmucYXMZnZjJjNkAdfxGRmM2E2yzVmZjPCIW9mljCHvJlZwhzyZmYJc8ibmSXMIW9mljBFRN1t+DVJx4EHJ/znJwM/LLE5dfK2NFMq25LKdoC3ZcPvRsSWQU80KuSLkLQWEZ2621EGb0szpbItqWwHeFvG4XKNmVnCHPJmZglLKeRX6m5AibwtzZTKtqSyHeBtGSmZmryZmT1bSj15MzPr45A3M0tYUiEvaa+kb0q6S9KXJb247jZNStJHJN3f257PSzqp7jZNStKbJd0j6SlJrRvuJulcSQ9IOiLpsrrbMylJ10h6VNLddbelKEk7JH1F0r29/1vvqbtNk5D0G5Jul/SN3nb8TenrSKkmL+m3IuJnvZ//Ejg9Ii6uuVkTkfQ64L8jYl3ShwEi4v01N2sikn4PeAr4FPBXEbFWc5PGJmkO+F/gtcBR4A7grRFxb60Nm4CkPwJ+AeyPiJfX3Z4iJG0FtkbEnZJeCBwG3ti2z0WSgOdHxC8kPRf4GvCeiDhU1jqS6slvBHzP84HW7sEi4ssRsd57eAjYXmd7ioiI+yLigbrbMaEzgCMR8Z2I+BVwA7BYc5smEhFfBX5UdzvKEBGPRMSdvZ9/DtwHbKu3VflF1y96D5/b+yo1t5IKeQBJV0p6GHgb8KG621OSi4D/rLsRM2ob8HDm8VFaGCYpk7QTeBXw9ZqbMhFJc5LuAh4FbomIUrejdSEv6aCkuwd8LQJExOURsQO4Hri03tZubtS29F5zObBOd3saa5xtMSubpBcANwF7+o7kWyMinoyIV9I9Wj9DUqmltNbd4zUizh7zpdcDNwNXVNicQkZti6QLgTcAZ0XDT57k+Fza5hiwI/N4e2+Z1axXw74JuD4iPld3e4qKiJ9I+gpwLlDayfHW9eQ3I2lX5uEicH9dbSlK0rnA+4A/jojH627PDLsD2CXpNEnPA84HDtTcppnXO2H5aeC+iPho3e2ZlKQtGyPnJP0m3RP8peZWaqNrbgJeSnckx4PAxRHRyl6XpCPAicBjvUWHWjxS6E3APwFbgJ8Ad0XEObU2KgdJ5wEfB+aAayLiynpbNBlJnwHOpDul7Q+AKyLi07U2akKS/hD4H+BbdP/eAT4QETfX16r8JL0CuI7u/60TgBsj4m9LXUdKIW9mZs+UVLnGzMyeySFvZpYwh7yZWcIc8mZmCXPIm5klzCFvZpYwh7yZWcL+H1Ig+oMrfV2HAAAAAElFTkSuQmCC\n"
          },
          "metadata": {
            "needs_background": "light"
          }
        }
      ],
      "source": [
        "# Generate synthetic data\n",
        "N = 100  # Number of data points\n",
        "x = np.linspace(-3, 3, N)  # Generate N values linearly-spaced between -3 and 3\n",
        "t = x ** 4 - 10 * x ** 2 + 10 * x + npr.normal(0, 4, x.shape[0])  # Generate corresponding targets\n",
        "plt.plot(x, t, \"r.\")  # Plot data points\n",
        "\n",
        "t = torch.from_numpy(t).view(-1, 1)"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 28,
      "metadata": {
        "id": "oGgROsxlmYrk",
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "outputId": "485b3883-a3f2-48ce-da94-c9e5d13dd17f"
      },
      "outputs": [
        {
          "output_type": "stream",
          "name": "stdout",
          "text": [
            "torch.Size([5, 1])\n",
            "i: 0     loss: 264.6275\n",
            "i: 100   loss: 96.3672\n",
            "i: 200   loss: 76.4440\n",
            "i: 300   loss: 61.9056\n",
            "i: 400   loss: 51.1965\n",
            "i: 500   loss: 43.2198\n",
            "i: 600   loss: 37.2017\n",
            "i: 700   loss: 32.5950\n",
            "i: 800   loss: 29.0125\n",
            "i: 900   loss: 26.1792\n",
            "tensor([[-4.2387],\n",
            "        [ 3.1587],\n",
            "        [-5.6641],\n",
            "        [ 1.0487],\n",
            "        [ 0.4730]], requires_grad=True)\n"
          ]
        }
      ],
      "source": [
        "M = 4  # Degree of polynomial to fit to the data (this is a hyperparameter)\n",
        "\n",
        "feature_matrix = torch.tensor(\n",
        "    [[item ** i for i in range(M + 1)] for item in x], dtype=torch.float32\n",
        ")\n",
        "\n",
        "params = {\n",
        "    \"w\": torch.randn(M + 1, 1).requires_grad_(True),\n",
        "}\n",
        "print(params[\"w\"].shape)\n",
        "\n",
        "\n",
        "def cost(params):\n",
        "    y = torch.mm(feature_matrix, params[\"w\"])\n",
        "    return (1.0 / N) * torch.sum(0.5 * (y - t) ** 2)\n",
        "\n",
        "\n",
        "# Compute the gradient of the cost function using Autograd\n",
        "\n",
        "num_epochs = 1000\n",
        "learning_rate = 0.001\n",
        "\n",
        "# Manually implement gradient descent\n",
        "for i in range(num_epochs):\n",
        "    loss = cost(params)\n",
        "    loss.backward()\n",
        "    if i % 100 == 0:\n",
        "        print(f\"i: {i:<5d} loss: {loss.item():.4f}\")\n",
        "    with torch.no_grad():\n",
        "        params[\"w\"].data = params[\"w\"] - learning_rate * params[\"w\"].grad\n",
        "        params[\"w\"].grad.zero_()\n",
        "\n",
        "\n",
        "# Print the final learned parameters.\n",
        "print(params[\"w\"])\n",
        "w = params[\"w\"].detach().cpu().numpy()"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 29,
      "metadata": {
        "id": "CY3XajWcmYrp",
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 282
        },
        "outputId": "fe0ca07d-2316-4e13-8219-423d946f3423"
      },
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "[<matplotlib.lines.Line2D at 0x7f4dd5f35040>]"
            ]
          },
          "metadata": {},
          "execution_count": 29
        },
        {
          "output_type": "display_data",
          "data": {
            "text/plain": [
              "<Figure size 432x288 with 1 Axes>"
            ],
            "image/png": "iVBORw0KGgoAAAANSUhEUgAAAXkAAAD4CAYAAAAJmJb0AAAAOXRFWHRTb2Z0d2FyZQBNYXRwbG90bGliIHZlcnNpb24zLjUuMywgaHR0cHM6Ly9tYXRwbG90bGliLm9yZy/NK7nSAAAACXBIWXMAAAsTAAALEwEAmpwYAAAmjklEQVR4nO3de5xVVf3/8deHEdFAMC4iCggm4q1UGqnR7EdJinjBS5ra1yz1CxZWGHlB85bwRfNeKomhQpkIooFFcVWJGsRR0QRBSZSLBAgKiAIOs35/rDN5GM6Zc9l7n8ue9/PxOI+Zs/eZs9eZgc9e+7M/ay1zziEiIvHUrNgNEBGR6CjIi4jEmIK8iEiMKciLiMSYgryISIztVuwGJGvfvr3r1q1bsZshIlJWXnrppfedcx1S7SupIN+tWzdqamqK3QwRkbJiZu+m26d0jYhIjCnIi4jEmIK8iEiMKciLiMSYgryISIwpyIuIxJiCvIhIjAUO8mbWxcyeNbNFZrbQzH6a2N7WzGaY2VuJr58P3lwRkfi5+WaYMyea9w6jJ18LDHXOHQZ8FRhsZocB1wCznHM9gFmJ5yIiAlBdDSNH8uYTr3DTTdEF+cAjXp1zq4HVie83m9kbwP7AAKBP4mVjgeeAq4MeT0Sk7FVXwwknwPbtjKIVzXc7kksvjSZ7Huq7mlk34GjgBaBj4gQA8B+gY5jHEhEpW889B9u3s2VHCx7ZcSFnH7aYffeN5lChBXkzawVMAoY45zYl73N+jcGU6wya2UAzqzGzmnXr1oXVHBGR0tWnD+y+O3+0/2EjezP4h3WRHcrCWOPVzJoDfwamOefuSmxbAvRxzq02s07Ac865no29T2VlpdMEZSLSFLh/VtPr292pa7kXC95siVn+72VmLznnKlPtC6O6xoAxwBv1AT5hCnBR4vuLgMlBjyUiEhfVVLFg9b4MvjJYgM8kjKmGjwMuBP5lZgsS264FbgUmmNklwLvAuSEcS0QkFu6/H1q3hgsuiPY4YVTXzAXSnYdOCPr+IiJxs3o1TJwIP/whtGoV7bE04lVEpMDuuw9qa+HHP47+WAryIiIFtGULjBoFZ5wBBx0U/fEU5EVECujRR+GDD+DnPy/M8RTkRUQKZMcOuOsu+OpX4dhjC3NMBXkRkQKZPBnefhuGDi3cMRXkRUQK5M47oXt3OPPMwh1TQV5EpADmzoV//hOGDIGKigY7EzNSUl0d+nHDGAwlIiIZDB8OHTrApZc22JE0IyW77w6zZkFVVWjHVU9eRCRi8+fDtGk+F/+5zzXYmZiRkh07/Nfnngv12AryIiIRGzEC2raFH/0oxc7EjJRUVPivffqEemyla0REIvTqqzBlCvzyl7DXXileUFXlUzTPPecDfIipGlCQFxGJ1IgRfiKyRqcwqKoKPbjXU7pGRCQiCxfCk0/6AL/33sVpg4K8iEhEfvELn6K54orEhghLJdNRukZEJAIvvAB/+hPccgu0a0fkpZLpqCcvIhKBa6+Fffbxg5+AyEsl01FPXkQkZDNnwuzZcO+90Opf1T6gt2vne/D1PfmQSyXTCSXIm9nDwKnAWufcEYltbYEngG7AO8C5zrkPwjieiEipcs734g84AAYdOW/nFM0998D69ZGUSqYTVrrmUaBfg23XALOccz2AWYnnIiKxNnEivPgi3HQTtPjnszunaNavh2HDChbgIaQg75ybA2xosHkAMDbx/VjgjDCOJSJSqj75BK68Eo48Ei68kMhHs2Yjypx8R+fc6sT3/wE6RngsEZGiu/NOWL4cxo5NzDQZ8WjWbBTkxqtzzpmZS7XPzAYCAwG6du1aiOaIiOSnuvqzgA07Be9Vq3wJ/FlnNeiwRziaNRtRBvk1ZtbJObfazDoBa1O9yDk3GhgNUFlZmfJEIBI7ycGiiAFAcpBc515RAWZQW/vfmvdho6qorYXbz38ZRk4rmb9tlEF+CnARcGvi6+QIjyVSPvIZFKOTQvEl17nX1fltzsH27cwb9ya//30V11y4igO/97WCD3hqTCg3Xs3scaAa6GlmK83sEnxw/5aZvQX0TTwXkVwHxdSfFK6/3n8t4JB4SZJ8E7V58/9+/2nzzzFo5rfZbz8Y1n18UQY8NSaUnrxz7vw0u04I4/1FYqU+WGQ7KCbVSUG9+cJreBMV4LnnuPe983ntvpZMmgStOx0Ltxd+wFNjNOJVpNByrbjI9aQg0WlwE/WdTlXceDicfnpicW4rfjVNQ+Zc6dzrrKysdDU1NcVuhkjpySYnr7x9QTkHp54Kzz8PixZBMYsDzewl51xlqn3qyYuUg0xleEWa4bApe+IJmDoV7rqruAE+E81CKRKVQs4dXqQZDpuq1ath8GA45pgMKz6VAPXkRaJQ6J618vYF4xxceil8/DH8/vewW4lH0RJvnkiZKnRFTAkMn/+vmN8bGDPGp2nuvRd69kxsLOHPrBuvIlFI1ZOHkg0EoYn5vYFly+BLX4LevWHGDGjWjJL4zLrxKlJoqWqqswkEJdwjzEqMa/q3b4fzzvOB/ZFHEgEeSv4zK8iLRCW5ImbkyMyBoAR6hIHF+N7A1VfD/Pnw5JMNqmlK/DMryIsUQjaBoMR7hFkppXsDuWrkKuqpp/yiTj/5CZx9doOfK/HPrJy8SJgaS7dkSsWUak++3FNI2Wjkd//229CrFxx8MMyd63eXGuXkRQohU5DONKCpFHuEpXriCVuaq6jNm+GMM/yswhMmlGaAz0SDoUTCEsaApKqqgq8B2qimMsiq4TJ97dpRN2Ik/9N/A4sW+dGt3boVu5H5UU9eJCwlfgMuL3H8TKkkX0W1awdDhnDt1puY4try6yuWceKJ3YvdwrwpyIuEpRTTLUGlKgUdOTI+ny9ZfTpt5Ege3Xoet7mrGGQPcnn7DcCwYrcubwryImEq8nqekaj/TE0kP//MbmdyqTuIE5jFb1pciX1jWrGbFIhy8iKSnSaQn//73+HcGw7h6EO28vSNC2g+OxHgCzXRXAQi78mbWT/gXqAC+J1zTssAipSjmOfnFyzw88MfcABMndOKvToMjcXVS6Q9eTOrAO4HTgYOA843s8OiPKZIbBRyquJs1Ofnb7mlPIJdDr+/V16Bvn2hTRs/J02HDokdMbh6ibon3xtY6px7G8DMxgMDgEURH1ekcKIYLFSqPch09xxKbcBUDr+/F16Afv2gdWuYPRu6dEnaGYOrl6iD/P7AiqTnK4GvJL/AzAYCAwG6lvLyKiKphBGMUwXIqKY4iOMJKcDvb+5c6N/f99xnz/apmp3EoGKq6NU1zrnRwGjw0xoUuTkiuQkajNMFyMZ6kLkG6vrXJ+q/Qw/G2f4OCnmCyaIH/uSTcOGFfrKx2bNh//3THKPMK6aiDvKrgOSLn86JbSLlJ1WQCno5ny5AputB5tprTn69GdTV+UcYVwfJJ49Mv4Ooevu5/v7wKzvdcQdcdZXfPHlyUg4+hqIO8i8CPcysOz64nwdcEPExRcKTqRcc9HK+sZNEqh5krlcOya9v1swP2zcLnl9uGLTvuQfWr0//O4gq/ZTj72/bNj+T5OjRcM45MHYs7Lln8GaUskiDvHOu1swuB6bhSygfds4tjPKYIo3KJWWQbS84yOV8rieJbK8c0vWyMwXjbDUM2uvX+zl3wmp3tu3L4ff3zjs+sNfUwDXXwIgRSQt/xFjkOXnn3FRgatTHEcmosZRBppt3YfaCG8rlJJFNUMu1l52PXNNU+bQ725ROFr+/Z56Biy7y5+inRy7mDPc0vJCmHTFT9BuvIgWTLmWQ7c27KIJlPjIFtWx72UHmvs+2B93wfXJpdwgpnY0b4Wc/g4cfhqOPhonDXuYLF32t9EpTI6QgL01Hut5nHjfvSlo2vexMVzXZ9KgzBe1ce+a5XB1kkdaZNg0uvRTee8+nZ268Efa4e1r5r76VIwV5aTrSBe1cb36WklTBLpuTU2O95rB61Lm+Ty5XB42cPJYtg6FD4emn4dBD/ct7907sjMHgplwpyEt5CVprnSpoB+ixb90Ka9bA2rXw4Yf+sXGj3751q6/mqKvzZXsAzZtDixY+vrRsCa1a+cfnPw9t2372yOqGYGPBLtPJqbFgF1YgzOd9sjmppjl5fPCBL428805/+2TECJ+q2WOPBu9fjldnASjIS/mIcmRlmuDinA/iixb5x1tv+Z7iO+/AihU+qIetosLXbXfs6AfodO7sv3bt6lcn6tbNb9stSI+7YbCDneeJDyMQRhVQG5w8Puh1AnffAPfeC5s2wXe/C7fdFt/BTblSkJfyEVWtdZK1a+Ef/4AXX4SXXoKXX4b33/9sf6tW0L27D7Rf/zp06gT77gv77ON743vv7edA2XNP/9h9d98rN/M/X1vrm75tG2zZAps3+8eHH8KGDf4e6dq1/sTyn//4fHJNjd+WbLfd4ICOV3CQq+Qge5ODm71NjxZn0PNtPzS/oiKLD1sf7NKdPMP43UYRUBMnjyUTX+P+Fafz6Dmd2LwZzjoLbrgBjjwyh/cqtTl3IqAgL+Uj5Btz4APrrFkwfTrMmQNvvum377YbHHEEnH66DxqHH+7zu506JQJ2nsGhosKna/baC9q3z/rH2LoVVq70VxDvvOOvJpYu3YN/v1ZF9bLj2bRtDxgKDPXvf/DBcMgh/nHoof7Rs2eagT8FOHmGZfNmP0J17NgqZs6sonlzX/t+1VU5Bnco/pw7BaIgL+UjU4qhXiP/eZ2DN97wgWLyZJg/329r08b3zC+5BL72NejVq0EuN1kRgsMee8BBB/nHzlrhHKxbB0uW+JPU4sX++1degUmT/D0B8Cenbt0+C/r1J4CeR/Wl/e63lOzNyHXrfKXMM8/4xyef+NTV8OG+eqZjxzzfuIxObkEoyEvhBblEzpRigF3/844bx+IJr/H4pv6Mn9vlv731ykp/eX/SSXDMMb73npUSCw5mPl20zz5w/PE779u2zd9HeOMN/1i0yH+dNcvv846hbeuNHNJhDT2O2J0es/fhoOXwhS/AgQf6NFR9uik0af4NOAerVsG8ef4lc+b4tJlz/j7FD34A558Pxx4bwmjVJlJpoyAvhRVWL7ixQJv4z/vBts/xR77Lww9exMuuF0YdfXptZMgDbTj9dNh/eSLQ0Ad2y6ENZRQcWrTwaacjjth5+44d8O67PuAvWQJLljRnyZLOzFgAYxuMT2/Txvecu3b1c63vt5+/D9Gpkw+89RVBrVv76qHG1NXBxhnz2TDgUtZ++nmWVyxixaBO/Lu2GwsX+pPQ+vWftf2YY+CXv/TzvffqFSCw51tqGgMK8lJYYfWCGwm0LzSr4r6vv8uTs/Zma21zjuIV7mYI5zabxH7f/hH8cFiwk00MgkNFhe+lH3ggnHLKzvu2bIGlS33e/+23/WPFCli+3Pew64NwKs2bw+c+5wN0/UwQdXX+qmHbNvj4Y3CuN5CYwqoOuM/fsD7sMH/z9Igj4KtfhaOO8n+awIKUmsaAgrwUVli94AaB9tPKKp74A/z6174ypnXrDlw8EC455jV6/ei47Ee55nL8mAaHli39Tcx0NzK3bfPVP6tX+4BfXxW0ebM/QWzZAp9+6n+1dXU+1dOihX+0bAltNy2j7QMjaL9jDV2br6bLMw/Q5lu9Ux8sDCWWXis0BXkprDB7wVVVbPlSFWPGwB3f8b3Nnu3Xc//QjVx444HstRfAl6BnjqNcpVEtWnyWvslPd/jOJUk30HekvoEelib+tzbnSmcxpsrKSldTU1PsZkgZ2LIF7rvPj3B8/304/shNXP3G9zm59hmatWieXfqlCdRIl7xCVSrF/G9tZi855ypT7VNPXsrK1q3wwANw662+tO7kk+G66+C4OffD9VOgbgdsd9ldksc45VI2wk6lpAvmTfhvrSAvZaGuDh57zAf0FSugb19fdfHf/7fN+oR3SR7zXl9JCTOV0kQGN+VKQV5K3vPPwxVX+ME9vXr5Jdu+8Y0GLwor169AUVhh3qNp4jdY0wk0nMDMzjGzhWZWZ2aVDfYNM7OlZrbEzE4K1kxpilau9ANf+vTx1RuPPeYrZ3YJ8PWqqvziGGEHColWGH83+OyqoKKiSd5gTSdoT/514CzgweSNZnYYftHuw4H9gJlmdrBzbkfA40kTUFvrF2G66Sb//Q03wNVX+/rryDXxSoySlU0KLQbjF6IQKMg7594AsF3HPA8AxjvntgHLzGwp0BuoDnI8ib8XX4SBA2HBAjjtND99bPfuBWyAAkXpySWF1oRvsKYTVU5+f2Be0vOViW27MLOBwECArvkX3kqZ++QT+MUvfA++Y0c/sdaZZ0YwZ0o2FChKi3LtgWQM8mY2E9g3xa7rnHOTgzbAOTcaGA2+Tj7o+0n5qa6G73/fz6B42WW+PLJNmwYvUM+66VIKLZCMQd451zeP910FdEl63jmxTeS/tm/3+fbbb/crHc2c6a/Kd6JqF1EKLZCo0jVTgD+a2V34G689gPkRHUvK0JIlcMEFfuWlSy/163K2bp3ihbpUF1AKLYCgJZRnmtlKoAr4i5lNA3DOLQQmAIuAvwGDVVkj4OcFf+ghOPpoP9Xt00/75ykDPKgsTiQgzV0jBbN5MwwaBI8/7kesjh3r5ybPSDl5kUZp7hopuldf9Wtx/vvfMGIEXHNNDgtA6FJdJG8K8hK5ceN8D75tW3j2Wb+WqkhGuoILhYK8RGb7dvjZz+D++/3/0yee8OuQimSkqqrQBF0KVySlNWvgm9/0AX7oUJgxQwFecqA5hEKjnryE7pVXYMAAv5jH+PHwne8Uu0VSdjQAKjQK8hKqJ5+Eiy7y+fe5c/3UwCI50wCo0CjISyic89MRXHut///41FOwb6rJMESypaqqUCjIS2Cffgo//CGMGePnf3/4Ydhjj2K3SkRAN14loE2boH9/H+Cvv94v7KEAL1I61JOXvK1e7RfSXrgQHnnEzyQpIqVFQV7ysmQJnHSSr6D585/99yJSehTkJWfz5/sUTUWFL36oTDljRgMavShSFArykpNZs3wN/D77wPTpcNBBWfyQRi+KFI1uvErWnn7a9+C7d/c18FkFeNDoRZEiUpCXrIwbB9/+Nnz5y/D881lOEVwvnznhq6th5Ej/VUTypnSNZPTgg37t1RNOgMmToWXLBi/IlG/PdfSi0jsioVGQl0bdcw9ccQWccoqfsmCXGvhsA3Iuoxe15J9IaIIu/3e7mS02s9fM7Gkz2ztp3zAzW2pmS8xMBXZl6Fe/8gH+7LP9NAUpBzkFzbenSstoyT+R0ATtyc8Ahjnnas3sNmAYcLWZHQacBxyOX8h7ppkdrHVey8ett8KwYXDeefD738Nu6f6lBJktMN1VgCanEglNoCDvnJue9HQe8O3E9wOA8c65bcAyM1sK9AZ0F60M/N//wXXXwQUX+HVY0wZ4CBaQG0vLaHIqkVCEmZO/GHgi8f3++KBfb2Vi2y7MbCAwEKBr164hNkfyMXKkD/AXXuinKqioyOKH8g3IDa8C2rXzDVDvXSQ0GYO8mc0EUk0ae51zbnLiNdcBtcBjuTbAOTcaGA1QWVnpcv15Cc/tt/upgr/73RwCfBDJVwHt2sGQIaqoEQlZxiDvnOvb2H4z+z5wKnCCc64+SK8CuiS9rHNim5Sou++Gq67yOfhHHy1AgK9XfxUwcqQqakQiELS6ph9wFXC6c+7jpF1TgPPMrIWZdQd6APODHEui88ADfsHts8/OcJM1SqqoEYlE0P/O9wEtgBlmBjDPOXeZc26hmU0AFuHTOINVWVOaxo6FwYPhtNPg8cdDCPD5TkSmihqRSNhnGZbiq6ysdDU1NcVuRpMxcaJPz5xwAkyZEsJiHw1LIu+5B9avV9AWiZiZveScSzkfrEa8NlF/+YsvkTz2WD/xWCirOSWXRG7bBpdfDnV1upEqUkSaoKwJev55P9nYkUf6BT92mYsmX8l59WbNfLDXzJMiRaWefBPz8ss+/96tG/ztb9CmTYhv3lhJpG6kihSFgnwTsnixX6avbVuYMQPat0/sCHPVpuSBUV/8om6kihSZgnwTsXIlnHiiz6LMmAGdOyd2RDmtr6YmECk65eSbgA0bfA/+ww99iqZHj6SdWrVJJNbUk4+5jz+GU0+FpUth2jQ4+ugGLwgyi6SIlDwF+RirrYVzz4UXXvA18Snjd76DkMLM44tIZBTkY8o5GDTI18P/9rdw1lmNvDjX3LmW5xMpG8rJx9QNN8DDD8P11/tgHyrl8UXKhoJ8DP32tzB8OFxyCdx8cwQH0GRiImVD6ZqYmTLFTzjWv78P9n7euJBpMjGRsqEgHyPz5/sJx3r1ggkTIp4yWDXwImVB6ZqY+Pe/fankvvuGPB+NiJQ1BfkYWL8eTj7ZT/j4t79Bx47FbpGIlAqla8rc1q1wxhmwfLlPkx98cLFbJCKlRD35MlZXBz/4Acyd65ftO+64CA5SXe3XX62ujuDNRSRqgXryZnYLMACoA9YC33fOvWd+LcB7gf7Ax4ntLwdtrOzsF7+A8ePhV7+Cc86J4AAa9CRS9oL25G93zn3JOXcU8GfghsT2k/GLd/cABgKjAh5HGhgzxnewBw2Cn/88ooM0HPQ0bpx69SJlJlBP3jm3KelpS6B+wdgBwDjnF5CdZ2Z7m1kn59zqIMcTb+ZMuOwy6NcP7rsvQy18kDlmkicvq6iARx7xE+KoVy9SNgLfeDWzEcD3gI3ANxKb9wdWJL1sZWLbLkHezAbie/t07do1aHNib9Eiv3TfoYfCE09kqIUPmm5JHvS0fDk89NDOUxkoyIuUvIzpGjObaWavp3gMAHDOXeec6wI8BlyeawOcc6Odc5XOucoOHTrk/gmakDVr4JRTYM89fS1869YZfiCMOWaqqmDYMPje9zSVgUgZytiTd871zfK9HgOmAjcCq4AuSfs6J7ZJnj75xJdKrlnjF+LO6qInzLniNZWBSFkKWl3Twzn3VuLpAGBx4vspwOVmNh74CrBR+fj81ZdKzpsHkybBMcdk+YNhB2ZNZSBSdoLm5G81s574Esp3gcsS26fiyyeX4ksofxDwOE3aTTf5/Pttt2WYFz6VfOaKV29dJDaCVtecnWa7AwYHeW/xHnsMbrkFLr4Yrrwy4oOpLl4kdjTitYT94x8+uPfpA6NGRTRtcDItBiISOwryJWrZMjjzTDjgAJ+H3333HH443VQEmaYo0GIgIrGjCcpK0KZNcNppftzRn/8Mbdvm8MPpUi7ZpGJUQSMSO/EI8jG6WVhb6xf+WLIEpk3LY1bJVCmXqqr02xtSBY1IrJR/kI/ZzcKhQ+Gvf4UHH4RvfjOPN0hXG99we7t2PnUTgxOjiKRX/kE+2x5qGfjtb+HXv4YhQ2DgwDzfJF3KJXl7u3b+IDE5MYpIeuUf5MMc1VlEs2bB5Zf7BbjvuCPgm6VLudRvHzkyNidGEWlc+Qf5GNwsXLLks0nHHn/cF7dEKiYnRhHJrPyDPJT1zcING3wlTfPm8MwzWUw6FoYYnBhFJDvxCPJl6tNP/YpO774Ls2dDt24FPHgZnxhFJHsK8kXinM/Bz57tF1yKZH1WEWnyNOK1SO69F0aP9lO1X3hhihdoAW0RCYF68kUwdaqvhz/rLBg+PMULYlb7LyLFo558gb3+uh/RetRRPk3TLNVfQBOFiUhIFOQLaO1aOPVUaNUKJk+Gli3TvFAThYlISJSuKZCtW/3yfWvXwpw50LlzIy9WiaOIhERBvgCcg0su8an2iROhsjKLH1KJo4iEIJR0jZkNNTNnZu0Tz83Mfm1mS83sNTPrFcZxytXw4fDHP8KIEX5kq4hIoQQO8mbWBTgRWJ60+WSgR+IxEBgV9Djl6okn4IYbfJnksGHFbo2INDVh9OTvBq4CXNK2AcA4580D9jazTiEcq6xUV8NFF8Hxx8NDDxVg+T4RkQYCBXkzGwCscs692mDX/sCKpOcrE9tSvcdAM6sxs5p169YFaU5Jeecdf6O1c2d46ilo0SLAm2lglIjkKeONVzObCeybYtd1wLX4VE3enHOjgdEAlZWVLsPLoxfCKlMbN8Ipp8C2bfD889C+fcD2aGCUiOQpY5B3zvVNtd3Mvgh0B141n4foDLxsZr2BVUCXpJd3TmwrbSEE1PpJx9580y/fd8ghAdsUo0VRRKTw8k7XOOf+5ZzbxznXzTnXDZ+S6eWc+w8wBfheosrmq8BG59zqcJocoYAjTZ2DwYNhxowAy/c1pIFRIhJAVHXyU4H+wFLgY+AHER0nXAEX07jjDn+DddgwuPjikNqkgVEiEoA5V/w0eL3KykpXU1MT7E2C5tTz/PmJE+Hcc/3j8cfTzEkTthDuH4hI+TOzl5xzKYdZxmvEa6qcOuQWCPMYafrPf/o6+GOPhbFjCxjgdUNWRDKIV5BvmFMfN85H3QgD4dKlcPrp0KWLn3Rsjz1yfIN8e+O6ISsiWYjXLJQNb1JCpFP2rlsHJ5/sv586tUGpZDa17fW98euv919zqYNv+FnbtVMtvYjsIl49+YY3KWHnnnyIlSkff+wX4F650i/h16NH0s5sUymprjyy7dUnf9Z27WDIEKVuRGQX8QrysGtOPYLKlB074PzzYf58mDQpxds2lkpJTs8kV/NUVMAjj0BtbfaBuv6zjhyp1I2IpBS/IN9QyFP2Ogc//jFMmQK/+Q2ceWaKF6UrxUzVw68/CS1f7usv8wnUAUs/RSS+4h/kQzZ8OIwaBVdeCZdfnuZF6WrbU/Xwhw3z+6urs0stpbpRq1p6EUkjfnXyEfrd7+B//9eXSz76aB6lkply9ZkqbVQ2KSIpNJ06+QhNmQKDBkG/fjBmTJ618Jl63JlSSyqbFJEcKchnYc4c+M45dXx5v/8w8ecraN78K/m/WZB7BMq9i0iOFOQzeOUVOK1/Ld0+XcrU9/4frU7bXLw0iXLvIpIjBflGvPWWT8+02W0L060f7evWwvaK4qZJtMC3iOQgXiNeQ7RiBXzrW1BXB9NHvU2XFms13a+IlB315FNYswb69oUPPvCjWQ/58tHQTWkSESk/TSvIZzEZ2IYNcOKJfrqC6dPhy19O7MgmTZLu/TUlsIgUSdMJ8lnUmG/c6HPwixfDX/4Cxx0Xwvurtl1Eiqjp5OQzLO23aZMP8K+84hcA6ZtyZds83j/gkoIiIkEECvJmdpOZrTKzBYlH/6R9w8xsqZktMbOTgjc1oEbWSv3oI+jfH158ESZM8PPDh/b+WqNVRIoojHTN3c65O5I3mNlhwHnA4cB+wEwzO9g5tyOE4+UnTY35Rx/BKafAvHl+2b6UE44FeH/VtotIMUWVkx8AjHfObQOWmdlSoDdQ3BUtGtw83bTJ9+DnzYM//AHOOSfc98+4XUQkYmHk5C83s9fM7GEz+3xi2/7AiqTXrExs24WZDTSzGjOrWbduXQjNyc6HH/oqmhdegPE3L+G8ZSlWVUq3ulM2qz6JiJSAjD15M5sJ7Jti13XAKOAWwCW+3glcnEsDnHOjgdHgZ6HM5Wfz9f77cNJJ8K9/wZPDFzPg5l7ZV8WoWkZEykjGnrxzrq9z7ogUj8nOuTXOuR3OuTrgIXxKBmAV0CXpbTonthXdypVw/PGwaBH86U8woO7p3KpiVC0jImUkaHVNp6SnZwKvJ76fApxnZi3MrDvQA5gf5FhheOstX/v+3nswbZrPx+dcFaMFtEWkjAS98forMzsKn655BxgE4JxbaGYTgEVALTC4qJU1QE2Nr6JxDp59Fnr1SuzItSpGC2iLSBlpEitDTZ3qK2c6dPA9+J49Q3rjkSPh+ut96qaiAm65xS/nJyJSQI2tDBX7Ea8PPeQHNx1yiC+VDC3AgwY6iUjJi+3cNTt2wNVXw513+ukKJkyAvfYK+SAa6CQiJS6WQX7jRrjgAp+m+dGP4J57oHnziA6mgU4iUsJiF+TffBPOOMNX0owaBZddVuwWiYgUT6yC/JNPwsUX+/T49OnwjW8Uu0UiIsUVixuv27fDFVf4CprDD/fTBSvAi4jEJMiPG+fz7j/5CTz/PHTpkvFHNP+MiDQJsUjXXHwxHHggfPObWf6A5p8RkSYiFj35Zs1yCPCg+WdEpMmIRZDPmQYxiUgTEYt0Tc40iElEmoimGeRBg5hEpElomukaEZEmQkFeRCTGFORFRGJMQV5EJMYU5EVEYkxBXkQkxkpq+T8zWwe8m+ePtwfeD7E5xaTPUpri8lni8jlAn6XeAc65Dql2lFSQD8LMatKtcVhu9FlKU1w+S1w+B+izZEPpGhGRGFOQFxGJsTgF+dHFbkCI9FlKU1w+S1w+B+izZBSbnLyIiOwqTj15ERFpQEFeRCTGYhXkzewWM3vNzBaY2XQz26/YbcqXmd1uZosTn+dpM9u72G3Kl5mdY2YLzazOzMqu3M3M+pnZEjNbambXFLs9+TKzh81srZm9Xuy2BGVmXczsWTNblPi39dNitykfZraHmc03s1cTn+Pm0I8Rp5y8mbV2zm1KfP8T4DDn3GVFblZezOxEYLZzrtbMbgNwzl1d5GblxcwOBeqAB4GfO+dqitykrJlZBfAm8C1gJfAicL5zblFRG5YHM/s68BEwzjl3RLHbE4SZdQI6OedeNrO9gJeAM8rt72JmBrR0zn1kZs2BucBPnXPzwjpGrHry9QE+oSVQtmcw59x051xt4uk8oHMx2xOEc+4N59ySYrcjT72Bpc65t51z24HxwIAitykvzrk5wIZityMMzrnVzrmXE99vBt4A9i9uq3LnvI8ST5snHqHGrVgFeQAzG2FmK4DvAjcUuz0huRj4a7Eb0UTtD6xIer6SMgwmcWZm3YCjgReK3JS8mFmFmS0A1gIznHOhfo6yC/JmNtPMXk/xGADgnLvOOdcFeAy4vLitbVymz5J4zXVALf7zlKxsPotI2MysFTAJGNLgSr5sOOd2OOeOwl+t9zazUFNpZbfGq3Oub5YvfQyYCtwYYXMCyfRZzOz7wKnACa7Eb57k8HcpN6uALknPOye2SZElctiTgMecc08Vuz1BOec+NLNngX5AaDfHy64n3xgz65H0dACwuFhtCcrM+gFXAac75z4udnuasBeBHmbW3cx2B84DphS5TU1e4oblGOAN59xdxW5PvsysQ33lnJntib/BH2rcilt1zSSgJ76S413gMudcWfa6zGwp0AJYn9g0r4wrhc4EfgN0AD4EFjjnTipqo3JgZv2Be4AK4GHn3Ijitig/ZvY40Ac/pe0a4Ebn3JiiNipPZvY14O/Av/D/3wGudc5NLV6rcmdmXwLG4v9tNQMmOOd+Geox4hTkRURkZ7FK14iIyM4U5EVEYkxBXkQkxhTkRURiTEFeRCTGFORFRGJMQV5EJMb+P/yJqOh1KRmBAAAAAElFTkSuQmCC\n"
          },
          "metadata": {
            "needs_background": "light"
          }
        }
      ],
      "source": [
        "# Plot the original training data again, together with the polynomial we fit\n",
        "plt.plot(x, t, \"r.\")\n",
        "plt.plot(x, np.dot(feature_matrix, w), \"b-\")"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "OWYEe11YmYrs"
      },
      "source": [
        "### Neural net regression"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "C7bnv5LemYru"
      },
      "source": [
        "In this example, we will implement a (nonlinear) regression model using a neural network. To implement and train a neural net using Autograd, you only have to define the forward pass of the network and the loss function you wish to use; you do _not_ need to implement the _backward pass_ of the network. When you take the gradient of the loss function using `grad`, Autograd automatically computes computes the backward pass. It essentially executes the backpropagation algorithm implicitly.\n",
        "\n",
        "![Neural Network Architecture for Regression](https://drive.google.com/uc?export=view&id=1iBNS40V_afm_Y1MUosDqeio0wbxgycfh)"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 30,
      "metadata": {
        "id": "0TuIWDkCmYr5",
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 265
        },
        "outputId": "09f7686a-8c00-44ab-d911-2920efbd268b"
      },
      "outputs": [
        {
          "output_type": "display_data",
          "data": {
            "text/plain": [
              "<Figure size 432x288 with 1 Axes>"
            ],
            "image/png": "iVBORw0KGgoAAAANSUhEUgAAAXkAAAD4CAYAAAAJmJb0AAAAOXRFWHRTb2Z0d2FyZQBNYXRwbG90bGliIHZlcnNpb24zLjUuMywgaHR0cHM6Ly9tYXRwbG90bGliLm9yZy/NK7nSAAAACXBIWXMAAAsTAAALEwEAmpwYAABACUlEQVR4nO2dfXQc9Xnvv799key8cCgbThwwvoSEpHDrFoNr2JAQpTYOJoDVum+3tHIxWBiwwUmoEjehx+fQYGJCcMAGS9jW8Z6S5vZeEdsEiLEJ4s2b2MIydWMaApQ4DtFNotSHtsGytPu7fzx6+vvNb2ZWK2lfZkfP5xwdaXdmZ2c1s88887x8H6W1hiAIghBPEvXeAUEQBKF6iJEXBEGIMWLkBUEQYowYeUEQhBgjRl4QBCHGpOq9Azbve9/79Nlnn13v3RAEQWgoXnrppV9prU8PWhYpI3/22Wejr6+v3rshCILQUCilfhK2TMI1giAIMUaMvCAIQowRIy8IghBjxMgLgiDEGDHygiAIMUaMvCAIQowRIx938nlg3Tr6LQjClCNSdfJChcnngfnzgZMngaYm4OmngWy23nslCEINEU8+zvT2koEvFOh3b2+990gQhBojRj7OtLSQB59M0u+WlnrvkSAINUbCNXEmm6UQTW8vGXgJ1QjClEOMfKOTz5c24tmsGHdBmMKIkW9kKpFYHesiIQhCQyNGvpEJS6yWa7QrdZHI5ejvtja5UAhCxBAj38hwYpWNdCYzPqPd2wsMDQHFInDiBBnr8RjpfJ724eRJetzdDTzzjBh6QYgQUl3TyHBi9c476ffg4PhKJjMZMvAAoDWwdWt401RQU1VvLzA8bB5LmaYgRA7x5BsdN7Hqevbr1oWHbgYHAaXIwAPAyAgZaXfdsLBOSwuQThtPXso0BSFyiJFvBMpNjtolk5kMsHp16dBNOUY6nwfWrjVhHfbW+eLS2ysxeUGIMGLkow570UNDQCIBbNoEzJ4dbvTZ+K5b5w3d5HL+19hGemAAmDEj/L2LRXp/90IwVommVO8IQl0RIx917ORosQjcfDOQSlEsnI1+e7v/dZkMLdeaOl67uykc43r1/JvDMdu3Axs2UCjn6FF6jg38ggXk1ZdrrEU7RxDqjiReo05LCxlYplg0Rn9kBFi50p8szecpVFMo0GuvvJLWDUvI2qWYJ04AN90EfPnLwJYttDyRAJqbx2fg3e1KUlYQ6oIY+aiTzZK3nk6TsU2nyTNnCgW/8czlyFgXi+TJz5hRWsOGSzE5Cct3DXxhSCTIu7cNfDkSxqKdIwh1R2murIgAc+fO1X19ffXejWhix7Z37AC+9jV6vrnZGwZxa9ebm6l2HSgdG+/qIg+eSypt3FDNeMIwEpMXhKqjlHpJaz03aJl48o1CNgusWUN/P/AA/U4kgFWryIiyR93bSx4483u/Z16fyZCh7uryb39w0P9cKkXefbEI7NlDhp07XE+cMOEdrq4ptd9i4AWhLkjiNaqEecAc52aP+7776G/2qI8f93rj+/cDn/oUcNttwPr19NxTT9FvO2Hb0kJe/4kTFLJRyrs/WlMuIJcDtm0ztfVaU1JXyicFIZKIJx9FOBxyxx3Ge2bsOHciYeLmJ05QsvXee/3bGxoCvvlN73Nbt3ofZ7MUd//Qh0xsvlAwxpwZGKDnbbiJShCEyCFGPirYiUy7KmVoiEIsbOjZGM+fD/zZn3k96v37/QaYOecc7+P+fu/Fgyty3niDtpVI0IXETvJqDTzxhLfaB6CwjiRVBSGSSLgmCuTzFFLhROb999NvLpXcuxd4/nkKxwBkjIeG/F62TSJh4unpNHD33cAXvwg89xwtLxa9EgZ2GCiRAObOBV5+merx2bPXmrz2TAb45S/pdUoB110noRpBiChi5KNALkdGG6DfTz5JBn3tWjLwtkrkrFnG+JcimQQ2bqSEKnvZ3/++WZ5KebVtbEVL9t6Hh/3vUywaA8/bASiZ299Pf0t8XhAigxj5KLJrF9W2L1liPGxOcN5/P3naYxn5kREy8GvWkAG+5x6vYuQHPgDcequ3C/bpp+lC0t0NHDhQ+k6BKRZp+/b+PPww8OCDwZ24giDUFInJR4G2Nm/su1gENm8GbrkFuOQSU+ly8iR5y5s2edcPIpUiWYIvfAG48Ubgtde8RvvNN+mOwO5GzWbpTmF4uDwDn0ya5imbQoHkF0o1SgmCUBPEyEeBbJY833TaW7o4MgK8+KJJdGpNXvLWrZR4dUkkgMsuA1pb6W/24MfCDt2w5o2LUt59mzkT+PznqewyaP1CgXIHYugFoa5Ix2uU4Eajhx82VTJsWMs5TitWAA89RMb6S18Kfk0y6a/AuewyitcPD9OF5k/+BHjkEbNcKWDxYmD3bpMPUIq29bnPAaeeSvX53d3eeL1SwLRpIkwmCFVGOl4bhWyWjDR79VzGWA7pNIV9APLGgwx8Ok3et9vo9PzzJu5/8iTw4x9710mlgI4OMtYLFphqm5ERasbKZKgL91e/8m5Xa0oYr18/ts6NIAhVQRKvtaRcHZf2dtKMz+WAI0coZBNU/37ttWSQzziDjDDryvT0BG+3UAAefdR/AXAfn3EGeeCsYb9xo9nftWuB733PSCcUChQ+4k5ZF61Ja2fXLgrtsIyxaNkIQk2QcE2t6OoiWeBCIVhUjKc5cRninDmUeGVj6oZtWluBb3/b+x7ukA+lyEhzZ2w5x5pr5FtaKAwTZIztz8KVPnby1a7Rt9/Tfr6pSYZ+C0KFKBWuEU++FuTzXoM9NGSqWVzVSMYtk9Sawi3FIoVwZsyg14bp2tjKkYcPk8JkuSWR+/fTT2end/tdXeS1n3EGeff9/ZQ/sPfzvPOAj36U/n7ySVNrzwae70iGhoAbbiDNejH0glA1KmbklVJJAH0Afqa1vkop9UEA3wKQAfASgL/SWp8stY3Y0tvrNYTJpGlQyuX8Bh7wlyVyc1N/PyU4H36YpjhxFyzfCdiDvNeupWU9PV4Dz7o3dt18ED09pta9q4tKMZldu4BrrvFfOF55hX6UootSezvdlQwO0oVjxw6z7pEj1OkrHr0gVI1KJl5vA/CK9firAO7TWn8YwL8DuL6C79VY8MBswBjr8Ri1RMI0F82a5Z3ylMtRiObLX6a7hVWrgDvvNMZ//nzqmmVlyVSKtvXss8DChf4krM2SJeZvN85fLAKPPWYSxG4ZpdZ0EXnjDcovrFlDeQN3PZkYJQhVpSJGXik1E8BnAGwZfawA/AGA/zu6ynYArZV4r4bBnZzERjaZJKPHy+fMKV1BoxQZd/ao3WlLgHccIFe78IBuO3xz+eWkXdPeTheZtWspwcqGPpGgi8C8eRSqsTtWL7jAv2/FIunVt7cDt9/uX661V4c+myXv3yaREHEzQagilQrXbADQAeC9o48zAI5rrXl6xTEAZwa9UCnVDqAdAGbNmlWh3akz7uSkpUuNbO/ICHD99cCrr9LjVAq49FIqYwyraz/lFKMxk82Sl85VOgDFtTm8w3NfOXbP2jIcvrHvIOxtZTLhVS9f+IKZRGWjNdDXRzH/pUtNaaW7DuvQ9/YCixZRrH5oiNb//Of9YwVlkpQgVA6t9aR+AFwF4MHRv1sAfAfA+wC8Zq1zFoB/GWtbF110kY4Fd92ldTJJuo3JpNYrVmg9fbrWiQRrOXp/lNI6lfIuTybpRyl6nEjQNjo7afv79pn36+zUOp2mdezt8Hu764+Hzs7gfU4kzL7ZnzFoXYD2K5mkdTo6vP+fzk56r337aDmvN9F9FoQpBoA+HWJXK+HJXwrgGqXUlQCmATgFwDcAnKqUSmny5mcC+FkF3qsxsBUdm5qoSamtjTzpPXuCvd1ikUoXzziDKmcASq7yusUieb/spduzVe26+oEB8pRZeGyyipBuLF4pSsDOmUOyBe5nzOWoAsdN6nJlEcfgucqmUKDKHy4dZR19W09HEIQJM2kjr7VeA2ANACilWgDcrrW+Vin1fwD8MajCZimAnZN9r4bBDanYjURc5ggYPRg28n19FF5ZtoyMqK0pz8nNQoEeBxnB7m56PpUCli83HbB2qAfw1uXbIRo7VALQ3xdcYMYFAsDf/A3w1a/S37Nn+z9jNkv7HjQUXCn6TNOmeZ8vFikHkE57w0sSqxfiTg3Ck9Wsk/8CgG8ppf4eQD+ArWOsHy+yWf9By2ZNQpQ9bh7KARjjzQbvyivJq+cSxEzGeM/JJKlMckLT1qS3vWg7N2BX3NgXD+5EtbetlLkb6OgADh2iahs7GWsPHLEf9/f7DXwiAfz2bwNXXUXv5cLjBpcvpwoiNvDuBUoQ4oKbu6uSxlNFjbzWuhdA7+jfbwCYV8ntNxxhV2lOMLNeDJc2clcqa8js3EnGftkyE3bhsExQrbyLPUbQLlW0B4HzhaWnx6zLy3g/Tj2VxMmCPl85Jyl3uR45Qj922eZ551GZpRteqtEXQBDqRtD3M+pGXrCwJQaUAq6+mipL2FtmYw7Q7z/9U+C97zUhF9vYd3aSMWfdF8BbK9/bS8Zx2zajJMmhGjs3wN6xGwZqaiIvnYXKXE8+LGwSdpK2tZnPETTghGfINjebgeLuxbDUF0AqcIRGJ5+nO/EahCfFyFeL3l7vmL4dO6h5KGjIBkASvf/wDyZ56Rp7O+nqlkaysevt9Rs/u0SSl4WVTdoxdv4MpQypm2Dm12Wz1MXK77Fqlb+r95xz6HPasfxyti0evtDo2OdwMmnyZw0Yk5/atLT4vdhCgbxs24tnuNnIPtB2pQzrvvD27Ni1bSiD8gCA3zCuWePfZ/f1Y510YQlmd1uzZ9OkqEOHzPKLLyYjn8sFn+C87VzO+3yNbnEFoWrY5zBA3+NqnsNhtZX1+IlNnTzT0eGtFU+n6bmgevnmZqoL37eP/laKfnNdfGfnxGvI3br9u+6q3mcOY8UKf529/X/hz7NvH627YkXwZ5ZaeqHRqcI5jCrXyQsuHDN++23jzSsFfOYz5M0GhWvYKz161FTJDA1RpcpDD9E2ly6l58d7axcW+qgn9v9geNh47LYipz1Dlv8/a9aE3z0IQiNQ6g64CoiRrzRuvI0NVSpl2vmDYOPrhifcbXIFynio8Un139gJUjsZm0p5Q08AhabWrvWWfxYKRjFTKYrv8+cR4y40MnaBweHDphmwCrF5MfKVxo63cXmk1uFDO847D/jkJ70H162SGSsOXU61Sa0No11dlEgAmzaZZGxLC53YN99sEsmPPRY8/QowJZirV1N8Xwy80OiEzZHo7q649LYY+Upjh0bsIRlBBj6dJsPFZZFAeJVMWLglqtUmdnVRsUgG3a4isKt5vvtdUsdk3vMe4D//k/52a/Yl0SrEgbA5ElU4x8XIVxpX2dGeCGWTSACf/SwZedvbZRngMLVI11uParWJW11UKJh6f74Q8X4++KD3tf/1X+bvVMqML+QLnNTJC3GlCtLbYuQrjav/ctVVFIrgeaesKb9pE3nwtre7cmV4OCJMQqBaSdXJGtJslj7jypXeTt6hIYq989Sq+fOBd94J3gYnq1mwjXMRUbxzEYRy4RkT6bRfyM+V3q4AYuQriZt05a5RbnhgDRpbEMz1dsM88bCwzHiTquUY74mEgIK2a6tjdnebea9791J37dKlwYlovhgmEsDjjxvVzXLyE4IQZezvVhCnnlrxtxQjX0lsA2THkgFqeLDFvQCvt1soUJv/eCUEeDvlGLpyjfd4DWlXl/cz2Nu1X3fwICltckkkECx7AJhkK+c0eD+iWA4qCOXiNkK5cAVZBREjPxnc0IytRVGu/gt7uxOVEBgP5Rrv8bxXPu/NOwwN+XVm7LubdNorRnbKKcD69f7t2oafQ1xHj9LjDRtIUG3JEvHihcbCLcyw83VKeYswKoQY+YlSKjRz9dVeieCxwijleOKVqHUv13iP5716e70GOZn0btf1XFw5hlwueGygDS/v6iLlTfb+n38+PIchyVkhirjfrR07zGjNUnfyk0CM/EQJC81obeLIySRpwrMHOlljM9la9/EY73Lei5X0OIGUSAAbN3pfFzQly/byt20rbeABk5hm7BDO+vXAb37j1bqPalmpIADeIorWVvqppkMSpndQj5+G0q6x9SeamozeDP+42jTpNOmxxEVrxf38pT7bvn3Bc2ZtTR2edRs2IzZsLq79HM+KjYJWjyCE4epTVVm7JlH5y8YUgb3iO+8EHniAVBVtk2MPxgDI0+3sJA+TS6gaGftOplAoraSXzZLmTJiccDJJv5ua/P+3MD7xCeDCC73PsTa9u11JzgpRIJ+nsZg33EC5Ky4pDpIyqSASrhkvXV0m6cfhgaBa7zPOAH7xC28dbJy6NiuRCHbDR0D4IHA3bv/ii8Cll3rX6e834xBFxEyIEmEyBjVAjHy55PMU/92xgx7zcOvBweAD93d/Z2rEjxwho6R1fDzLShnSoO5erofPZEjUbdcuf5lloUBSCLbxLxbNBdTeriRhhXrT2+t3XICJCQ6OEzHy5cCJPNdb7+mhzk32aAHgtNNIdMyu+pg/34y827AhPoamEqJnQQbY3q47FNytq+f/KxB8AZUkrBAFWlqoQIHtRDoNXH99VSdCMWLky4Hjzy7vehf95glGW7fSGL9f/pKUJZ991ryWNeWrUAfbsJQywGz8Bwa8r7nmGuCtt4D9+81zxSL1JwRdQKVDVogC2Szl7rZuBaZNA84/vyYGHhAjXx5uA8O55wKvvgrs3Ak88YTRk7Fvx4aHKbzT0WFeyw09HDee6rgGOJczYRoeeG43UKXT9P8EgE99yiuJoHXwBVQ6ZIUokM8bMcJiEXjhBa9YXxURI18Obvw5lwNeeYWWca32E0/4X7dzJ7BokfH0u7upmadGBzfy2AY4maT/D8+ztWvj3QaqfB647jrvDNymJro4rFvnD/1IElaoN/YdPeCddiZGPgK4cWO35GnfvmA5Ya1J0+XZZ8lIjYxI2MDGNsBHj9IFsFCgGDt3EbsNVLZOTipFzWbcXczevxv6kUlSQj3hpsFUyoyzVMo77ayKiJEfi6C4cVubt8zvF7+g36ycyLLCgFGWlLBBMGyA83m6w+H/z4YNfsXOXM5cCABad+dOinHy44leRKUCR6gG9oQ0pSin9JGPAPfdR+dqDaadiZEfi6DEXSYDvP/9wLFj3nW50uPSS03JJOtRSNigNKX+P+y9B41Q5N6DgQH630+kTFUqcIRqYU9IA0jyZMYME46swV29GPmxcD3w48eBv/3b8PWHh0k4K50Gli3zhhokbFCaoP+Pq3LJ8IBv1gh6/HH63yeT4y9TlQocoVoETUibjEMyAUTWYCxs+YKnnwYOHfIuP/NMEhhqbjYt+VqP3eovlEeQyuWKFXQhffZZOi6XXGJCZ4UC1dbb5POUkA2TkxAZBKFa8MyIdJoMeypFxQKce6pB34x48uVge5hLlphuV4A6W9vbTcyYK0TEWFSGlha6gLpzcG3uuMP7eGCANEK4xt6uwAkKxUgoTagmfL729FBvDY8DrVHfjBj58WIfsCVL6O+LLyatmo4O05IvxqIyBOnb2GWSuZx3yg6PDAxqIS8VipFQmlAtuEaeS4V5sFCNHEEx8mGEVVvk8xQOOOcc4PXXvVONvvMd0lNZs6bWextv7Aoce1DLlVeSQWd4YMvOnf5tcDmm3F0JtWaswTlVRox8EGHVFq6SnCuLOzJCnqV4hJUnnyedIK5UKBSMWBxAx2L5crqTeuIJvwwFxz8Bf8OUIFQLu0YeIEcEqOn5J0Y+iLBqC1dJrtREI6m7rhx2rTHHMt3/va3mt2wZxeNffZU6k3nd/n7TWu7G9+V4CZXGdhaVAmbOBH76UyoJrmHXuxj5IDIZb4lTJmMSeamU19DzegBl0NvapO660tgt4YkEMHcu0NfnrbpZtAg4fNjfDfvjHxuphIMHgRMnTNfhypX02v5+b8JcjpdQCXI5c74BwE9+Ypa5A++riBh5F06ScInTqlX0w7f/3G5ve5I33uiNsa1bJ3XXlcTtVdiwwavtDwA/+hFVLdjdsPbykRHgwAHvcRsZ8TdZ8fECxLMXJk4+D2zZEn63r1TN8kNi5F1caeBDh7yeu13JAZDRdyVDRcKgsgSVOHZ0UGkkK1H+67+OPRCcl7NuSCJBx9N+nu/c5E5MmAy9vX5bYZNI0J1nDRyJSTdDKaXOUko9o5Q6opT6oVLqttHnT1NK7VFK/Xj0929NfndrgNsYs2QJhWGYRMLb2LBxY3jdNTdQiYGYPO6c2GwWeOYZYOFCb8isHLQmDZFNm6gGn4/1jTfS8ervp9ts+05MEMYDDwkJo1Cgu8g77qj63Gelx/PlCNqAUh8A8AGt9UGl1HsBvASgFcBfA/i11vpupdQXAfyW1voLpbY1d+5c3dfXN6n9qQichMtk6As/MAD8+tdGjyaVMpIFgNzW15NSszPd6if7XF+4ENi9259wdbfX3EwXEzm2wnjp6qLQonuXmUgYqQOW5bjzzkmVXiulXtJazw1aNulwjdb65wB+Pvr3fyilXgFwJoDFAFpGV9sOoBdASSMfGbJZupW66SaT3EsmTcKOJQsAua2vN9ksXXA7O/3ePOvSB8GNbG4TlH2brRTp1ssxFcaLLYmdTnslsQcHvYNxqhzSrWhMXil1NoA5AH4A4P2jFwAAGADw/pDXtANoB4BZbDjrTT4P3Hyz10DwwbKbakTYKhq0tQHbtvm9eVfzRilKsiacKKXtzbv5lCoPWRZiiCuqNzICzJvn99Rnz65JFKBiRl4p9R4APQBWa63fVtatstZaK6UC40Ja6y4AXQCFayq1P5MiKGmSTFL83dY4P3y4pmpyQgjszW/eHL7OpZeSqBlgyidnz6bH7t2Y6NgIk8EV1QurpKmRlEZFjLxSKg0y8I9orR8dffr/KaU+oLX++Wjc/heVeK+aYHtzAB2kBx/0CmO5pZY1UJMTStDWRg0mJ07Q8fr4x2mOJtfWn3aaN5wzPEx1zLNm+e/G7ASvIIyXlhbK27H9cO8ca0wlqmsUgK0AXtFaf91atAvA0tG/lwIIEBSJKDxZnW/x02nj9TF2qWXYEGmhdnBF01e+Qsb97rtN5UwqBbz1lv81W7dSbLSpyXwR9++vaqWDEGNY0hqgO0uOZhSL9a3Q0lpP6gfAxwFoAP8M4NDoz5UAMgCeBvBjAHsBnDbWti666CIdCfbt03rhQq0TCa0BrZNJrVes0Pquu2gZrzN9Oi2bPt08L1SPffu8x6Cc9Ves0Lq52RxL92fhQq07Oug48nPNzXI8hfKxzzO2B52dNbUPAPp0iF2tRHXNCwBUyOL5k91+TbE14YeHza1+Mhnc9i6x29oxUamIN94wxzKIPXuA733Pu1yS6EK58HlpyxecPEl39hGxD9LxygQdrEQCWLCAZIV5gLRtAESDvHaMt5LJFTVj7FFsgJnixd2vgCTRhfLh89It381kvBLZdVQ+FSPPBB0spaie+vXX6Xluh89k6rabU5bxSEW4ssSu1pD7OJGgxDqPDXRlKnibuVz4cmFqwuel7UwUCsCtt4ZXb9X43BEjz4QdrJtv9pZTFgpUVTN7tnzRa0m54THXg2f5Ca6Rb2oCfv/3abgLc9ZZZOBt423XzgPeLtjubumCFQg+L9eupdBfkNBdnXtpxMgzYQfLrZfXWmK29aKc8JgrS7xgAR1TXhZktN98k2rs2XgDXu9r6VKvSJ0cf8Emm6VzjM89gJyK/ftJArvOYoVi5G2CDlYQqZTEbKOKG9ZZu9Yrasb09oZ7X0ePmtwMnwfptPlbYvaCCw8VWr8e2LWLnIwdO0gp9f77vU2UNaa+VfpRwq5x7e0FVqygVmS7kYFj8pMUdROqSLkKoNks5Vvs48syw9u2eY/xnDnmnFixQkI1QjDZLNkM+9zhSps6NtiJJw/4B0SzwmRbG119h4eNfAELlMntenQpJ6yTz9MwGK6sueYa0qh3JS2KRcrBPP008NBD1dxrIQ6wxDDf9UXgrl88ecBfntfZSUb/8GHTtcY68qw9Lrfrjc369eaLWCySQmA2a0Y/8nG3czCCUA5XXmnOoTpLGgDiyRMcx+U4LH+xe3rMaLhiEbj+eu+YP6ExyedpVKDNwYMkD7tqlblzS6XouMtFXSiHfB741KfMtDKA7Eed7/rFyAMmjpvLkZ7JyAh9wZcsIeVCW3ZWjHvj09vrz6scOOAdDl4sUghn3jzy7tmTl+MvhJHLeQ08EIlwjRh5hr+827bRb62pFj4irclCBWlpIfEyu7uZ7+BsZsygdWUwjDBRIjB0pv4Bo6jAXZIcnrGTqyI9Gy+yWZKG/tCHwtdJJOjOLUhOATDVWKJYKTBz5nhj8M3N9Fydz5Op7clzq/rAANWz2qJkEoeNLzwLgG+t7QQZV9Ykk/SbE7Esj5DJTFwsTYgvPO5Pazp3rr6aGqHsEX91Ok+mrifPA5s3b6amBVvnZMEC+eLGGbcr9vLLKfeyfLlXAzyXIw2S4WFzd7d6NT3P3v3QEN0Bikc/deFxf3yeaE25nMHB4LvAGjN1jXxvr7dVndGaEq5i4OMLV1Mlk3RLzV2xbW3AtGn0fDJJ+jZ2Io2rrgYGTIlcsUhds5ddRt6cEG+CwnRB4/6OHjUDaepcdj11wzVu0wKTSNAV2BaoEoMfL4LEzvh4b9hAYmXd3cCRI/7XKkWhvULBdD9rTbkcnhsr50s8CQrTAWTQbVuiFEmTNzXR+VRHSQNgKht5HvG3YoW3qiKZpCuwxFzjjd0V6355ly4lox2E1iZ348pcSCd0vHGT8Dxg6ORJcg45d8Pnji1pUEembrgGII/N1RXfuDEysTShRrhfXsA799WmUDCzBdyyy3SaPDapvIkndpivqYnCdkNDJl/D854BOj8iUrwRf08+LOySz5uaeIAOyu23A+3ttKzO8qBCDXGVK1m3KJejkM0LL/inSXGC1ua668z2hofJ6ItnHx/spkmAjLwNOwW2/lUEjn28jXypUjdXiAqg8E1rq8xvnWqExei3b/c2TNlwqRyfQ83N5sLAdwN8Sy/nT7zYvp08eKXoHCgW6fdVV1EDHRv3Oo/9Y+Jt5EvNBQ3Tq5H5rVMT++LPv/lWPIh0mkJ7PDKQ5Yhd7+7gQfqyy7kUD/i84Du7ZBJYvJiS8Y89Zu4EI9RLEW8jP9Zc0KVLTSMUj4aT0Ew8Gatayv1SrlrlDdEA9IWeORN497uB226j0J772mSSLgBcntvXR8skeR8PWlq8w+CLReBf/sUk4yM09o+Jt5EPC7u4X+g6T24Rqkw5XpV91zc0BDz6qH9AjNbAW2/Rl3nVKjMX1n4tQE1Vb7wB7N1L6544IWGbuJDNAps2UbksS6C8/rrJ0ySTxlGMSF4v/tU1QdozuRx98fgLXefJLUKVCdOfseG7PvbS7C8uYErkCgX/3AG36aWtjRrqbE367m6ptokL7e3As89SpzQPEwK850u5E8pqQLw9+SDyeWDLFnNgikUauNvVZeKrEcmKCxVirLAd4B3kzh44DwJfsoQcgUyGZA3cPM7goPeOEaD17HBPBHTFhQoSNg96eDhyeb34e/IuQVU1O3YAN95IOjabN5Pwv3hd8WE8c1/XrqVKGVvyYPZsWs7S04sX03JbyM6+Y+QvvlsznclI/XycyGapVNLGDtdEhKnnyYfJGdgMDUkMNW6U61W5eRzAG8/fsAHYvZsMeCJBj93tsnKl1jQ0Ytkyqr6JgCKhMAlYtRYwd/ttbaakkpspI3Zcp56RZ09r/Xpg1y5/BQXT3S1hm6mKfUFYt84bz9+61Vs739NDvzmcw7o3PCD8gQcohutuR0I3jQWr1rJz2N0NPPNMQ/TUTD0jz/zmN97b6cWLqXLiwAEjOCVfRMGO5yeTwEsvefM5e/YATz3lr8QB6LnBQf92pFS3seCBQrZq7dBQ5GLvYUw9I8/i/vYB05oE/mfP9t6ayxdRsD21/fspf2NjV1bYcDnd0aOmGSriHp8QAJffBjXGHT9el10aL1PLyLO4v6swyPLC8kUUgmBP7aabvM/bTTEus2YBx46RU7F9u4nByznVWNgDZty7tfvuoxGSEe+xmRpGnrsdjx71fykTCaqiYK9dvohCGG1tRlo2NfrV4QSs1t5W92PHTBWXNEM1LnaYTSmjQgqYGQLFYqST6fEvoeTbrTvuINXJdJq+lKkU0NEB/P3fR/bgCBEjm6Vk2403UrUMf+FdidmLLvJ3ykozVGPCd/fLl5u5vwDZEBaoi7gkefw9+aCW81mzgm+vZBqUAIx9HnDJHDdMpVJk3Fn/6PrrgcOHvVU4kshvXLgij2UMuEnu9NOBf/xHMzwmk6n3ngYSfyMfpBVejkCVePdTk7HOAztGCwBz51KtPC/jC8Ps2WZy0PBwpI2AUAauHTn9dOCRR8xyHvIewfGP8Q/XlNvtWI6+iRB/xjoPWlq8t+0vv0y/ueMVoJp4gByKRYvMwO/VqyVk04jY83/vvJN+f/Ob3nVsqfKIEX9PHigvmSp1zAIw9nnAreydnf5+CldyWCmvvIE0QTUGdrgO8Hc8b93qL6e0JS4iRtWNvFLqCgDfAJAEsEVrfXe137MkQfFW+0od8XIoocqUU0bLrez2hYAbZjhWz+Ec1xhIyCbaBA115zu7d94Bbr7Zr3117rk0+jGidqOqRl4plQSwCcDlAI4BOKCU2qW1PlLN9w2kq4uuwP393pInQGLxgpex7vzsC0Em442928lYrb1Nd4UCcOutkYzbCqMEDXW3xzy6Bh6g0uyIGnig+p78PACvaa3fAACl1LcALAZQWyPf1UVlbzYRnOAiNBB8jsyf762iUQo45xzgj/4I+MY3/K+TcyzaBBVqDAz4O51tIl45VW0jfyaAn1qPjwG42F5BKdUOoB0AZs2aVfk9yOeBe+7xP2/HzyQWL0yEoDmwWtNUqHvvDe6GVYrkEWTuazRxw3WbNnkNPDe+uTLSEbYbSocNKq7ExpX6YwBXaK1vGH38VwAu1lqvDFp/7ty5uq+vb/JvzDF2d8gDc9llwGmnmcnqgNTHC+Mnn6dzyZbJCBIqS6fpvHrhBWP4m5uNiqEQTf7yL71lkgDQ2kpS05xcX7YsEmq1SqmXtNZzg5ZV25P/GYCzrMczR5+rHnbihEvX+Et32mlU1/z00ya2xpKhXP4mCOWSzQKf+xzwta/RY1vewDb0H/6w39GwVQwBf0GANObVHreqxi2TVIq65Ds6GurYVNvIHwBwrlLqgyDj/ucA/qKq72gnTrgTjb2rf/93koW1kRipMFHyedKLB8x5xs5DKmWqbF55Jfj13/1ueJmeDBipLUFVNe4d2eWXG+PeQE5hVZuhtNYjAFYC2A3gFQD/pLX+YTXf0zOQmWNntqF3iXg8TYgwdvdrsUiVNHzOaU13jTzYOYjnnqNRk7lc8GASacyrHUFVNdOn02+lgIULgeefJw2s+fMbqqmt6h2vWusntNYf0Vp/SGv9lWq/338nThYs8E5STzk3LckkxdckLipMFHYokkmKu9udsFoDF15IzzNBBn9oCHj2WTo/k0n66e/3nrfihFQfHtnITU1z5gCf/jQwbx7NfeapUI144dVaR+bnoosu0hVj3z6tp0/XOpHQOpXSeuFC28fSuqOjcu8lTF327dP6rrvod2en1smk1kpp3dREz+3bp3VrKz3P59706d5zEaD1V6ygH15XKXosVBfXVnR00PHgY9PcTMd2+nQ6NtOn02siBIA+HWJX46tdk81SbDOZpFvpvXu9yw8dqsdeCXGDNWtYlIzvGNlrz2bJG7TLKd95x7+dkRFSR21rM3cH06aZ6i+hethht5ERSrhyyAagv/v7KU6/fHnD5UjirV0zOOhtMbe54IKa744Qc2w5WrtBpqUlPCfEaG2qNVxZBam0qQ52qbXd1XrsmHe9RIKq8FhKusEuvPE28qwYyJU2dgPDqafWc8+EOBImbpbNArffDqxfH/7ahHVTbcsquFUfoq9UGezZrYkE8LGPUSI8iI9+FPjRjxq2Kz7eRh7wD1pWipJhkswSKk0pcbOvfpXmgW7dCvT1+e8uCwXgi18ErrjC68HbomdDQ2bcXIQacRoS7lbmO/0XX/R68zYf+Qjwb//WsF3x8TbyuZxXIAowJZWCUA1KiZu1t5MXvn9/8PLnnqOu2OZmUytvT6BKJMgIFYv0u7PTOyRcKB9XDZT/x4D3f93URM1PixYBPT3AkiUN97+Or5HP52mmaxARFxQSYkxLCxmQoDwRQM+fPEkGhZOBPG5uyRKvTIc9qELOZT9hXcQsd8J392zU+ZgoBdxwgxkTCpjmtOefbzgV0fga+d7e4FuvCIv7C1MErvgC6Hy05Td4QPS73mUqdZqaKGzjjhXkRKCcy36CchlsqO3/t60a+sADwWNC161raKXa+Bp5OwnG8cs5cyRpJdSX3l6vx9jeTgaltxc4fpx+9/cDjz1G5+3y5cApp5CRv+ACKhhoazOvkXM5GLeDle+MCgVzIQXoWLzxBhl4N6mdz9MFdWDAe8FtsItqfI08QHWtgCSnhPpjhwpcvXKABk90d3vHBQJkYDZvpr+feoouDNOmURy+gfRTao5b6bRkCYVabM++p4f6Zzip3dNDF1MAuOkmSpJzTi+dpgtuA9qSeBr5fJ40QdwvkiDUg1JlkIB/8AhgdMrfesu7LYnDl0dQpdPs2f7Hzz9vktt799JydzYvYJrVGvB/Hs+O11zODHMYGqLHglAv3NDB4KDxwrlE0m2UUoouBtdf732eQw1HjzaUSFZdsLuRgxrKbJ0rjtMPD/sNPNCQYRomnp68IESJoCYpuxknqNJGa+/FoKeHYvJvv01hna4uYMsWmlzU3l7DD9OAuHdSdslpNkv/V5Yg15pCM9yLcOWVZrhQA3rxQJyMvH2lnjPHVDCk0xKuEepLUOiAKzbCSiltz7G93YQa3n7bDAwvFqk5CpCCglLYd1InTtCdvd1R/PWvm3WVIvtx4YUNbdht4mHk7St1MmkkDJJJyprH4EAJDY7bJGV79zxNijn9dEoUAqbCg0smXQ2ckRHTBVuO7MFU1MFh7SCA/nfd3caA29VOvLyvDzh8ODbOYTyMvH2l5gPGNbCDg3XdNUEIxPbujx/36tocP07hmIcfphg8i54FYS8/ccJr8N1O2FJhi0ZjrIuVXc3U3+815CdPmjzd0aN0t89VNHzBjVFyOx5GvqXFjFtLpcyJ38DJEmEK4Hr3jz4KvPvdwMsvm+eCGvpsZs4EfvIT+pvVL8MqcNwEcKMasbEuVm6+w7370ZrKI/nuKJmkkNicOd6xizGxHfEw8oBp8y4Wgc9+lppGptItqdC48KzYsCQsi+qdf75/DsLRo971uD0/mfQbqTCVzEZjrIuVrQ8PBN8FjYyYZVpTeaSd+4iR7YiHkWcdb4AO/H330Ui1mBwkIebkcv46eZtEgjzMRx/1L7NfwxIJ/DyHJNySwUY3YmNdrHiUH98F8UXykktIbZIlDZhi0QiWlRKYa1TCRkbV42fC4//27aOxXXxdTiRoJJsgRJ19+7yj5njsn1Le89keH8g/6TQ9n0rROu5ypSI5qq4i2GMX3eeDxiteey09b/9f7R97zGLYtiMMSoz/i4cnn81SvfDKleTRJxJ+KVFBiCK2kJ5SwCc+QeEbvjNlz9yNzV92GXDttUa6eMcO/7YbrTt2PJU/YR43h2pcfvCD4CYnhitugPgkp0eJh5EHTEPIypX0hVi9mh5L/bAQZdzQw2mneatpwuYfPP88cOAAlUy60gfnnUeiW8PD9NpGcHgqVfljT4Ozec97qCgjLJHN8uNAPJLTFvEx8gCVStnlZDfdRM83N8fiiizEEDtOnskAq1b5vU2WIHbruXlSlG24kknj4NgOT9Q10CtZ+aOU+eGAzKFD3h4ahgeF8MVw9ux4JKct4qNdw3XFtgfEXYFDQ+YqLQhRgzVWBgeDPc1UCnjoIaC1lbz0dJoMlj0piitrtKYLxTe+QQ6PXfMdZfiOJpmcnHG1h6m7CdZCwfs4nabZu4BpKgPoonvnnbFxDOPhyefzwC23hN+KBZWTCULUYENn13cnEsBnPkPLd+82Xd3Ll5u67qEhWm5PizpyxGw3kYj++V9O5U+pmL3d/BQ2qzWZpAvm8DD9TzZuBJ580twhDQ/75+zGgHgYebc12SaVooMZkwMmxBg3dNPfTwnBxx4DvvMdc2cKUF337NnApz9Ny12v1Sbsu+FSKcmDiW6nVPliUMwe8I7zGxoi4/2xj1HOws1nJJPA/fd783Rbt3rf54UXqMwyJklXIC5GvqWF4u7vvON9vrWVhvDG4EAJUwTb0K1bR2EEe5oR68xnMn4dejfezHDNPNfNs26LbYyB4MTneA12JaUT7Pd2Y/a5HA0x53F+HIopFslIp1Km0o6fLxS8yp4ASTnbg9VF1iCisAe0di2wZ49JVM2bF4uDJExRbLmOdNrrhfb2+nXoWdKDdVjs5+0pR93dtC27hX/pUn/iEwgfdhL2vSqVQB3PBSNo0IqdEAXM+7jVR3anKwsV8sXRDVtxVR5LOdtzXqMe4iqTeBh5gE6aJUvI2LOGTUwOkjCFsQsJ3AoZu+JGKfJK7dmvAAmf9fUBx46Z17kzT7mu3DaimYwZaFIs0l3yzTeb9cI89LBu1PF6+EGDVuyYPUCevCsFkUh4u105t8Gj+wC6Q7IvNO3txti3tsZuyFB8qmvyeaoq4Nu2sPikIDQK3CilNf22K2S4ATCdJqM2bZoZ+J3J0PLDh6lJyjbwgJl52tRkSgiPHKH4/vLl5DWvXm3mnzKFgt/Td+G7arc6JcjDL0VQtY096cme6mSXQZ5zDvC5z5lB3YDRpgHoQnPHHbS9m24Knq61fTtV6s2fH4/pW2GtsPX4mbCsgdbUhuy2LNutyoLQaHCLfiJB0gWdnf7lra1az5tHbfv2ud/ZqfX55/vb91tbTbt+Z6dfLqG5mdbh71KQDEBz8/hb/vmzJJPlSy248gL79tF3esUK73P8P2IJiOnTte7oINkHfszbsj9vkOyDvU4y2TDyKCgha1B3w27/TMrIB2mATORkFIQo0dnpN1Za+8931xifd57fOLuaTkGOERs3Vx/Hfp8g56kcvZfJaMK4n5e/22z4580zhp6Ns/1+vF5zs/czu4Z8IhejCFDKyMcrJt/bS7eZBw4Ybe2YZMiFKcrgoCmdtBOZvb3eBKt2wpNupRlAFWh2nqqlhcI9ttYLj80ETJx/zhzTPdvc7J2Y5E6uKhVvn4zCo/t5h4a8FTbJJH0We44Ev19Xl9n/VApYvJjq44NmTsRFqdMiPkYeoAOyYYM3wSPJV6GRCUtkugaajbMerSy74ALgzTfNdricGDByH5ykzeWAgQEaWH3KKWZKldZk4MN01jmZapdxVqv0MJPxl4gODJg4P0D5BH7eTp7ecosRfBsZoaq7jo5wQx4zueF4GXkglldiYQpT6nxetswYZ3uqUTJJz3V0kGbLBRfQEJ3Dh6k4gS8MDz8MPPigtyKnt9dU7SQSZnxmkOHjZCob3qAyxXLKJssZ5bd6dXBTl30BPOUU4N57jdHv7gauu877Ou5+j5khL0lYHKceP5OKyQvCVCEsbmzHnTlZe+21FFd3NertOH1zM22rqYmSrvx4rJi0vR9NTf6EKO9LqW2V2gbjJkztworOTq0XLqTP6Wrqc/6gVPI6JmBKxOQFYaoQ1nBkx+o5jv/II6W3xbF+PVqmuXMnhYG4rryUtxt0l5HPUziou9vr5YeFcezPUigAnZ3Atm10lzJnDt1JZDJ+TZ90mpbfeqvR7nFpaqJ1mLE+T0yZlJFXSt0D4GoAJwG8DuA6rfXx0WVrAFwPoADgVq317sntqiAIAEqPv2tpCdafLwWHZ9gHLhSorrwcg2iHPYJi9EyYrj1/Fn6NHhVY27zZ7Ftzs+m2ZU2fgQFS2gwy8EpRcnXRIm9Xr50wnkJMthlqD4Df0Vr/LoBXAawBAKXU+QD+HMD/BHAFgAeVUsnQrQiCUD5BDUf5PHVyHj48PiOvFCVV7dcEdYvz9sOag/J5asRyDXwyaZLCq1f7X8+f5cYbyZgHSRScOEGGfc0a2tdt26jJy1batN9v8WLKRwwOjq8BK6ZMypPXWj9lPfw+gD8e/XsxgG9prYcA/JtS6jUA8wDEoH1MECJAkAfNYl12CeQnPkHKimFKlHp0oAajFCUrbS9+LEkCLlG0J1rxti66CHjppbFFv2bNIj2drVu9gmG8j7ZapKvNY6MUqXLu3u3Xu5milXaVjMkvA/C/R/8+E2T0mWOjz/lQSrUDaAeAWdx6LAhC+dhxbVet8u67ybu/6abyJIdTKRPW4KqXo0dLi47ZJYo2TU1UZ3/4cLihtWvYOSxz8CBtj716rcmwb95Mn80NBdnwfgTp3UxEKC0GjGnklVJ7AcwIWPQlrfXO0XW+BGAEwBhZHj9a6y4AXQAwd+7ciQnOTLGDJgge3Bh9kFpkKkXLEwng4x8H3n7b68EzfCGwvXcetgH4DXXYLAe+I+Aa+yDRL/cCMTREYRke+sGqmnbcPWwwkPvebkOU/Z4xG9Q9FmMaea31glLLlVJ/DeAqAPNHS3kA4GcAzrJWmzn6XOWZggdNEDyM1RvCQmcAGcDTTqOhGkFobbx3Oxl6zTXAb35Dwmb29nmWgxuLTyS8iU7uTN2+3XxH7f3ifeMGJz2q657NAs89593HRMLMcHVH+vHyDRuC7UApKeSYMqnEq1LqCgAdAK7RWv/GWrQLwJ8rpZqVUh8EcC6A/UHbmDTjVbcThDhiKzS62IqOySSwa1dwyEMpMtiZDLBli1mnWAQef5yMs5s8tROn6bR5PmGZlrDvaCbjn8H62GNeeeUXXvDvJ48/3LSJ1DfZ6NtwExfDiWMux5zsLNkGYrIx+Y0AmgHsUfRP/r7WeoXW+odKqX8CcAQUxrlFa13GfdYEKFVOJgiC19M/etSUJ9o0NVFtOne/umERTqoGeb92SKSz03jhvF7Yd3Rw0C9VYL9vWA6Bwzu23II9AtAt1wwaQDLW8JMYMdnqmg+XWPYVAF+ZzPbLQmQMhKlMufkoNsT5PJUg2po3QY1PrnAZa+Ikk3ShyOf979fW5hUM27+fEr5z5pBW/VtvUSKWXxckkFYOWlOzFTdL2Z+dk7irV5shK0EDSOwRgHEnrBW2Hj8iayAI42Cisrhhuuyudvu8eUYKIZGgx7ZkQpBEAG87nfbLELgSwXfdRdtobaXP4MouBMkw2HIMLIMcpBdvSwg3qHzweIDIGghCDJloErFUxYlSwIUXksftKrpeeCGNEmTJhJUr/SMJs1mqpAmrZXclgrlYYtEi4J57gNde866/cCHwzDOm2kYpU17JIaShIXM3ExQWmuJ3+2LkBaFRqVQ+yr5YABRm2b+f4uvuXNUtW0ysfGSEDLa9PJfzNi4F8d3vGh2akyeN0T9xwrue1mTgN240YRne3+PHjSRysWhGHoYZ86mkOukgRl4QGpVKeah8sXAHjfT0UHLz8GGSLFiyhOan2nrzW7eSkeUGLFuUDABmziQD/PLL5jnWuecY/8GDwXo3AF143Bh6Nkuxfk7ajiWJPMVRulT3WI2ZO3eu7uvrq/duCMLUo6vL3xXb2Um/b7zRPNfaSiWYbuWL3Z3qPh9mY+bNI+PPqplB20un/WGofJ4uTJywbW4mj38KG3el1Eta67lByyYrUCYIQhzgckamtZW8+J4e73pvveWtgWcSCW/9+bx59FyYgW9uBs44gww1ywfb78+vCxJbs2P+QVo7ggcx8oIgeBumpk83owKXLPGud/31wLnn+l9fLAKf/CTwwQ9S+eKGDf7mqKYmunisWEFiZE88YYx5UxOpR7qGnuc0M1wCar9uikoIl4vE5AVBCB8AYmvJJJP0+PTTgVde8b5ea+CpUVFaOyEKkOG+5hq6cLDHvW6dV2rhuuvIWO/ebZKyfGFwtXLc14kXXxIx8oIgEHbD1B/+IUkM8DARwExuSqepnDFIeZL55jdNSEVriuPPsHQO9++n51lMjJux+EKTyQR3pboVReLFj4kYeUEQDG5S04WlDS67DDj/fBrc8eKLfhmEc84Bjh0zj4tFM9qPpYPtZcxY1TFTvOZ9IoiRFwTBwDNibZJJ4NJLvWqQzz1HFwROmnJIJ5EAbr+dYu+XXeb19vkC4TI8TCWaa9cGG21XukHKJMeFGHlBEAwtLRSKYUOfTAIPPkihE1fy165wYZQCTj3VSATnciQf/PjjpSc67d1L8seuVLhIiU8aqa4RBMELG2028O3tZPyTzphmlvhNpYLle7NZ4KGHgG9/m6pySr2fPR7QpreXErGFgpEvEMaFGHlBEAyu7ozdSbp8uXddTsoWCsDFF9PyIE983TpSjEw5gQOuniml757JmJg9yxcI40LCNYIgEG4NeirlNbptbaRd41bVFIsUmvnBD7zVLm6o5aqrgJ07TVXNggUUhwfCE6mDg3Qx4JJKdxiIMCZi5AVBIMaqQc9maRrTypVGAdLGVcJ0VTJnzKBJTmz07URrWJydxwvKUKAJI0ZeEASinBp0exrT8ePAvfeaC4NthPN5Gi5iDwBvazOTpzIZE1+XksmqIgJlgiAYyp00Za+fy9Hf3NBkh2mSSTNWkLfX1WUmODU3kxEHxJBPglICZeLJC4JgCBooUsr4BtWsu/r0s2aZdfJ54OabzbKwISJi6CuGGHlBEIKZaI16qWEmuZy3O5bLNScy4UooCymhFAQhmKDxguXAcfQ77xz7wnD11RTKKVVGKUwK8eQFQQhmMuMFw6QH2tqoTHN4mIw6i5YFJVfHmx8QApHEqyAI4VTD0HKytrubSjGDQkEiZzAuZDKUIAgTI5ul+aqVNLDZLCVjR0bCQ0ETDRUJPsTIC4JQe+xJVEGhoLGWC2UjMXlBEGrPWE1O0gRVMSQmLwiC0OBITF4QBGGKIkZeEAQhxoiRFwRBiDFi5AVBEGKMGHlBEIQYI0ZeEAQhxkSqhFIp9UsAP6n3fkyA9wH4Vb13osbIZ44/U+3zAo37mf+H1vr0oAWRMvKNilKqL6xGNa7IZ44/U+3zAvH8zBKuEQRBiDFi5AVBEGKMGPnK0FXvHagD8pnjz1T7vEAMP7PE5AVBEGKMePKCIAgxRoy8IAhCjBEjX2GUUp9XSmml1PvqvS/VRCl1j1LqX5VS/6yU+rZS6tR671O1UEpdoZT6kVLqNaXUF+u9P9VGKXWWUuoZpdQRpdQPlVK31XufaoVSKqmU6ldKfafe+1IpxMhXEKXUWQAWAjha732pAXsA/I7W+ncBvApgTZ33pyoopZIANgFYBOB8AP9LKXV+ffeq6owA+LzW+nwAlwC4ZQp8ZuY2AK/UeycqiRj5ynIfgA4Asc9ma62f0lqPjD78PoCZ9dyfKjIPwGta6ze01icBfAvA4jrvU1XRWv9ca31w9O//ABm9M+u7V9VHKTUTwGcAbKn3vlQSMfIVQim1GMDPtNYv13tf6sAyAE/WeyeqxJkAfmo9PoYpYPAYpdTZAOYA+EGdd6UWbAA5acU670dFkRmv40AptRfAjIBFXwLwt6BQTWwo9Xm11jtH1/kS6Pb+kVrum1B9lFLvAdADYLXW+u167081UUpdBeAXWuuXlFItdd6diiJGfhxorRcEPa+Umg3ggwBeVkoBFLo4qJSap7UeqOEuVpSwz8sopf4awFUA5uv4Nlz8DMBZ1uOZo8/FGqVUGmTgH9FaP1rv/akBlwK4Ril1JYBpAE5RSv2D1vov67xfk0aaoaqAUupNAHO11o2oZlcWSqkrAHwdwCe11r+s9/5UC6VUCpRYng8y7gcA/IXW+od13bEqoshT2Q7g11rr1XXenZoz6snfrrW+qs67UhEkJi9MlI0A3gtgj1LqkFJqc713qBqMJpdXAtgNSkD+U5wN/CiXAvgrAH8wemwPjXq4QgMinrwgCEKMEU9eEAQhxoiRFwRBiDFi5AVBEGKMGHlBEIQYI0ZeEAQhxoiRFwRBiDFi5AVBEGLM/wcJWgx2BA+3CwAAAABJRU5ErkJggg==\n"
          },
          "metadata": {
            "needs_background": "light"
          }
        }
      ],
      "source": [
        "# Generate synthetic data\n",
        "x = np.linspace(-5, 5, 1000)\n",
        "t = x ** 3 - 20 * x + 10 + npr.normal(0, 4, x.shape[0])\n",
        "plt.plot(x, t, \"r.\")\n",
        "\n",
        "x = torch.from_numpy(x).float()\n",
        "t = torch.from_numpy(t)"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 31,
      "metadata": {
        "id": "yphlCjMmmYr9",
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 630
        },
        "outputId": "bb55c52c-e85a-4b13-ad4a-f1cd800b3d79"
      },
      "outputs": [
        {
          "output_type": "stream",
          "name": "stdout",
          "text": [
            "tensor(349.4772, dtype=torch.float64, grad_fn=<MulBackward0>)\n",
            "i: 0     loss: 349.4772\n",
            "i: 500   loss: 26.4050\n",
            "i: 1000  loss: 14.5051\n",
            "i: 1500  loss: 14.2317\n",
            "i: 2000  loss: 14.1320\n",
            "i: 2500  loss: 14.0251\n",
            "i: 3000  loss: 13.9017\n",
            "i: 3500  loss: 13.7529\n",
            "i: 4000  loss: 13.5631\n",
            "i: 4500  loss: 13.3184\n",
            "i: 5000  loss: 12.9951\n",
            "i: 5500  loss: 12.6725\n",
            "i: 6000  loss: 12.3672\n",
            "i: 6500  loss: 12.0879\n",
            "i: 7000  loss: 11.7822\n",
            "i: 7500  loss: 11.5532\n",
            "i: 8000  loss: 11.3860\n",
            "i: 8500  loss: 11.2600\n",
            "i: 9000  loss: 11.1957\n",
            "i: 9500  loss: 11.1557\n"
          ]
        },
        {
          "output_type": "display_data",
          "data": {
            "text/plain": [
              "<Figure size 432x288 with 1 Axes>"
            ],
            "image/png": "iVBORw0KGgoAAAANSUhEUgAAAXkAAAD4CAYAAAAJmJb0AAAAOXRFWHRTb2Z0d2FyZQBNYXRwbG90bGliIHZlcnNpb24zLjUuMywgaHR0cHM6Ly9tYXRwbG90bGliLm9yZy/NK7nSAAAACXBIWXMAAAsTAAALEwEAmpwYAABPZUlEQVR4nO2deXgUVdaH39udTsQViXxuuI8zwoiyGY0LxlFx3KMouA0OKBAQkVEHZRwcHB1xXEZGREnYJIrrAO6KG3GBKIRN3McVHUURxBWydJ/vj9s3Vd3pTgIk6U7nvM9TT9JV1dVVXdWnTp17zu8YEUFRFEXJTAKp3gFFURSl+VAjryiKksGokVcURclg1MgriqJkMGrkFUVRMpisVO+An5122kn23nvvVO+GoihKq2LJkiXfikjHRMvSysjvvffeVFRUpHo3FEVRWhXGmM+SLdNwjaIoSgajRl5RFCWDUSOvKIqSwaiRVxRFyWDUyCuKomQwauQVRVEyGDXymU55OYwfb/8qitLmSKs8eaWJKS+HY4+FqirIzoYXX4T8/FTvlaIoLYh68plMWZk18OGw/VtWluo9UhSlhVEjn8kUFFgPPhi0fwsKUr1HiqK0MBquyWTy822IpqzMGngN1ShKm0ONfGunvLx+I56fr8ZdUdowauRbM00xsNrQTUJRlFaNGvnWTLKB1cYa7aa6SZSW2v8HDNAbhaKkGWrkWzNuYNUZ6dzcTTPaZWVQWQmRCGzcaI31phjp8nK7D1VV9vWMGTB/vhp6RUkjNLumNeMGVq+/3v5du3bTUiZzc62BBxCBadOSF00lKqoqK4Pqau+1pmkqStqhnnxrJ35gNd6zHz8+eehm7Vowxhp4gJoaa6Tj100W1ikogFDI8+Q1TVNR0g418q2Bxg6O+lMmc3Nh1Kj6QzeNMdLl5TBunBfWcd66u7mUlWlMXlHSGDXy6Y7zoisrIRCASZOga9fkRt8Z3/HjY0M3paV13+M30qtXwy67JP/sSMR+fvyNoKEUTc3eUZSUokY+3fEPjkYiMHw4ZGXZWLgz+kOG1H1fbq5dLmIrXmfMsOGYeK/e/XXhmJkzYcIEG8pZtcrOcwb+uOOsV99YY63aOYqScnTgNd0pKLAG1hGJeEa/pgZGjKg7WFpebkM14bB970kn2XWTDcj6UzE3boRhw+Cvf4WpU+3yQABycjbNwMdvVwdlFSUlqJFPd/LzrbceClljGwpZz9wRDtc1nqWl1lhHItaT32WX+jVsXCqmG4R1Tw3uxhAIWO/eb+AbI2Gs2jmKknKMuMyKNKBXr15SUVGR6t1IT/yx7UcfhVtvtfNzcmLDIPG56zk5Nncd6o+Nl5RYD96lVPqJD9VsShhGY/KK0uwYY5aISK9Ey9STby3k58OYMfb/iRPt30AALr3UGlHnUZeVWQ/ccfDB3vtzc62hLimpu/21a+vOy8qy3n0kAs8/bw27q3DduNEL77jsmvr2Ww28oqQEHXhNV5J5wC7O7Tzu22+3/zuPev36WG980SI45hi47DK4+WY777nn7F//gG1BgfX6N260IRtjYvdHxI4FlJbC9Olebr2IHdTV9ElFSUvUk09HXDhk7FjPe3b449yBgBc337jRDrbedlvd7VVWwv33x86bNi32dX6+jbvvt58Xmw+HPWPuWL3azvfjiqgURUk71MinC/6BTH9WSmWlDbE4Q++M8bHHQv/+sR71okV1DbBj331jXy9bFnvzcBk5H39stxUI2BuJf5BXBJ5+OjbbB2xYRwdVFSUt0XBNOlBebkMqbiDzjjvsX5cq+cIL8OqrNhwD1hhXVtb1sv0EAl48PRSCm26Cq6+GV16xyyORWAkDfxgoEIBevWDFCpuP7zx7Eeu15+bCmjX2fcbAwIEaqlGUNEWNfDpQWmqNNti/zzxjDfq4cdbA+1Ui99zTM/71EQzCnXfaAVXnZb/+urc8KytW28avaOm89+rqup8TiXgG3m0H7GDusmX2f43PK0raoEY+HXn8cZvb3rev52G7Ac477rCedkNGvqbGGvgxY6wBvuWWWMXIXXeFkSNjq2BffNHeSGbMgMWL639ScEQidvv+/ZkyBe66K3ElrqIoLYrG5NOBAQNiY9+RCEyeDJdcAocd5mW6VFVZb3nSpNj1E5GVZWUJrroKhg6FDz+MNdqffmqfCPzVqPn59kmhujqpga8hyLfk8gH780tgW694yk84bOUX6iuUUhSlRdBiqHShpMRKFNTUxBpYZ8zdgGowCD17Qvv2XiqkIxCAI4+EDh1syMcZ67hzLMAvbM137Mg6OrAutAvfXXQl676pZh25rJv7Mt/JDnYZHbz16MAP7FC7nb13WMcbld35v6ovEj9Z5OXVrZRVFKXJqa8YSo18OuEKjaZM8Yy68+ITnKcwAb7HM8brTvoD311wKesemMe6Jxawjh1jDLT//ypyku5Glqmhg6ylQ3QLHVhHhz22ocOXb7Nj5Fs6yFqCRLiCW+nVaTUvDX2Y0I/rbJjHH683BrbaSoXJFKWZUSOf5oTD8PXXsG5ddHpwHt+VPMK6SHvWkct30p51CYz1enasd7vb8qMz/9ZYm/V06Lo7Hd4s821tHTuyng6srX29zSG/xVT4YvKhELz8sv1/3Dhb/SrCA5zDeTzAyN+t5N/lh3qFVH6MgdNPt169ShsoSrNQn5HXgdeWJEEV608/Wfv37rv+FU+IThCkxvOmWcf/8Q0H8J593bUTO37/KR12CtDh3BPY8fDOdPhsGR2Kx9P+5UfJpjr2800AftkX+LD+/dx9N3hrK0/D/s47PeM8bhy89BLU1HAuD7KIQ5nw0igOoS8XcF/dbYlYrZ3HH7cVtU7GWA2+orQI6sm3FC7mHg7HiIpNnmx1wcYe8xpd999gQyI5v9Chx950uHYE24W/w0DdsE1hIcydG/sZ8U0+jLFG2lXGNuZcuxz5ggIb909kjH3HUm2yOS78LIs5hIUcTjdWxObo+z/TPz87W5t+K0oTUZ8nj4ikzdSzZ0/JSBYuFMnKckOgIoGAyI03SiQi0nW/n6SbWSYRb4jUWyd+XigkEgyKZGeLFBXZ7fq58Ua73L2/Tx+7TnFx4u01NBUXx26/uFgkL0+ksND+X1QkEgzKav5PduML2YePZO2vD7PLCwtFcnK8zw0EvH1zU5cudY9BUZRNBqiQJHa1ycI1xpggUAH8T0ROMcbsAzwI5AJLgD+ISFVTfV6roqwsNvskGISCAl57DVZ+tA1TuBMT/574bBVX3LRsmR3gnDLFdnFyVbCur6u/kfe4cXbZ7Nl1M3YCgdi8+UTMnu3lupeU2FRMx+OPw2mngQg78w2z6UtvXuH8D67lyQ9OIWjExvKHDIHu3W2IZtEiG7pxvPOOrfRVj15Rmo9k1n9TJ+By4H7gyejrh4Fzov9PBoY1tI2M9uRzcqz3GgzWesjnnCOyQ/bP8hNb1+9RBwKeV+331oNB6023a2fXycoSGT3arrNwoZ3cMhAxxq5TXGyX9elj5zXGk+/Tp+7yYNDz1gMBmcwQAZGxXOd9nnuacN9D/BOFMXZ/FUXZbKjHk2+SYihjTCfgZGBq9LUBfgf8J7rKTKCwKT6r1RDfOUmi8r3BIHTtyuonK5j9SJiBR33ENsHK5NsxxnrDzqOO77YEse0Ab7/devSuQbdfj+b44612zZAh1nMeN86mOLp4fyBgi6jy8qC4OLZitVu3uvsWiVi9+iFD4MorGUIJA5nO9VzL45xqj9mvQ5+fb71/P4GAipspSnOSzPpvyoQ15j2BAuBJYCfgQ9/yPYC3krx3CDbMU7Hnnnu2xE2v+XEedDBo/0Zj17VeeefOcr0ZKyDyfui3Ir17J/eo471zt32/t+6P9xsTG7vPyfH2I1H8222ruDj2M/yMHp08ph8IeMdojPzCVtKTxbI96+V99vfWKSryPsc91Rhjt51ofzRWryiNhno8+aYw8KcAd0X/32Qj758yJlxTX0gFpJqgdGKVHM+82DCK35AGg3Zyxt8Z00TGuLjYGnYXsnHbcZ+9JUazuDi5cXf75j9GkE/ZU3JZI114S35kG+9m5W42o0fHfj8uLBR/c1RDryiNormN/HjgC+BTYDXwCzAL+BbIiq6TD8xraFsZY+QTGStfDHwOhQIiczk91mi6zJWioljv37+O89LjjeDChfY9LqulqQxlfCzeGPs5xcWJj7GoSCQUkuc5VgLUyNk8FJs5FAza44w/rvhjDgY1Vq8ojaQ+I7/F2TUiMgYYA2CMKQCuFJHzjTGPAGdhM2wuBB7b0s9qNeTn26yX+PZ948ZBWRmTqi5hD1ZxCk/ZmLSIjW9XVNiY+KBBNiPFrynv8t3DYfvaLyrmmDHDzs/KgsGDrfAZeHLC/mbfLhvHX5jkL9YC+3+3brEaOX/+M/zzn/b/rl3rHmN+PnTvznHDhjE+MoaruJlDeYMr+JeN/Wdn23EAP5GIHQMIhTzp4uxsjdUrmU8LNLpvzorXq4AHjTE3AMuAaQ2sn1nk59c9afn5vHfP67x4Xndu6DyLrI+DUB3xBj6d8XYG76STrOSwS0HMzbUNQ5zm+6pV3oCmX5Penxp57LFeSqVLt/QXTAUCXiWqf9vGeDLEo0fD8uVW+tg/GOtvOOJ/vWwZRCL8mVtYRB6juZnuZgW/6/wVnHKK/ax4ROwNbPBgq4TpDHz8DUpRMgVXvOj/fTbDdd6kRl5EyoCy6P8fA3lNuf1WR4K79N1zdyEUDHPxISvgvSov6yYry6tKFbEn/rHHrLEfNMhrxNG1q6f5Hp8rH4+/jaDz/CG2Ebi7scye7a3rlrn9aN8e5s1LfHz1XKQGmMFA3ua39JcHWPpOD/Z45+bYJuGdO9uWg+6G4o6zhX4AipIyEv0+m+EaVz355sIZqb/+FY46Cs44g58nTueeR7bmrMjD7HzfbV6Bkgj062eLjXJyYiUMnGd/7LG2IMkZatfA210cAwZYY+hCIgMG1E239HeAcn1aAwH7um9fb91QqO77EpHsJjJgQO1xbBfcwFzOpJIc+jKbjeTY4woEoF0721B8/ny4/vpYQ55s2+679aenKkpro7zcPolnZTX8O9tCVKCsuSgri23T9+ijzHpsZ35gEJfInSBxFa1r1sB991kD6Tx11xFKxG5rxAi7vWCwbuw6P99+Znx8z40NuNz5goLYef6YvD/G7o6hvlCJv2Wg/yLNz7eGO/oZB1x6KTOrLuRM5nIpE5nCENtYvLQ0NpbfmG2rh6+0dvzXcDDojZ+1wph826agIKZNnwCTZBgHs4LDSeCBumIj/4levdo2/6ipsR66P5Tij137DWWCcQCgrmEcM6buPsS/v6GLLtkAc/y2unbljOHD+cvyf3Aj15DHIgYfusEa+dLSxBe423Zpaez8FnrEVZRmw38Ng/0dN+c1nCztJhVTxqRQOkaPrk0TfI3DrVJAn/8kLizKyfHSEHNybKpiTo6XF58oZbGxxOftpyI1sahIaghIH56VbDbKG+ZQ79hDodhCL5dOmSxNU3PpldZMM1zDtIRAmeLDDbj+8EOtN38Xw9k+62fOr5mZuFWe80pXrfKyZCorbabK3XfbbV54oZ2/qY92yUIfLUyQCPdzHj1ZQl95hCX05P9YY7OBnMdeUGD3E+yjrEsvdd/PmDHJnx4UpTVQ3xNwM6BGvqmJj7cFg3wd6cgjnM0wKWGbl55I/D5nfOPDE/HbdIOqm0ILX1S1+LOLBgyAGTPIrfqOOcF+HFHzMv15iOc5nizCNjQ1blxs+mc47ClmGmPHENzxqHFXWjP+BIOVK60zB80Tm0/m4qdiyohwTbymezAoN/AXAZH3+HXdME3nzrHa8AsXWs0ZY+xfp+VSX7glHfVe/AqYfuXL6H7e88eXBESu4FZP8iCRfIKTd3CyDul0jIqyubjfebKw7SaChmtaEH9oxBhqaqCYoRzLC/yGD2LXDYVsAdLatd68ZFkyycIt6Zpt4s8uikRg+PCYLIIL82Hxj6u5bfYV5O3/I/3evc5777bb2r6IUDdnXwdalUzAKcTG0xzXeDLrn4opIzx5kRhlx7mBMwVE5lAYe8cOBOzAbLy329A26+sGlU56L/HqmE73xueNV1aKHN71e9nG/CQr+W3sev5B2XgtnnR8clGUTaGoKPmTq3ryaU6c/stduXl0WvM5p/KkXe405SdNsh6839sdMcLmqie6iyeTEGiuQdUt1dTIz7fHOGJEbCVvZaWNvY8bRzbwyH8voIcs4EzmsIg82vO9tw1j4OSTrbQDeGMR6fjkoiiNxRXxhUJ1u7NdcYXG5NMaf2pUdra8H/qtgMj1wWu9lMB4XfgEvV8b3HYiBcrGeraNWXdzUrySbdelRMb3e3Ua9IGAvMKRkkWVnMpjEsZ43nwwWFd1M12fXBSlMfh/W4nGoTbzekY9+RbCX+QQiXC3XEyIKi6WqbDniFhxL4j1dsNhKwWwKRIC9RVBJaKx8ftNLTgqKYk9Bv92/e9butQqbbqUSIBAgKMir/EvLmckE/kH1zCWG6wXH4l4BSNuP9IkHVRRNov4Qqh4XAZZE6JGfkuIl+Z1WhTAz4HtmFH9R/qaOeyS811yYzRkSGLJ3niawrg11nhvymeVl8Mll9iQDNhwjH+78SmloVCsGNn228PNNzOCO1lEHn/jOnpRwYmRZ73PcCGuVavs6wkTrKBa374aqlFaF3GJGbW/G7Cv/UkYTUUyFz8VU6sK18SFZmoHB7OzRQoLZcrR9wqIvDK4tOkGCLd0wHFTwjCN/awbb4yt4PVXr7rl8V2y/NuNtg0UkJ9pJwezTNqzTj5k39gwlut8lSiEsyX7rygtjf/adK01tzBFmObsDNWUU6sy8n7jZUxMm75IVki6sVS6mjclcnphbB58qmlK4xcfb0+UIdTQWEJcrvBH7CM7slYOYrn8TLu6Mcv4TITCQtu9yv+5Kn2gpDvxfZq38DepRr45SOTJR439QvIFRCYzJNbDTSdjv6XEH399x9aY9E9f6uTT/F4MYTmP+2JbB8anY8anaDpDr4OzSjoTr0/VzNo1qie/uTipgOuvh4kT4dBDa83NJIazPd9zPrO89aurPV34TNBB98f3w+H6lfTy863mTDI5YaenHdXDP5Fn+TvXcj/nM5FLE2/zqKOgR4/YedOmJd6uDs4q6UB5OQwbBhdfbMeuJJpSnEjKpAnRgddNpaTEG/Rz2TLHHgsbNgDwDVanZug2s9i2qgr8abCSQVWbTTEQHK+pA/aCnzaNv1TfyGIO4QpuozvLOMos8JqsACxYAEccEbu9Zcu8dogqYqakE+XlseJ7LYga+cZSXg433wyPPmpfu+bWa9fGnLhpXEQVOQz787bQ52VrtN55xxolkczxLJvKkCbSsB8wgEBZGaVbr+OQa9Zw9s+PsFR6sBtfeeuFw/DKKzYjwRn/SMS7gfq32wLNkhWlXsrK6hY+weYJDm4qyeI4qZjSNibv4s/xceE+fWJi0zWBkOwZWCW/a18RW/DUWOmCtkgDg05v9btOtuFHyWeBVBJKrMVfX3aCDsIq6UB8kkETj9GhxVBbiIs/x7P11vZvtIPRU1O+ZlV4D/61fhQc/QS8/LL33kik+fJgWyv1FWdFve/fVi1jOoPoz8P8iduZdNoL8OWXsGiRt51IxNYnTJhQ11PXTlJKOpCfb8fupk2DrbaCLl2ateWfHzXyjSG+gGH//eGDD+Cxx+Dpp2v1ZO4KD2E3/sfpPAbVYRveGT3ae68r6HFx47ZOvAEuLfV6z44aVfud9QsJi6tv5VauJO/AE7hw9Bo45hivuQpY/yjRDVQrZJV0oLzcXtNOq+q112DmzBbRXtLsmsbgz6R55RU4+mhrmNxA6s03899przCP3zOUYtsEA+xNYOVK+97Bg+0NYsqUzMmw2VL8WTDBoG1ePnasraCtrPQydy66iPHX13BMj+8puvVXLH1/Gxg4EAoLrYyCy6LJzYXx42O/W/+5UzEzJVX4n+ghptuZCJx2GtxzT/N8tHryjSF+4C4+5WnhQiZXX0UW1QxmijdfxGq6vPyyTTGsqdGwgR//4O2qVfYGGA7bTlDBoL0pRgemsvLzeWgo9PzNT5w5cHuWmP+QG/oBTjrJqlR27+55//GhH+0kpaSS8nJP8kTEC91Gu529+io88QScemozfX6yYH0qprQceE00cLdwoR04iQ6i/Ew7ac866ceDdj2/HrpTltQBwPqJ/34SKXYWFcmiwKGSzUY5nnlSQ8DTqC8q2rICKJVBUJoDf+KFq9AePdqT6WjXTgp7r5XcXJFfftn8j0EHXreARAN3ubmw887wxRcAPMg5rGdHLmGS9UKPOMJLmXTKkpq7XT/1fT9O5bKmhkNEmMQlDGYqY7meG+Uae15Wr7bf/eakqaZrdy2l9ePvkAbw1FP2yTPaQ+Ljyt157NUdGTMG2rVrpn1IZv1TMbUKT3706Jj0vQhIDyrkQN70SvBdf9ZMkjFIFYk6TIEMNiW241bgTPtduyerYHDT01RVBkFpLhL1jCgsrPXkRwXvkKxgWL74Yss+BpU12ALiB+6WL49ZvKjjKSylJ8OzSjDG2JkiDZf6K42jrMzzgsDG6ouKmDi/K3ldfuTC0P28d1A/r9AkHLaVr37Ky+sOyPpRGQSluXA9I0Ih+6SZlQXPPAPhMD+YHZgWGkr/cwLsvnvz7YKGaxqDf+Cub1+v2hWYtN9tbLcRLphzAcyusRkiTi9djcWWU1BgQ16VlfZHMmkSDBlCDvCfZ6HnQdWcUfEXFvEo2xFt/r16tdUIWb3avn7mGe+cJArFaChNaU6c/Mns2ba25oknIBJhuvkjP27MZtSo5v14Yz399KBXr15SUVGR6t1omKh+zZo+59Pp6vMZ3GEOdx5+v82JBzUWTU18cxbf9zv/tNs57omRFPIo/+EsjMvMSVRCHgzaJ7IxY1pu3xUlvnGOMYSrI+wv77Nb11xeW7H9Fn+EMWaJiPRKtEw9+WQk0zspL7fhgH33ZfoTO1FVE2T4N3+DR9+FJ5+0efRqRJoW9yQV/2M56SSOefYpbuYLruQ2bjZXc9Vp79n6hHhcOqY+XSktTXzLv8GDeeKnY/nkvn24eWzzf7wa+UQky7bwKcmFCXA3V1HAfLrwrn1fTY3NoVcPvukpL4dx47xMhXC4Vizucv7FIvL4C/+g5zHvcdzTT9eVoQgErOwB2Pi8PmkpLYE/Rx6scwJMePt49trL1vM1N2rkE5FM78SnJPcMJ/IZe3MrVybehiofNh3upusMvF95EjDAtOzhvL37qZxz7f4sOftK9vr5HSs98e673rrLlnml5b74fu1n6PlSmhK/s2gMdOoEn3/OsuJFvCw7cOuIT8nK2rvZd0ONfCJyc2NzrnNzvYG8rCyormYSl1idGvME1sxgR9AHDNC866bGXxIeCECvXlBREZN1s+1JvZnb43F6XXsiZ846k9dCd9Lu5N/Bf/9rn7CMgaVLYeNGr+pwxAj75mXLYgfM9XwpTUFpqXe9AXz2GQATGMk2/MRFO84BLm/+/UiWW5mKKS3y5OOlgUePjpUIDQblQ/YTEBnHtXa9+ObUmnfdtCSqFi4sjM2d79xZJBiUxzlFQOSPTK/bOtBfiexeh0Kx89350gpYZUtIUt/xFTtLiEoZYSY26bWFVrxuAvHSwMuXx2ZqhMNMZqinUxMM1pUMVeXDpiVRiuPo0TY10ilRvvceiHAqTzKWv3M913Iob1BEsbcd51E53ZBAwBOac/Pdk5s+iSlbQlmZN9Dq426GUUMWI4N3wcpRLRMiTGb9GzsBewDzgXeAt4HLovM7AM8D/43+3bGhbaWVJ+/XUPF58r+YraUD38pZPFJ/ExD1BJufhQtt45a4RiI1BOREnpIQlbKQw+p4UwL2SaC4OHEz8qIiz7vXJzFlc4hvEgKygRzpyNdyKo/ZazYUajItK+rx5JvCyO8K9Ij+vx3wAdAFuBm4Ojr/auCfDW0rLYy8iGegi4vtD76wUKR3b5FgUErNAAGRF0+93RMrU2OeOhL8mARkLTvKvnwou/GFrGbnuqGaPn2898cLofm3l5Oj51bZPIqLbRgxeu1NY6C1HeZY6yA656QJHIlmNfJ1NgiPAccD7wO7incjeL+h96aNkRexJ8jvIQaDIoGA5LNAfsO7EvmHKkumDX7P2zctN92kHT9Lb8qkirj4aLInMP94ijF224qyqRQXe0qT2dkSOb1Qunb4XLru/q21Hf6nyGb25JtUu8YYszfQHXgD2FlEXOfl1cDOSd4zxBhTYYypWLNmTVPuzuZTXg7Dh8dqpoTDrAh0p5zDKQpNwxxTkDjVUml5BgywmU1xHCzLmcJgXuFoRptbvVzlQNxl79e28evYbLVV8zdZVjKP8nLb+Ka62tqQmhrm73Q2K9d1YtTfczF/GWNTd1uqmU0y67+pE7AtsAQ4M/p6fdzy7xraRtp48jfeWMcrlGBQhvZ+W7bKqpK1zy6y6/nv1urJp5aiorrnLDqNZIKAyP2c680PhbxwW6J+ARqCUzaXG2+sEwU49ci10rGjyIYNzfORNLcnb4wJAbOBWSIyJzr7a2PMrtHluwLfNMVntQjOm3MYww+3TeG+JV0454IQHU44xOvZ6DoZJWoirbQcAwZYQW6XNdO7d63HfqsZzVG5b3MRU3mTrnb96mqvp2yiwrcxY/R8KptHQYH31Aj81/yaJxfsyLBh9uGwpdliI2+svu404F0R+Zdv0ePAhdH/L8TG6lsHrrO6a0EXCnHf50fz88+2JgqITbUUSdxEWmk5XJrlP/5hmyTfdFNt/9dQCB7e40ras54zmcN3tLfvmTbNpktmZ3shnEWLtP+usnm4sB/AoEHWdgB3hIeTFYh4tqOlSebiN3YCjgQEeBNYHp1OAnKBF7EplC8AHRraVtqEa+JS8yKBoHTt8Ln02O1LiSzwZWHooGvLsqlhFJcOmZMjEgjIQg6TEJVyEk9KmOhAbZ8+tuDNDbZqRo2yqfivM3/qdbt28l2gg2zDjzLgxK+bdRdoyeyaLZlSbuTjjILr5PJa1tECIlPM4FiDrrHblmNzbqoJ8ujvokhA5G/8TWozaLKyYrNzjNHceKVxuOsySdX0rSe+ICCydGnz7kZ9Rl4rXh1Ob8avNREIwHHHcfcnf2f7/37PuTIrNm7rbyaiNC/JYufJiBc1i1JkSnhDDuU6xtGLCk6Rp7xxFVehqFXKSmNx16WzGY7cXGoOyeeOt+Ho7t/T/dm7YGNBSuyFGnlHopNlDGv6nM8jV/VkCJPZxmwAE7RxXKVl2RSpiHhZYp9qpTFwtwznTQ7iAu6jgl78KvAp3HWX1zYwXqbCbbO0NPlypW3irku/MxEOw8iRPLrmaFat+g3/Xj0E3pydOomMZC5+KqaUhmv8wmS+9Lt/mqsERN6ii/corzH41NCY8Fj8eYwWo8TES3v3lk/YSzrwrRzIm/LTnp3rNl33f5ZWwSr14cKCcSG/I/ZaJft2WCc1AV+T+WYKA6Ix+UYSd7LCGNmXD6U3ZTGGX/VM0hh/xWogYM9nfO571Gg/x3ESoEbO4X6rWOmMd3z8P76iVmP2SjxxjsAikycgMqHfghZJ0FAjvyn4Ttaz9BEQeYD+sUZePbn0pbEDtNEb+niuFhD5F6M84x0vUFZUpJ680jBOAjsQkPO4T7bje/k+eyebadPMCRr1GXmNyTv8nYHKyqC0lLsfPIeO67/hDObadaJ5r4gk2YiSchLJEidbr29frnphOIsih/BnbqF71tsU5ObaeL7/HHfvXntNABqTVxKTnw95efzv0cU8TD9GcCfbV6+1NTQp7PusRh7qNogeNIjPT7iYJ0p6MDpwCzlSDYGg/eG7/qINZXcoqaMxWU/l5XDppZhImHvMIPK2eYv+oSdZ8nEJnfw64JGIrWx+8UW4++5m3W0lAygo4K6AIRwJcikTbeVrijO1mlSgrNUSn55XXMyUs59DBIZmTbPrBAJWBCsY1BS7TODmm2ubfW8v3zP35Gn8Up3NWQ+fTaXZKvapTcXnlEbyy8YAxVnDKeQx9jWf1hXDSwGp34N0wKVBRX/Y1RJkas2FnJi7iL3DH3ke/MCBLaMapzQv5eXwxBMxszp/8jQzz3+ONz7Zmctqbq2Vs9CbutJoysu5r08pa6u2ZxS3W7tRU5NyB0HDNeDFcUtLYdo0Hq8+ja/YjZL+b8J0X262xmIzg7KyuuMqixdzZsWJXM0N3MQYDoks5qLT1kFenq2LcD9UPf9KEmRmKRNqLqU7SzmKV+3MNAjXqJF3uB/v9OncTRF78hknnrMDnN+IQTyldVFQYMXL/NXN0dyZG/grFfTiEiZxcOB2ehUUaL9XpVE8/0Vn3qULMxmAcTMHDkz59aLhGke0SvKD6n14keMYYqYSfLVMZWczkfx8Kw293351FgWJ8ADnsjNfc+aCy1nz5BuJG8P4G40oCjDhs0J24Sv685CdkZNjM7NSfJ20bU/elaqvXg3PPAPV1UyWW8iimoty7oOC+1O9h0pz4HoBVFba106DHiAcZifWMierH0d8V865c87iWTOGLBOVR8jNjc3GUu9eAd697mGeeasff+dacoJhOLUQTjzRXmcpvk7arifvWr1NngyPPgqVlWyIZHMPf+TMnReyy0v36w83U/H3AggE4Pjj4dVXYfDg2sH3nrKEyYffy4vvdeKamnE2nBMO2x9taann3VdW2rx69ejbLuXl3HHdd+SwkaFMttdKXp7Nj0+D9qBt18iXldnuQD4eph/f0YFh5/+gBj6T8fdxzcmxRjo/3w6sb7WVnR8M8sdvbmYYd3EzV/EIZ3nplKtX25uDMfZG8fzzthNVSUmqj0xpbhKE6dY9/Toz5Q+czyz+jzX2uli1ymtIk+IMLSNpVL3Zq1cvqaioaJkPc558NFca4DDK+Z4deOcfj3qNunXANTPxVzjn53uvc3OtGuWMGVBZSRUhjuZlVtKVReTRJeu/9kfrHAR/s/dQCF5+Wa+XTCVRmA74559Wc/UbZ/AmB9HVvGWvDxG7zoQJ1qNvZjtijFkiIr0SLWu7MXnX4q+oCERYRjfe4DAmBC7H7HSAxlwzHX9VbPyP98ILbX4zkE01/+EserCUM5jLokg+O0S+9ySMfTLGWgmd4cQXTZaWUj39XiZWvcexvEjX4DsQofbaoaoq5ZIG0JbDNWA9tugP9G6G0Y5fuPC2g9Imlqa0EPE/Xojp+7o7X/IIZ/MR+3FhZLp13p1x9z8Jh0LWY9PMm8zEH+bLzobVq5lddQr/o5MtfnL9nsFeH+lSRJdMuSwVU7OoUCbTIPepTa5ne9man2RQ10XeMu3f2nZIdL5dK8jevWu16ScwUkDkH4yJlR52k9Okz862y7Oz9drJNNx1UVQkUlgoh1Iu+/O+7RkcDNopO7tuf4JmhjYrNVyfsfbpjk/kEgGRxTlHaP/Wtkr8+U7QuzMCch73iSEsz9IncfPvoqK6hl/JHHxNacoDhwuI3Mkltk9wYWGscW9BG1Kfkc/smHx9fUGjj16yYSOTKaIXi+lV87r2b22ruHPtQnNlZTY90heOMUAJQ3iLAzmXB1jy93ns8/krdqGTI169Ona7S5fasI1eS5mBuy4iEW5nJDuwngtPWQvPB60ekpM/SaNaisw28g31Bb3wQl57a0fefu1ApprB6RNDU5qe+GyaRMv9P8pLL43NnAEIBtmmU0fmhK6i15ePceYj57JgwblsvSJOqjoU8rJvKirsMh28zwwKCiAQYFVkd2bTlz8xgW3fq7DnOxKJHcPblMbzzUhmG/lkDSR8P+i7mcUO7So556rfQB/9IWYkjfGq/E99lZUwZ05s5gzY/7/8kv0iXzDL9OOU5XMp6ruWmUeVYdx7wRZVffwxvPCC/eFv3GgLqPTaav3k58OkSUwa9gMSMYxgInz0ub02jLE3eecoNrbxfHOTLI6TiqnF2v9F27t9TUcJUSkj8xe1zOcqqcHf9zVZf974BuAuFu/+BgI27upr9H4d19qY7Dmv1h37KS5OHLNXWj0//STSfrtqOXvnl2OuhzrnWWPyKaK8HKZOBRGmM4hqsinauhRKltmUSlBJ4UyjobAdeE9948Z5HnggAMcdB3372rTa3FwraxBVr/wr17OYQxj18Il0u2sRR6x7wtv2qFGx4R6nK67XVatn5kxY/2MWo27dES7NiimopLo67cb12l7F6/jxcM01RAT24yP25lPm87vYdXJyYP78tDhBShPRUEzev16Cqsba94LtKvXEEyDC+uz/45CdPuanmnYsXQq77oq9xsaO9cI3xli5hBaqflSaj0gEOneGHXaAN94AM3yY1b9ypKjqub6K15SHaPxTi4RronnMT/N7AZGHODv2cUtT3xT/Y3Z8Gm5xsRfWycoSKS6WlStFtt5a5IgjRCorxa4TCtl1XM60e5/WXrReFi6Up06cKCAya9wHtfPir4dUQJvNk0/GwoVy6i5vyM58JZWEEht5jaEqInXj+Xl5sXH6Pn1Eiovlgf5zBUQuPfAle+3E/+gbMy6gpC9R5/B45slufCFV2dukVU1NfUa+TcoarFqdzVOre3IR08mm2j5OFxZaeVDXwDkNejMqaYC/lD0YhCVLvIwbp0A5dCjnPHwml3MbE986hnsrz/ZK3NeurbudVGdbKJtGtKHQ21X78zx9GMGdhKp+jm0JmcaNhdrewGtJCSXD1iJ0ZwjFdp6IFfjv2jU2Hqs/RMWfhrtoke094McZfBH+yVUspQdDKOFA3qZ78G0rOeuKoRKl8yrpjRujqaxkApPZig0MISopvX59Snet0SRz8VMxNXu4ZuFCqQy2k535Sk7hcS80Ewh4j89p8OilpCnxkgXx6XMgX9NRdudz2SfrM1kb2MmuozH41ks0zPYNO0kOG2QIk73zHQrZcFwa2AvafAqly6xYtYrHIqfyNbswjLvtskDAZtM4rz1N0p6UNGTAAKszX1UFWdGfjoi9hkQgEuH/WMPsQD9618znPO7lKU4mqMVQrZdomK1kYxGVshWXcYe3rKYGRoywobl0liRPZv1TMTWbCqXLasjOlmPMfNmLT6QmmC0yenRa3IWVVoQTIcvLiy2c8hdP5eVJsRkqIPJX/q4D+a2cypfLZdetv5MTzLzkxXEpHkynTXvyvnL19+Q3zJcCbuxTRnBcWd27bmNzqZXMpqHrYObMWpEqAgHr1RtjPbvsbLjoIga/OYpFGw/hBsbSiwpOr3lKi6FaKQ+vOoyvfoHpgX9DRLwiuY4d4YEHvOYxubmp3tWEZL6R91U7TmYYoawIF91bAP8Xt14aqcYpKaSh68DfBBygVy9b5OSWRW8MpmtX7px+PyumVfAHuZfF5PObNDUCSnJE4Pbb4YC9fqHP12VQHc2O6tgRZs3yVnRN3rt2TTu7kfkplNGshl/Gjmfm1kX0PSvA/8UbeEgsS6y0PRq6DgoKbBqkY8UK+9el0YGteAW2GnQes48vJodKzgw/zE+XXaPdoloZCxZYtehR3V4m8O/b4frr7U39/vtjVxRJW7uR+Z48QH4+D72Xz/ofYdiwJOs0Rt9EyXwaug7y82HQICgutj9svyZNeZzksDHsWVXFQ3zE8TzPoMq7eGh+GSbNPD0lDl+4bsLYX7Mjhj880Q+eC1sDP21arDop2BBOmtqNZteuMcb8Hvg3EASmishNydZtTu2avDz4+Wd4a0o55uWyutLDZWU2pqbaIsqmas87fRu/uJkrqov+vm7hSkZzC7f0fZ0r/3NYSxyFsjn4zu0nWfvzq6q3Gc0tjJer7fJg0NMkcuy/PwwcmFK7kTLtGqxh/wjYF8gGVgBdkq3fXHnyFRV2APyOPW+xua3xvTxVU0TZVFw9RXGxzbZxUgYu8yI7215rvtaBZ/GwBKiRF+94K9V7ryTDJz9xufmXZAVq5PPQPnXqIdJNAoUUyhrkAR+KyMciUgU8CJzezJ9Zh7sve4+t+ZkBq663UqD+eKvG4pXNIT/fem6jRtnQjcu2MQb23dfOD3g/LwNMZxAH8B79r96HVatStN9K/UTDdT8GdmCqDOLsY7+j08kH1/+eNJdAaW4jvzvwue/1F9F5tRhjhhhjKowxFWvWrGnyHVj/3CLuX7gX53E/O/CDt8DFz1RTRNlcEvSBRcR2hbrttlidcWA7fmKOOYuqKqHvCT+xcWPL7q7SCKKJGvec9DA/sAOjAnfESlkEAl4oDuz/aW43Up5dIyIlItJLRHp17NixaTZaXm4zHEpKKD3lITZIO4rwaT737g0nnGCrEMHGVK+/XtMmlU0j2u8zBmOsRx8Oe8Y/FLLXXCDAb+Q9SmvOo+K9bbm039ctvstKw4Tz8vn3u33I3+kD8uZdH7vwtNNsbwDnFA4dmv52I1kcpykmIB+Y53s9BhiTbP0micn7YuyRYJZ05m05hDds7KxDBysNq23ZlKZi9Ggbg3cVkMGgVxXpps6dYyWKQa7hegGRkpLoduI1k1RDqeWJfueP/fPdaK+JfrHn0RhvHC/Nzg2p0pPHpmh+DOyDN/D622TrN4mR9w2clHG0gMh0BnonKX7QxBjV9lY2D3/DiGDQDrQ6iYO4frDxUw0BOWHH1yU7FJbXp7yZuDGJJgO0HD7n8JjAfNlj27VSTTD2vPXpk3bG3VGfkW/WcI2I1AAjgHnAu8DDIvJ2c35mbYw9EOBuimjPd/Q3D9vHaH/s1JHm8TQljfFXv0YidlDfbxZ69YqN3/oIEuH+705k9+rP6DuqE19XtvcG/6dNs31kNRmg5YieyxXh3zI/UsClB5aR1S7bLjMG+vSBV1+1bR2PPbZVFbU1e0xeRJ4WkV+LyH4i8o/m/jw3cPL1UWcxhzP5I/ewtdngqQY6gkHbKER7uSqbi3/QPhSKrYQVgR497HxHnMHvwHfM4QzW/tKOc+QBagLRbS1b5jkkWVnqhLQEubkQCDCBP7E1P3Nxvx/suF1enu3hWlDQerPwkrn4qZg2N1zz+usiF1wQOx2w188CIu+ZA+yjc58+sY9eo0dv1mcpSgz++GxxsReTz8724reFhbHjQO3axVyL93K+gMgVBz9nc+7dusZor+GWIBqqWW12kWw2yvDuC+z584/bpXkIjUxXoVy7FhYujJsZ3JprT17Cb577CMIRW4noZ/nylto9JZOJ7z+QlWU9Pee15+dbb/Cxx7x1NmyI2cQFzOINDuO2FSM4pO9e9M+e6VXTDhjQAgfRxomGaibLEKrI4bIvr45Nf62qsk9XF15oXw8Y0Kqe/jPCyJ90kp3qMP45eDbiKQb66datuXdLaWuUldnCGInTtCkoSD4mFOU2rmDZQQMYdNOv+W3JQg78/BmvTF4lsJsHn5xJZaAdd4WHczJP8uuvX41dLxCwzWKclHQru/FmhJFPilMMDIdjf2TGQPv2qdwzJRNJJm6Wnw9XXgk335z0rdnBMI/c8AE9hvTijOu6sXhxN3uJxuvkTJig+kpNga93K4EAD+z7d775YGdGMaHuur/5Dbz/fmw8vhV99ykvhmp2nGH3G/hQSAezlKbHNetOVFj3z39a+YO8vLoFVADhMLveegX/Ob2UTz+J8Ic/QGRBuRU9q6y0Bqay0rabGzvWXr/DhrWqLI+0wlUrRyJITQ0TPjiRA1nJsbxYd91f/7pVV8VntidfWmrT2vyIJE1rU5Qtpr4ewUOGWC980aLEy195hSNee40JwWWMePJ2/vHs84yNvOB1oAoErLF3FbXFxbZLVbpXXKYjvgYuL3M0K+jGVDMYI8R+19nZMHo0nHgizJ4Nffu2uu86c418eTlMn554mT9eqigtiZNCSDROBBCJMJyJvLHryfztq7/Sk9c5KTDPtpvr29cKn23c6OXmtMLwQYsRP5bhlxQfNar26X4Cf2In1nAe0UYgxsDFF8Oee3pe+6hR9rt+9dW07P5UH5lr5MvK6uo+Q1qL+ytthGDQM/JO8CoSsUYnEMBkBZnccworn9qJ82UWFaEj2W/cOGtYuna1T6j+gUC9luuSaCzDGWrf9/0hv+JxTuWa/Pm0Wy5QFfQGV50hHz++bo68Gvk0wD8IFgzabj7du+uglZJayso8A2+MDeEMGGDnr19v/y5bxtbPzGZO1nJ6BpZyxvYvUP7XoWzTq7NNGBgwwHuPXsuJiZcQnz3bex0I1BauTYyMIIsahi8bDP+eEGsfysvtDXX1aq+YshXeVDPXyEOrzWtVMhB/qMCfgePS8Vatst55VVVtGGEfPuKBXrdyYvlYhrzUn/teugBjjFVBfPFFr6esUpf4TKe+fW2oxefZf//gM0yfP4hzeJBdqz6zN4Jx4+z7hw2z8hJuTC8UgsGDW6UtyUwjX14OxxyjBSVKelBfGiTYZS7O7ojqlJ8QfpobqOQabuRQ3mCkTGyVIYMWx2U6+Z92unaNeT397T78NH87LjMT7dPVCy/Y5cbE3GwBGxrbc89W+Z1nppEvLbXpUWD/lpa2ypOjZAjxoYO1a60XXu5LkYwvlDLG3gyAqxcVsYg8ruA2upk36R0st55/eble1/Xhz3SKG4QNh+GOx/fmqIN/oOfOHeB544nMQd3z0QrDNI7MNPKKkk4kKpLyF+MkyrQRqb0ZBICZD84kb/kR9Nv4GEvC3dm9pASmToVJk2xcX0lOgsbrU9/M59NP4bbZ28Mb3eC55+y6IjY0E4nYuP1JJ8Euu7TKMI0jc4y8/07dvbuXwRAKabhGSS2JQgcuYyNZKqXfcxwyhB26dmXuQ3PJmzSAs2pmUUYBOZEqWxwFmlBQH/4nqY0beWvCC4x6PJ/jj4fCncvhX//y1jXG2o8ePVq1YfeTGUbef6cOBj0Jg2AQJk7MiBOltHLii6T83r1IrLHv2NEOFIKX4TFjBl1qarhHXuRsHuJP3M5dXGJjxSNG2Pc3RvagLergOO0g4BfZiv6P9GWHHau4995sAtPLYr97EaiogJUrM8Y5zAwj779TuxPmKlvXrk3prilKQvze/fr1sbo269dDSQlMmWLT/ZzoGXAWDzOaHtzMVeSxiD8G7vOWb9wYa/DjK2EThC1araFv6Gblz2ZatqzWLozkDt6VA3ju8Ins/HGeHdsIhWJj8ZFIRg1uZ4aRLyiweayRiP3rfhiteLBEaQPEe/dz5sA228CKFd68BAV9/+AaltCTIiZzUMc19Fj9tF3g1C+TVcLGDwC3ViPW0M0qfrwj+mR/P+cyjYu5hhs4bt7f4fmonQgG7bhG9+5ewVQG2Y7MMPLglXlHIvCnP9mikbb0SKq0XsrLbVgx2SCsE9Xr0gWWLyeLMA9wLj1ZwpmrJ1FBL3ZirV3PSSYEg3WNVDKVzNZGQzcrf1tGABH+y68YSjFH8irjGAc1vid+EZseOWRInTTLTCAzjLzT8QZ74m+/HV5+OWNOkpLhlJbWzZP3EwhYD3POnNpZHfmWOZzJkbzGuTzAs/yeYMB44nsidrvg/Q4SDQC3Rhq6WUVb+bmnoEpyOMc8RHYwwv2RP5AlkdjvOhLxBMvqE5hrrSRrGZWKaXPb/8nChbbFn7svBwK2JZuipDsLF8a2mnNt/4yJvZ797QPdFArJNHORgMgYbqy73Ji0bFXXJPjbLsbPj2uvOJIJAiKPZ/eN/V79k7/NYrJtpzFkevs/8vNtvvCIEdajDwRipEQVJW3xC+kZA0cdZcM37snUeebxsfneveH88xm0di2LSuYx/tMx9GIxZzLXW6e1qVRuSuZPMo/bhWqiPMrp3MFljNphBqf+9Gjyp6UZM7xsmkwZnI6SGUYevIKQESPsD2LUKPta84eVdCY+9NChQ0w2TdL+B6++CosXw4QJ/Hun61n+6Q5cyEy68A4HdA7Axx/bjBFjWofD01SZP75ucKvYg0FMpycV3LTX3fB+VmJlWvDkxyEzBqd9ZI6RB5sq5U8nGzbMzs/JyYg7spKB+OPkublw6aV1vc2oBHGdfO5op6iccJj/cBY9WcIZzOWNoa+zfbvqWIcn3TXQmzLzxxiqCXEuD1BDFg/Rn5w3P46toXG4Ll3uZti1a2YMTvvInPZ/Lq/Y7wFFok28Kyu9u7SipBv5+VbLZu3axJ5mVhbcfTcUFkLnzjbTJhiM6RTVyXzJQ+Zc/sv+DPxTe2TCv63D48/5TmfcE82WttiLJmH8jXEs5AhKGMJ+fGyXhcOxBj4Usr13wSsqg+QtHFspmWHky8vhkkuSP4olSidTlHTDGTq/dxkMwskn29fz5sEHH9j5gwfbcaicnNrGIwWUcQt/Zo6cwc3vnuIZtEAg/a//+vrjOsrLrRxEor62blluLs+ZExjPXxhMCefwkLdOMOh9X1lZcOed9vt0T0jV1XD11a0/+yiOzAjX+BsxxONOZoacMCWDiQ/dLFtmBwSfeAKefNJ7MgWb1921K5xwgl0e7XQ0igksIo+/cCM9WMrxvJD8txFPU0kebO526ktfTBSzh9h2fpWVrDa78ofAm/yWt5jAqNjxjGAQ7rgjdpxu2rTYz3ntNViwIGMGXSFTjHxBgb1Db9gQO7+w0DbhzYATpbQR/IZu/HgbRvB3M4rqzJObW1eH3hiMCFO5mLc4kHN5gAp6sbd8bnPmXd68E97yG2NIPPC5qQa7KaUT/J8dH7MvLbVNzF07v3CYsBgu4B5+DG/F/Kzz2Tq80X5vLnQbDnsyz46LLoptrK6yBmmK84DGjYPnn/cGqvLyMuIkKW0Uv1xHKBTrhZaV1dWhj0p6bFO9kbnSl16yiL7M5rXg72jn73I0Y4bdlr+E/8IL6w58QvJmJ8l+V/UNoG7KDSNRoxX/gCh4nxP11m/ial7kOKZxEV3kbS8L3n9zjA9buay82bOhWzdbeZxBg66QKUYe7EXTt6819k7DJkNOktKG8ScSxGfI+DNujLFeabT3668KCrhvwVec+ueeDA9NYfov/akNXMT3PHV55X4jmpvrNTSJROxT8vDh3nrJPPRk1aib6uEnarTir9YF68lH9+9VjuRa/s553M/AYKn3vTjtGte6D+wTkv9GM2SIZ+wLC70nnkwhWZVUKqbNrngVqVs5GAq1qoo1RanDjTd6la7BYN0q7uJie50HArbKc/RokT597Pzo8r/xNwGRuxnq/TZycuw67dp51bS9e4sUFtrKT/+yRNWhifbFT6KK0YaOJdE22rWz6yar2l24UKRPH/nW7CSdWCW/4gP5Yd+D7feQqALev83sbHusySpm6/vcNIR6Kl5Tbtj90xYZ+RtvrFuy7C9VVpTWhjM4gYA1Ws54+5cXFork5Ymcf37stV9cLNKli4QxchJPSohKWchhdn1nuIqL68ol5OTYddxvKZEMQE7Ophu/zTGe8TeLhQvtb9pnnCMLFsopgackm42yhO6xNzz/DdBty3+8iWQfNvVmlCa0DSOfSANkcy5GRUkn4r11v8HzX+/xxrhz59r/19Fe9uO/shtfyFdj/u1tO5Fj5Iyb/4k43jAmcp4ao/eyJZow8ccb/W3fftnHAiJ37HmL9+ThjLP/89wNIicn9pjjDbl68mls5EXsCcnL805iK7oTK0pCknmWyQy0m/beO+b1CrpKO36Wow7+XqqqottO5BgFg7FevAvfJLrRuG0449mchjHB8S4+80YJUSmFzJVIKDv5Pvj3PzvbPqnUt78qUJbG5OfbUXj/AI8OviqtmWQDmQUFNuPGDZq6nsYSzSzr1g0+/bR2MwcV7sfUbv/j/HH78+eeLzHhiEdqB2kpLYXVq23D6u2397pUidhGGsl01t1gqj+Ns7lSD3NzYyQJvmd7+j87kF35imkMwkTCcNFgu+7q1bGDp5dc4gm+1dTYrLvRo5Nn+mSY3HBmGXnIHM1sRYH6r+dBgzzj7O9qFAzaeaNHw/Ll1uC3b895ufNZHHiGCStHkrdyKudNOQruussz9i4102XtBAJe+8xEhs9lwDgDnyhNsTFpk41p5TdqVG3GjABDKeazDR15Jfs4OoR/sJ+7/fZw221e5fuMGTBwYGwxmKt+zzBDXi/JXPxUTFscrlGUtkCyuLE/dOIGa88/34Yqohr1VWRJb8qkHT/LCrra9VzoojGhjGT7EZ+t0tgwTmMyXuIGTEu4WEDkxry5NhTTp489zvhsIBduqm/wOkOgzYRrFKUtkKzgKD/f/l9d7UkgzJoV89YQNTxMP3qwlDOYS0WkFztWfW/NYjgMjz1mw0Aur7w+bzfRU0Z5uVV/nTEj1stPFsbxH0s4DMXFMH26fUrp3t0+SeTmWk+9spKVkS6M5A6ONy9w1cBvYORImyufiOxsuw1HQ8eToWyRkTfG3AKcClQBHwEDRWR9dNkY4CIgDIwUkXlbtquKogD1t78rKEisP+9jZ75hNn3pzSuczyyeNKcTIOz5wOGw1cZpjEH0hz0SxegdyXTt3bG494jY45o82S4PBKxkyYQJ/PzVD/Qvvoj231dxb+/7CExcnNjAGwOnnw4nnhhb1euKodoYW6pC+TxwoIgcBHwAjAEwxnQBzgF+C/weuMsYE9zCz1IUBRIrNjoVxpUrGzTyAIfxBncwkmc4ies63hn7nkTV4vUpQLrl48bVNfDBoDcoPGpU3fe7Yxk61Brz+H2PROw2ly1j5KoreW91e+6r6s/Oz86Ed96pux/BoDXwo0fbp4BEUg1tjWRxnE2dgDOAWdH/xwBjfMvmAfkNbUNj8oqyGfjj2llZXmzaGFvJmqRyNQIykGkCIk9wcmwcO9n2E8XWXYpifEqnMTaluaHiIpeyWFxs10+wr/cFBwiIXNPj6fpTR7OyvP10lbutLOd9c6CFYvKDoFa8eXfgdd+yL6Lz6mCMGQIMAdhzzz2bcHcUpY3gj2vHq1XedJP17ocNqyM5bIBJXMKbHMQF3MdiDmH/rM+8sIbLelm1qn7RMX+Kop/sbKuns3Jl8pTmkhKvg1U0LMPSpXZ7Ua/+v7IfReE7OZJXGbe8sG4oyI/bj0R6N5sjlJYBNGjkjTEvALskWHSNiDwWXecaoAaYlWC9ehGREqAEoFevXvWcvXpoYydNUWKIj9EnUovMyrLLAwE48kj44QdYvpx2bGQ2fenJEs5kDuXhI9kWYgXFgkH7fqhrqJP1cjDGpi+6HPtEol/xN4jKSquhH+3RSlYWlWYr+lc9RDZVPMC5ZEWq6m4n0We7/YxPlWxKKeRWQoNGXkSOq2+5MeaPwCnAsdHHBoD/AXv4VusUndf0tMGTpigxNFQbUlbm5Y4bY5uFv/pq7eK9WMWDnMMJzOPiSAkPzC/DfL4qdjD0tNPgl1+s0qt/+66XQ3wsPhCIHeh02u8zZ3q/Uf9+uX1bvdrLyolE+PPOM1j2ZQ8e51Q6ORMS7YTldOTrePaBgL3RJbIDTdlLtpWwRQOvxpjfA6OB00TkF9+ix4FzjDE5xph9gP2BRYm2scUkOmmK0tZwfWLrk/91g6CPP17HMB7Hi9zINTxEf25fVgBTp3rrRCLw1FPWOMcPnvoHTkMhb37AZ1qS/UZzc2P3Ixy2Xa6i8x6NnMbEL/syits5lSe99YJBr/3hVlt5Rt+PK+Jy+NoDNkkv2VbElsbk7wRygOeN/ZJfF5EiEXnbGPMw8A42jHOJiCRpwLqF1JdOpihKrKe/apWXnugnO5vRA79n8ftrGT3nUHpEjqSAMm95TY2X3hjv/fpDIsXFtV547XrJfqNr18ZIFQC1nv1n7MlAmUYvFvNPrordVxfe8cst+FoA1knXTNSApKHmJ5lEshHZVEybnV3TCgWFFKVJ2NRrP16ULBiMqTL94QeRA/b6WTrytXzO7rGZK07gK1FVqtu2v3rVr0/vJJH9FaeJBNJAqsiSfBbIdnwvH7Jv4iwap4nvP/ZkQmqtVD54U6DNqFAqSltic2VxE+iy+28W774rst3W1ZK3zUrZSI5n4PPyYiUTEkkEuG2HQsmNs5P+dWmThYUx6pdXc6OAyIP0T54qGQh4MsiJ9OL9xryVygdvCmrkFSUTaSoPNT7PPi9P5hTNExAZEpzqGceiotic+2Td14qKkhtn8PRk/Ea3uFjkV7+SZ+kjIDKYYmv0+/SJlQl2WjhZWV6+fKLOT4kkkTP4ab8+I6/aNYrSWmmq8Sj/wCjAokWcsegExvx+KeOfvYi8M3fhois72GVTp3opkzU1NjXS33e1tBSmTav/85591usdW1Vl3zNzJl9taM8fuJcDWckERtlbwvz5cOedXgzd7e/69Z4kciRiY/D1ZRm1JdXJONTIK0prpalktd3NYsOGmNnXh/9CxfHPcMnjJ3DQ//7EIYO6wuWXx+rNT5tmjawrwPKLkgF06mQN8IoV3jync+8Kt5YuJbyhigu4l5/YljIK2JrovoTD1sCPGRN73MOGeYO2DUkit3WSufipmDRcoygpori4rvxBcbF8e9s9shefyB58Jt+wk42fJ5JJiEoZJ5yfLGzji/FfzzUCItMYGLu97OzEnZsStAJsy1BPuGZLBcoURckEXDqjo7AQhgwhd979zOFM1tCRc3iQmv99HZsD7wgEYvPP8/LsPL9X7ycnB3bbDaqqeDVyOH/jOs5jFgOZYZe79yUSWysttXLKbvnAgeq914MaeUVRYgum2rWzKo4AffvSg2VMpoiXOJa/7Hg37L9/3fdHInD00bDPPjZffcKEusVR2dn25lFUBHfcAU8/zbfSgXN5gH3NJ0w++UmM36iL2Li/v8CxvNzqzbubQBuWEG4sauQVRUkuXxzVkrmQUoabu7jluYN5hLPrvl8EnnsOPvzQxuwffdQboDXGyiKUlcHcuXD33bB2LVITZiAzWENHHu77ENtdM9KrYAXvxhCvleOXaFAvvkF04FVRFIsbtCwvhzPOsBIDrjk4cLuMYhndGPjun/ltcC5dwiuTb+v++72QioiVUtjFp3O4aBETZCRPcip3ZP2J7pf3ix1Izs1NXJUan1GkXnyDGEkWM0sBvXr1koqKilTvhqK0XcrLrSGtSqz2+CW70oOl7NCuikXn3M4OHy2FBQtihcYAeveGV16JnWeMDeGIsLj6YI5gASfzFHOy+mNeebnxHrmqztbBGLNERHolWqbhGkVRPFyPWD/BoDXawG58xSOczccbduXCe44h8tpCa7yD0cZvgYCN5990kydP7BCrffN9dTv68xC78hXTuAhTU227StXXdcrflao+MTalDmrkFUXxKCiINc7BINx1F/z+97WzjuI1/sXlPCanMT4y2nrx/vh7+/bWAL/yih1kLSysHYQVYAglrGJPHuBcOvCdfd8LL1gRsXhD78TFxo5NvFxpEDXyiqLE4jJcnIEfMsQaf+etAyO4k/OZxViu59nASYnle/Pz7SDr3Lm2QxQwhcE8TH9u4K8cTrn3ea76NV4qvKzMVseGw/avSolvMmrkFUXx8OegQ2wl6eDBtbMNUMJgDuJNzguX8slBp9vl8U17XKile3dWBrtxGf+mD/MYzc1e9kx9+u65ud5TgpMvUDYJza5RFMUSn4OelRVrdAcMsNo1UT33rdnAHM60rQMXX82CFb9ja3+2i0/H/edQe/q1W077n9ZTygACBjjuOBuLh+QDqWvX2ptBJBIrX6A0GvXkFUWxNJSDnp9vuzGFQrUhnX35hPs5jxUcTFHVv5H5ZbHbiwqfXVp5C+//tBv3ZV/EzsG1Nh9+3DgvbbO+rlY5OdbTz8nRpkCbgXryiqJYGpOD7u/GtH493HYbJ4af5e9cy1huIG/tJ4wA68WvWgVZWcyKnMsMGchf//gFxw4ZC2VH2bCLi6/XlyXTVCJsbRjNk1cUxWNTc9DLy6G0lIgYCt++gWde78D8iW9x5OV5UFXFB4ED6EkF3btU8VLF9jZxp6QERoywTw05OdaIgxryLaC+PHk18oqiJGcTjP7338Mhh8CPq39i6c8HsGPkW/IpZ1W7A1jxQTs6dYpu76ijvLBQIGCfDmbO9J4g4gdvlQapz8hruEZRlMTEN8BuwPjusAPMmQOH5bXjbB7mQLOS5dKdx697j06dDrArlZbGVse6dE3XtCRRo3Bli9CBV0VREuPvGJUohz0BBx4I0+8JsiByOMUylD/1/5JT/3xA8jeceqqN/deXRqlsEerJK4qSmM1sL9ivH6xeDUuWwE1TdotdOGCATdOsrrZG3YmWJRpcVY2aJkFj8oqiJKc5DG10sJYZM2zOfaJQ0CaGito6KlCmKMrm0RxiYPn5sOee1sAnCwVtRqhISYwaeUVRWh5/J6pEoaCGliuNRmPyiqK0PA0VOWkRVJOhMXlFUZRWjsbkFUVR2ihq5BVFUTIYNfKKoigZjBp5RVGUDEaNvKIoSgajRl5RFCWDSasUSmPMGuCzVO/HZrAT8G2qd6KF0WPOfNra8ULrPea9RKRjogVpZeRbK8aYimQ5qpmKHnPm09aOFzLzmDVcoyiKksGokVcURclg1Mg3DSWp3oEUoMec+bS144UMPGaNySuKomQw6skriqJkMGrkFUVRMhg18k2MMeYKY4wYY3ZK9b40J8aYW4wx7xlj3jTGzDXGtE/1PjUXxpjfG2PeN8Z8aIy5OtX709wYY/Ywxsw3xrxjjHnbGHNZqveppTDGBI0xy4wxT6Z6X5oKNfJNiDFmD6APsCrV+9ICPA8cKCIHAR8AY1K8P82CMSYITAJOBLoA5xpjuqR2r5qdGuAKEekCHAZc0gaO2XEZ8G6qd6IpUSPftNwOjAYyfjRbRJ4TkZroy9eBTqncn2YkD/hQRD4WkSrgQeD0FO9TsyIiX4nI0uj/P2KN3u6p3avmxxjTCTgZmJrqfWlK1Mg3EcaY04H/iciKVO9LChgEPJPqnWgmdgc+973+gjZg8BzGmL2B7sAbKd6VlmAC1kmLpHg/mhTt8boJGGNeAHZJsOga4C/YUE3GUN/xishj0XWuwT7ez2rJfVOaH2PMtsBsYJSI/JDq/WlOjDGnAN+IyBJjTEGKd6dJUSO/CYjIcYnmG2O6AvsAK4wxYEMXS40xeSKyugV3sUlJdrwOY8wfgVOAYyVzCy7+B+zhe90pOi+jMcaEsAZ+lojMSfX+tABHAKcZY04CtgK2N8bcJyIXpHi/thgthmoGjDGfAr1EpDWq2TUKY8zvgX8BR4vImlTvT3NhjMnCDiwfizXui4HzROTtlO5YM2KspzITWCcio1K8Oy1O1JO/UkROSfGuNAkak1c2lzuB7YDnjTHLjTGTU71DzUF0cHkEMA87APlwJhv4KEcAfwB+Fz23y6MertIKUU9eURQlg1FPXlEUJYNRI68oipLBqJFXFEXJYNTIK4qiZDBq5BVFUTIYNfKKoigZjBp5RVGUDOb/AQ3CiceIrCt/AAAAAElFTkSuQmCC\n"
          },
          "metadata": {
            "needs_background": "light"
          }
        }
      ],
      "source": [
        "inputs = x.reshape(x.shape[-1], 1)\n",
        "\n",
        "params = {\n",
        "    \"W1\": torch.randn(1, 4).requires_grad_(True),\n",
        "    \"b1\": torch.randn(4).requires_grad_(True),\n",
        "    \"W2\": torch.randn(4, 4).requires_grad_(True),\n",
        "    \"b2\": torch.randn(4).requires_grad_(True),\n",
        "    \"W3\": torch.randn(4, 1).requires_grad_(True),\n",
        "    \"b3\": torch.randn(1).requires_grad_(True),\n",
        "}\n",
        "\n",
        "\n",
        "# We can define an optimizer which takes care of updating parameters based on their gradient. We can use more complex optimizers like SGD+Momntum or Adam.\n",
        "optimizer = torch.optim.SGD(params.values(), lr=0.0001, weight_decay=0.0001, momentum=0.9)\n",
        "\n",
        "# Pytorch also has implementation of wide range of activation functions such as: Tanh, ReLU, LeakyReLU, ...\n",
        "nonlinearity = torch.nn.ReLU()\n",
        "\n",
        "\n",
        "def predict(params, inputs):\n",
        "    h1 = nonlinearity(torch.mm(inputs, params[\"W1\"]) + params[\"b1\"])\n",
        "    h2 = nonlinearity(torch.mm(h1, params[\"W2\"]) + params[\"b2\"])\n",
        "    output = torch.mm(h2, params[\"W3\"]) + params[\"b3\"]\n",
        "    return output\n",
        "\n",
        "\n",
        "def cost(params):\n",
        "    output = predict(params, inputs)\n",
        "    return (1.0 / inputs.shape[0]) * torch.sum(0.5 * (output.reshape(output.shape[0]) - t) ** 2)\n",
        "\n",
        "\n",
        "print(cost(params))\n",
        "\n",
        "num_epochs = 10000\n",
        "\n",
        "for i in range(num_epochs):\n",
        "    # Evaluate the gradient of the current parameters stored in params\n",
        "    loss = cost(params)\n",
        "    if i % 500 == 0:\n",
        "        print(f\"i: {i:<5d} loss: {loss.item():.4f}\")\n",
        "    optimizer.zero_grad()\n",
        "    loss.backward()\n",
        "    optimizer.step()\n",
        "\n",
        "with torch.no_grad():\n",
        "    final_y = predict(params, inputs)\n",
        "    plt.plot(x, t, \"r.\")\n",
        "    plt.plot(x, final_y, \"b-\")"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "mWU6AagIE3l6"
      },
      "source": [
        "## Part 3.2 Neural network models"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "lkdDq61Ea38-"
      },
      "source": [
        "### MNIST classification"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "hiJiOPA5a-lu"
      },
      "source": [
        "[MNIST](http://yann.lecun.com/exdb/mnist/) is a famous dataset containing hand-written digits. The training set contains 60k and the test set contains 10k images. PyTorch has built-in functions for downloading well-known datasets like MNIST."
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 32,
      "metadata": {
        "id": "eeeIOIMha5da",
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 849,
          "referenced_widgets": [
            "4e110a04fa804028ac51cb93e91b3918",
            "d619b748aea0474084b1bf6bf02106d9",
            "60609425f5694effaeaccce1e8fbfcef",
            "47432ea2dc86471d93cf86efcce67706",
            "85836827ca4d47498a72d309772bf7ac",
            "328c6754a97b42318a08e64b5d96c8f3",
            "7a23052817d5472f9dc8a156b65285d0",
            "94ad419e14e84460b6f87ad87dd554db",
            "d87c3e20cffb45499b60578d0c61472b",
            "e28753ae72d24f228eaed71d5b6cc914",
            "f3d4797e9fbf42669609893ba72f8559",
            "21a31e5b9ea5408f88dfb2c7c3f48669",
            "a1f03502f4e34eb8852b1332037e487f",
            "7bbc448f2c3847e1bbada58144f5e14b",
            "23a1a90003a84e39977a4401cd01f9ee",
            "8cfeddd313f041ff92ddadc5c53114c9",
            "8b5c927c5288473e93419093f9ba120d",
            "bd112bcf89cb427ca7dd5d3acd1bb62a",
            "0276c153bc524bc0ae25027610cf9b86",
            "a19525b4396b4e5f9dbf1f4f7798fa43",
            "4c89c4f392d44c4ba3baaca4a1c19e66",
            "7eb7cc40b022495a8788956f228d5948",
            "ac9ed86d3d524b78bfbeb00c6aca9537",
            "0d7b21754f7d4cf89489f9c4be22a9d8",
            "adf998d250ff47dfa581f6a82cefea92",
            "20cefde820e84fb1bf60dc74d5bc6800",
            "5bbd95963f2d4096bc1dde86ab62ad4a",
            "3cf8e724375b4eb28a32006a77a525e8",
            "aa529581902c4b239200c4ab29d694b3",
            "7779a750a21147f0b2f1469f84c7ee10",
            "857bbe9630664a7ea71603fde979a0ef",
            "99e19f71d71a4b30be41fab4fff940f1",
            "d2fff2577d4c4c0fa1cd984b7b1f9fc5",
            "df6122ae6e9b4416a111b306e13fc2db",
            "4b5d8d8fef684333950ce3a83886737f",
            "5cf9b8ecd2524695b538184cabe9df16",
            "8a90fa4a020c4f19b2a4e54ffa3ecc2d",
            "6df204f32846409c8c66e9fb046670ff",
            "2c1750dccfb64a5a99320b655c620ee6",
            "fd255bd71f6448f59ed5635f2bafd923",
            "4bdd03c2d6cc486082e65d62654e77e6",
            "c8ab123f08864784b9dc2a065590fa08",
            "42ea377247934693bdc86b60e7b31a04",
            "4dc6d88a399e487c91484d27029c3e2e",
            "0682caa3f84a438b9a3dfb495be6b696",
            "e6fe40ad6ad545c685aa6fdb23b828a0",
            "a80b1b2beee64d459a93af857e25af95",
            "ad8218da526a4837b4ebe389bf30fa37",
            "17bcdf2629074184b950479d3a5070ca",
            "f3ca3b9ee7f04164bee010121ebee005",
            "edd56c491a6d40b787a7a757f5e328d0",
            "1d64edc1df8a4402ad2a84952a2c2c21",
            "f11b420f7ece46eeb5ede36ba9e09210",
            "32ab8103f798447f8d24636057e1c4fe",
            "8cf97cdf227648ca91d8e17c6f3861fc",
            "8f21f583c2b945e49be68d7f6c5f677e",
            "d16eb7c10c8145f7854a2a9f9dc4d8cf",
            "d6899de509134307b7ac12b45e5b72d6",
            "c284fca7832240368f13f4effb2a71e9",
            "35876458314a4510a0a9e134f9d1e95c",
            "d666260fd53742918de352714ef19ff4",
            "7fd7006eae5544919ec02cce074d265e",
            "0abb37de8fb44e7ea4eff476ddb7540f",
            "c76055834cc743ecbc41121aed1c8885",
            "0508ffaa73b64caea80b9e53b7f61454",
            "e5ff2ddbd95844ae9aa97063659184ed",
            "3bcd756036a54100bf14fd785667c092",
            "47b5d1a112f64bf49de801fe48dcde2c",
            "3b745cd3f2b041c4a3df8284ddb44ec2",
            "a41f3fa24c474a8db5173182002b8b3e",
            "0679b226b013462083c4a137f05b8a3f",
            "221be29a10384bec91cfc7786c44e0a9",
            "67b3f4d9637240df882850326a34baa4",
            "961f36cc634144a3b921640552cb5323",
            "00a97b29029a4c2795fd9d673dbe7215",
            "b004c8409add445c9042c581644d61c0",
            "d25a9981b9724054ac196455e563bea0",
            "b0666d0566af4c8ea716b835e3f10bd1",
            "6399e318eaad4903a4b798489aeecbb1",
            "9d5710c9d09b42eaada8450872ec5dab",
            "638dca8a30db4b61ae8bb190ef433acc",
            "841d7f6acb604fb39489ecabefd2a198",
            "ac333eb723034faa93b92d165a5c6db1",
            "7bee58598d9640df9415deebf1f5dc55",
            "c1ed3eeee590463098cf4da80a5a2cdd",
            "a8c3e0744e3246d78d047afacde5a9d5",
            "54467dd23b2c48c5b323b1fb920c010e",
            "7b15de41f3104996bc7e83682c141758"
          ]
        },
        "outputId": "6c0500e7-f201-451b-8c65-d314f380c511"
      },
      "outputs": [
        {
          "output_type": "stream",
          "name": "stdout",
          "text": [
            "Downloading http://yann.lecun.com/exdb/mnist/train-images-idx3-ubyte.gz\n",
            "Downloading http://yann.lecun.com/exdb/mnist/train-images-idx3-ubyte.gz to data/MNIST/raw/train-images-idx3-ubyte.gz\n"
          ]
        },
        {
          "output_type": "display_data",
          "data": {
            "text/plain": [
              "  0%|          | 0/9912422 [00:00<?, ?it/s]"
            ],
            "application/vnd.jupyter.widget-view+json": {
              "version_major": 2,
              "version_minor": 0,
              "model_id": "4e110a04fa804028ac51cb93e91b3918"
            }
          },
          "metadata": {}
        },
        {
          "output_type": "stream",
          "name": "stdout",
          "text": [
            "Extracting data/MNIST/raw/train-images-idx3-ubyte.gz to data/MNIST/raw\n",
            "\n",
            "Downloading http://yann.lecun.com/exdb/mnist/train-labels-idx1-ubyte.gz\n",
            "Downloading http://yann.lecun.com/exdb/mnist/train-labels-idx1-ubyte.gz to data/MNIST/raw/train-labels-idx1-ubyte.gz\n"
          ]
        },
        {
          "output_type": "display_data",
          "data": {
            "text/plain": [
              "  0%|          | 0/28881 [00:00<?, ?it/s]"
            ],
            "application/vnd.jupyter.widget-view+json": {
              "version_major": 2,
              "version_minor": 0,
              "model_id": "21a31e5b9ea5408f88dfb2c7c3f48669"
            }
          },
          "metadata": {}
        },
        {
          "output_type": "stream",
          "name": "stdout",
          "text": [
            "Extracting data/MNIST/raw/train-labels-idx1-ubyte.gz to data/MNIST/raw\n",
            "\n",
            "Downloading http://yann.lecun.com/exdb/mnist/t10k-images-idx3-ubyte.gz\n",
            "Downloading http://yann.lecun.com/exdb/mnist/t10k-images-idx3-ubyte.gz to data/MNIST/raw/t10k-images-idx3-ubyte.gz\n"
          ]
        },
        {
          "output_type": "display_data",
          "data": {
            "text/plain": [
              "  0%|          | 0/1648877 [00:00<?, ?it/s]"
            ],
            "application/vnd.jupyter.widget-view+json": {
              "version_major": 2,
              "version_minor": 0,
              "model_id": "ac9ed86d3d524b78bfbeb00c6aca9537"
            }
          },
          "metadata": {}
        },
        {
          "output_type": "stream",
          "name": "stdout",
          "text": [
            "Extracting data/MNIST/raw/t10k-images-idx3-ubyte.gz to data/MNIST/raw\n",
            "\n",
            "Downloading http://yann.lecun.com/exdb/mnist/t10k-labels-idx1-ubyte.gz\n",
            "Downloading http://yann.lecun.com/exdb/mnist/t10k-labels-idx1-ubyte.gz to data/MNIST/raw/t10k-labels-idx1-ubyte.gz\n"
          ]
        },
        {
          "output_type": "display_data",
          "data": {
            "text/plain": [
              "  0%|          | 0/4542 [00:00<?, ?it/s]"
            ],
            "application/vnd.jupyter.widget-view+json": {
              "version_major": 2,
              "version_minor": 0,
              "model_id": "df6122ae6e9b4416a111b306e13fc2db"
            }
          },
          "metadata": {}
        },
        {
          "output_type": "stream",
          "name": "stdout",
          "text": [
            "Extracting data/MNIST/raw/t10k-labels-idx1-ubyte.gz to data/MNIST/raw\n",
            "\n",
            "Downloading http://yann.lecun.com/exdb/mnist/train-images-idx3-ubyte.gz\n",
            "Downloading http://yann.lecun.com/exdb/mnist/train-images-idx3-ubyte.gz to ../data/MNIST/raw/train-images-idx3-ubyte.gz\n"
          ]
        },
        {
          "output_type": "display_data",
          "data": {
            "text/plain": [
              "  0%|          | 0/9912422 [00:00<?, ?it/s]"
            ],
            "application/vnd.jupyter.widget-view+json": {
              "version_major": 2,
              "version_minor": 0,
              "model_id": "0682caa3f84a438b9a3dfb495be6b696"
            }
          },
          "metadata": {}
        },
        {
          "output_type": "stream",
          "name": "stdout",
          "text": [
            "Extracting ../data/MNIST/raw/train-images-idx3-ubyte.gz to ../data/MNIST/raw\n",
            "\n",
            "Downloading http://yann.lecun.com/exdb/mnist/train-labels-idx1-ubyte.gz\n",
            "Downloading http://yann.lecun.com/exdb/mnist/train-labels-idx1-ubyte.gz to ../data/MNIST/raw/train-labels-idx1-ubyte.gz\n"
          ]
        },
        {
          "output_type": "display_data",
          "data": {
            "text/plain": [
              "  0%|          | 0/28881 [00:00<?, ?it/s]"
            ],
            "application/vnd.jupyter.widget-view+json": {
              "version_major": 2,
              "version_minor": 0,
              "model_id": "8f21f583c2b945e49be68d7f6c5f677e"
            }
          },
          "metadata": {}
        },
        {
          "output_type": "stream",
          "name": "stdout",
          "text": [
            "Extracting ../data/MNIST/raw/train-labels-idx1-ubyte.gz to ../data/MNIST/raw\n",
            "\n",
            "Downloading http://yann.lecun.com/exdb/mnist/t10k-images-idx3-ubyte.gz\n",
            "Downloading http://yann.lecun.com/exdb/mnist/t10k-images-idx3-ubyte.gz to ../data/MNIST/raw/t10k-images-idx3-ubyte.gz\n"
          ]
        },
        {
          "output_type": "display_data",
          "data": {
            "text/plain": [
              "  0%|          | 0/1648877 [00:00<?, ?it/s]"
            ],
            "application/vnd.jupyter.widget-view+json": {
              "version_major": 2,
              "version_minor": 0,
              "model_id": "3bcd756036a54100bf14fd785667c092"
            }
          },
          "metadata": {}
        },
        {
          "output_type": "stream",
          "name": "stdout",
          "text": [
            "Extracting ../data/MNIST/raw/t10k-images-idx3-ubyte.gz to ../data/MNIST/raw\n",
            "\n",
            "Downloading http://yann.lecun.com/exdb/mnist/t10k-labels-idx1-ubyte.gz\n",
            "Downloading http://yann.lecun.com/exdb/mnist/t10k-labels-idx1-ubyte.gz to ../data/MNIST/raw/t10k-labels-idx1-ubyte.gz\n"
          ]
        },
        {
          "output_type": "display_data",
          "data": {
            "text/plain": [
              "  0%|          | 0/4542 [00:00<?, ?it/s]"
            ],
            "application/vnd.jupyter.widget-view+json": {
              "version_major": 2,
              "version_minor": 0,
              "model_id": "b0666d0566af4c8ea716b835e3f10bd1"
            }
          },
          "metadata": {}
        },
        {
          "output_type": "stream",
          "name": "stdout",
          "text": [
            "Extracting ../data/MNIST/raw/t10k-labels-idx1-ubyte.gz to ../data/MNIST/raw\n",
            "\n"
          ]
        }
      ],
      "source": [
        "from torchvision import datasets, transforms\n",
        "\n",
        "mnist_train = datasets.MNIST(\"data\", train=True, download=True, transform=transforms.ToTensor())\n",
        "\n",
        "mnist_test = datasets.MNIST(\"../data\", train=False, download=True, transform=transforms.ToTensor())"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 33,
      "metadata": {
        "id": "nTdP7vjzb35H",
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "outputId": "7717965e-d758-42ed-e7ba-153c7be8a29c"
      },
      "outputs": [
        {
          "output_type": "stream",
          "name": "stdout",
          "text": [
            "Dataset MNIST\n",
            "    Number of datapoints: 60000\n",
            "    Root location: data\n",
            "    Split: Train\n",
            "    StandardTransform\n",
            "Transform: ToTensor()\n",
            "Dataset MNIST\n",
            "    Number of datapoints: 10000\n",
            "    Root location: ../data\n",
            "    Split: Test\n",
            "    StandardTransform\n",
            "Transform: ToTensor()\n"
          ]
        }
      ],
      "source": [
        "print(mnist_train)\n",
        "print(mnist_test)"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 34,
      "metadata": {
        "id": "1dWTw-Rgbh5O",
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 282
        },
        "outputId": "af233fc8-5eba-48b1-83d5-b96b288e3664"
      },
      "outputs": [
        {
          "output_type": "stream",
          "name": "stdout",
          "text": [
            "Label:  3\n"
          ]
        },
        {
          "output_type": "display_data",
          "data": {
            "text/plain": [
              "<Figure size 432x288 with 1 Axes>"
            ],
            "image/png": "iVBORw0KGgoAAAANSUhEUgAAAPsAAAD4CAYAAAAq5pAIAAAAOXRFWHRTb2Z0d2FyZQBNYXRwbG90bGliIHZlcnNpb24zLjUuMywgaHR0cHM6Ly9tYXRwbG90bGliLm9yZy/NK7nSAAAACXBIWXMAAAsTAAALEwEAmpwYAAAQ2UlEQVR4nO3df4wUdZrH8ffDrETC6Ihy4sgQWI3RKPG8w3AmroLRM4gKLlmNP0Jc8DIbs4jH0VHjOUFybgRsd9F4e+6uksWL52bNStCNeufpZohGL4Ai8sM90AOZCTccEUWMiOBzf0zRmYbpbzfd1d0l388r6cy36umqeujhM9Vd1d1l7o6IHP+GNLsBEWkMhV0kEgq7SCQUdpFIKOwisXD3ht0AH3hrb2/3I+dl5ZbV3rLal3rLTm+l8lfTnt3MppjZn81sq5ndd6zLz58/v5bN11VWe8tqX6DeqtWo3qoOu5m1AP8MXAOcD9xiZuen1ZiIpKuWPftEYKu7f+zuB4DfAdPTaUtE0mbVvoPOzH4ETHH3v0umZwJ/4+5zjrhfJ9AJ0NbWNqGrq6tQ6+jooKenp8rW6yurvWW1L1Bv1Uqzt1wuh7vboMUaDrb9CHhqwPRM4IljOUCXz+ebfnCk1C2rvWW1L/WWnd7qcYCuFxgzYLojmSciGVRL2FcD55jZ981sKHAz8GI6bYlI2r5X7YLuftDM5gD/DrQAy9x9Y2qdiUiqqg47gLu/DLycUi8iUkd6u6xIJBR2kUgo7CKRUNhFIqGwi0RCYReJhMIuEgmFXSQSCrtIJBR2kUgo7CKRUNhFIqGwi0RCYReJhMIuEgmFXSQSCrtIJBR2kUgo7CKRUNhFIqGwi0Sipm+XlcoMHTo0WL/11luD9csvv7wwPvnkk1m2bFlRffz48SWXvfjii4PrNhv8SkGHrVy5Mlh///33C+MzzzyThQsXFtWfeuqpksvu2LEjuG5Jl/bsIpFQ2EUiobCLREJhF4mEwi4SCYVdJBIKu0gkdJ69AebNmxesP/zwwxWvq7u7mxkzZtTaUoG7B+vTpk2ruN7d3c3NN99cVA+9h+Caa64Jrnvr1q3BuhybmsJuZtuAL4BDwEF3D7+DQ0SaJo09+xXuvjuF9YhIHek1u0gkrNxrtuDCZv8D7AEc+JW7/3qQ+3QCnQBtbW0Turq6CrWOjg56enqq3n49pdnbGWecEayPHj264nXt27eP1tbWWluqi8F6+/rrr0vef8uWLcH1hZY9VrH8X8vlcrj7oB94qDXso92918xOB14D7nL3VYH7F20sn8+Ty+Wq3n49pdnbvffeG6wf6wG6SZMm1dpSXQzW20cffVTy/o08QBfL/zWgZNhrehrv7r3Jz13ACmBiLesTkfqpOuxmNtzMTjo8Bq4GNqTVmIikq5aj8aOAFcnnob8H/Ju7v5pKV8eZWbNm1bT8tddeWxjPmDGDJUuWFNUPHDhQ0/pDTjjhhGD9ySefLOrjk08+KaqfffbZJZd9/PHHg+ueOnVqBR1KpaoOu7t/DPxlir2ISB3p1JtIJBR2kUgo7CKRUNhFIqGwi0RCH3FtgNBXPVfi0KFDhfGVV17Jq68Wn+Gs5V2Qtbr77rsL4+uvv55f/vKXRfUVK1aUXPayyy6rW19yNO3ZRSKhsItEQmEXiYTCLhIJhV0kEgq7SCQUdpFI6Dx7Axw8eDDV9TXzvPqRTjzxxMLYzIqmy9m/f389WpIStGcXiYTCLhIJhV0kEgq7SCQUdpFIKOwikVDYRSKh8+wSVO6z+EuXLi2MN2zYUDRdzuLFi6vsSqqhPbtIJBR2kUgo7CKRUNhFIqGwi0RCYReJhMIuEgmdZ4/c8OHDg/WHHnooWB81alRh/OGHHxZNA/T29pZc9plnnqmgQ0lL2T27mS0zs11mtmHAvFPN7DUz25L8HFHfNkWkVpU8jf8tMOWIefcBr7v7OcDrybSIZFjZsLv7KuDTI2ZPB5Yn4+XADem2JSJps0q+z8zMxgF/dPfxyfRn7n5KMjZgz+HpQZbtBDoB2traJnR1dRVqHR0d9PT01PYvqJOs9pZ2X0OGhP/en3XWWcF6W1tbYbxv3z5aW1uL6t98803JZTdt2hRcd5rf3ZfV3yek21sul8PdbbBazWFPpve4e9nX7WZWtLF8Pk8ulyu7/WbIam9p91XuAN2zzz4brE+bNq0w7u7uZtKkSUX10AG6CRMmBNe9a9euYP1YZPX3Cen3Virs1Z566zOzdoDkZ3q/FRGpi2rD/iJwezK+HViZTjsiUi9lz7Ob2XPAZGCkmfUAC4BFwO/N7A5gO3BTPZuUsBEjSr+Cuvrqq4PL3nbbbcH6ddddF6wfOHCgMHb3ommA+fPnl1w2zafpUl7ZsLv7LSVKV6bci4jUkd4uKxIJhV0kEgq7SCQUdpFIKOwikdBHXI8Dq1atKlm74IILalr3xo0bg/VZs2YVxrNnz+aee+4pqq9Zs6am7Ut6tGcXiYTCLhIJhV0kEgq7SCQUdpFIKOwikVDYRSKh8+zHgVdeeaVkrdbz7GPHjg3WJ0+eXBifdNJJRdMA69atK7lsml87JeVpzy4SCYVdJBIKu0gkFHaRSCjsIpFQ2EUiobCLRELn2Y8D999/f8laX19fcNkbb7wxWJ84cWKwvmTJksK4u7v7qK+mbm9vL7nsAw88EFz3V199FazLsdGeXSQSCrtIJBR2kUgo7CKRUNhFIqGwi0RCYReJhM6zHwdCnwt/9NFHg8s+/fTTwXo+nw/WZ8+eHazPmzevZG3KlCnBZWv9LL4UK7tnN7NlZrbLzDYMmPegmfWa2brkNrW+bYpIrSp5Gv9bYLA/wb9w94uS28vptiUiaSsbdndfBXzagF5EpI7M3cvfyWwc8Ed3H59MPwj8GNgLrAHmu/ueEst2Ap0AbW1tE7q6ugq1jo4Oenp6avoH1EtWe0u7r5aWlrLbCxk5cmRhvG/fPlpbWyve9v79+4P1cteZOxZZ/X1Cur3lcjnc3QarVRv2UcBuwIF/AtrdPXykpn+5oo3l83lyuVzZ7TdDVntLu69TTjml7PZCBh6g6+7uZtKkSRVve/PmzcF6mgfosvr7hPR7KxX2qk69uXufux9y92+B3wDhj0aJSNNVFXYzG/i5xR8CG0rdV0Syoex5djN7DpgMjDSzHmABMNnMLqL/afw24Cf1a1Hq6bPPPgvW58yZE6zv2LGjMD733HNZuHBhUX3BggUllz3vvPOC6166dGmwXu6pr76XvljZsLv7LYPMDr8TQ0QyR2+XFYmEwi4SCYVdJBIKu0gkFHaRSOgjrhJU7i2tjz32WGG8YMGCommAq666quSyl156aXDdc+fODdYfeeSRYL23tzdYj4327CKRUNhFIqGwi0RCYReJhMIuEgmFXSQSCrtIJHSeXWoy8COyhw4dOuojs3feeWfJZdevX1/TtmfOnBmsL1q0qKb1H2+0ZxeJhMIuEgmFXSQSCrtIJBR2kUgo7CKRUNhFIqHz7FJXfX19JWvlLu9U7oowN910U7Cu8+zFtGcXiYTCLhIJhV0kEgq7SCQUdpFIKOwikVDYRSKh8+wpGDIk/DezpaUlWHf3YP27fOnh3bt3l6y98cYbwWXLnWcfeLloKa/snt3MxpjZn8xsk5ltNLO7k/mnmtlrZrYl+Tmi/u2KSLUqeRp/EJjv7ucDlwA/NbPzgfuA1939HOD1ZFpEMqps2N19p7u/m4y/ADYDo4HpwPLkbsuBG+rUo4ikwMq9Xiy6s9k4YBUwHvjE3U9J5huw5/D0Ect0Ap0AbW1tE7q6ugq1jo4Oenp6qu++jtLsrf/hqd7A39Hx9JiNGTMmWD/99NOD9c8//zxY37p1a9W9NVKaveVyOdx90P9wFYfdzFqBbuBn7v6CmX02MNxmtsfdg6/bzaxoY/l8nlwuV9H2G+1YemvkAbrv2mMW+kO3dOnS4PruuuuuYP2ll14K1qdPnx7sLSvS7q1U2Cs69WZmJwB/AJ519xeS2X1m1p7U24FdaTQqIvVR9tRb8hT9aWCzu/98QOlF4HZgUfJzZV06/A645JJLgvU333wzWN++fXuwfsUVVxTGQ4cOZdy4cUX1bdu2BZevpwsvvLAwHjZsWNE0wMSJE0suW27PXc7ixYtrWj42lZxnvxSYCXxgZuuSeffTH/Lfm9kdwHYg/OFiEWmqsmF39zeBUi+8rky3HRGpF71dViQSCrtIJBR2kUgo7CKRUNhFIqGPuGbA2LFjg/W1a9cWxu+9917RNMCXX35Zl74qcdpppxXG77zzDm+//XZRfdiwYSWX/fbbb4Prfuutt4L1Wi/5HBvt2UUiobCLREJhF4mEwi4SCYVdJBIKu0gkFHaRSOg8ewo2bNgQrM+dOzdYnzFjRrA+efLkwrilpYURI4q/EOjI6WYZMmTIUefVV69eXfL+zz//fHB9+Xw+lb6kn/bsIpFQ2EUiobCLREJhF4mEwi4SCYVdJBIKu0gkdJ49BXv37g3Wn3jiiZrqA+Xz+aLvkc+SLPcm2rOLRENhF4mEwi4SCYVdJBIKu0gkFHaRSCjsIpEoG3YzG2NmfzKzTWa20czuTuY/aGa9ZrYuuU2tf7siUq1K3lRzEJjv7u+a2UnAWjN7Lan9wt31DQMi3wGVXJ99J7AzGX9hZpuB0fVuTETSZe5e+Z3NxgGrgPHAPwA/BvYCa+jf++8ZZJlOoBOgra1tQldXV6HW0dFBT09P9d3XUVZ7y2pfoN6qlWZvuVwOd7dBi+5e0Q1oBdYCM5LpUUAL/a/7fwYsq2AdPvCWz+f9yHlZuWW1t6z2pd6y01up/FV0NN7MTgD+ADzr7i/Qv8Y+dz/k7t8CvwEmVrIuEWmOSo7GG/A0sNndfz5gfvuAu/0QCH/Fqog0VSVH4y8FZgIfmNm6ZN79wC1mdhH9Tx22AT+pQ38ikpJKjsa/CQz2gv/l9NsRkXrRO+hEIqGwi0RCYReJhMIuEgmFXSQSCrtIJBR2kUgo7CKRUNhFIqGwi0RCYReJhMIuEgmFXSQSCrtIJI7pO+hq3pjZ/wHbB8waCexuWAPHJqu9ZbUvUG/VSrO3se7+F4MVGhr2ozZutsbdL25aAwFZ7S2rfYF6q1ajetPTeJFIKOwikWh22H/d5O2HZLW3rPYF6q1aDemtqa/ZRaRxmr1nF5EGUdhFItGUsJvZFDP7s5ltNbP7mtFDKWa2zcw+SC5DvabJvSwzs11mtmHAvFPN7DUz25L8HJGh3jJxGe/AZcab+tg1+/LnDX/NbmYtwH8Dfwv0AKuBW9x9U0MbKcHMtgEXu3vT34BhZpcD+4Bn3H18Mm8J8Km7L0r+UI5w93sz0tuDwL5mX8Y7uVpR+8DLjAM30H8h0qY9doG+bqIBj1sz9uwTga3u/rG7HwB+B0xvQh+Z5+6rgE+PmD0dWJ6Ml9P/n6XhSvSWCe6+093fTcZfAIcvM97Uxy7QV0M0I+yjgR0DpnvI1vXeHfgPM1ubXG46a0a5+85k/L/0X003S+aY2frkaX5TXmIMlFxm/K+A/yJDj90RfUEDHjcdoDvaD9z9r4FrgJ8mT1czyftfg2Xp3Om/AGcDFwE7gUeb2YyZtdJ/9eG/d/e9A2vNfOwG6ashj1szwt4LjBkw3ZHMywR3701+7gJWkL1LUfcdvoJu8nNXk/spyNJlvAe7zDgZeOyaefnzZoR9NXCOmX3fzIYCNwMvNqGPo5jZ8OTACWY2HLia7F2K+kXg9mR8O7Cyib0UycplvEtdZpwmP3ZNv/y5uzf8Bkyl/4j8R8A/NqOHEn2dBbyf3DY2uzfgOfqf1n1D/7GNO4DTgNeBLcB/AqdmqLd/BT4A1tMfrPYm9fYD+p+irwfWJbepzX7sAn015HHT22VFIqEDdCKRUNhFIqGwi0RCYReJhMIuEgmFXSQSCrtIJP4fsvg3AYftORAAAAAASUVORK5CYII=\n"
          },
          "metadata": {
            "needs_background": "light"
          }
        }
      ],
      "source": [
        "i = npr.randint(1, 50000)\n",
        "example = mnist_train[i]\n",
        "print(\"Label: \", example[1])\n",
        "plt.imshow(example[0].reshape((28, 28)), cmap=plt.cm.gray)\n",
        "plt.grid(None)"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "F5cYaF7PcB7v"
      },
      "source": [
        "Pytorch's DataLoader is responsible for creating an iterator over the dataset.  "
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 35,
      "metadata": {
        "id": "4WDyu63wcV3Z",
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 248
        },
        "outputId": "164c95d9-9d19-46ba-a33a-d970012f6cb4"
      },
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "<matplotlib.image.AxesImage at 0x7f4dabf3a670>"
            ]
          },
          "metadata": {},
          "execution_count": 35
        },
        {
          "output_type": "display_data",
          "data": {
            "text/plain": [
              "<Figure size 576x576 with 1 Axes>"
            ],
            "image/png": "iVBORw0KGgoAAAANSUhEUgAAAe0AAADWCAYAAAD8ftohAAAAOXRFWHRTb2Z0d2FyZQBNYXRwbG90bGliIHZlcnNpb24zLjUuMywgaHR0cHM6Ly9tYXRwbG90bGliLm9yZy/NK7nSAAAACXBIWXMAAAsTAAALEwEAmpwYAACcBElEQVR4nOy9d3hc5Zn3/znTNF0z6r33asuyJUvGHWNTDIQQICQhIWTTeXd5N5vs5r3IbhK29/w2lSWEJAQwHWzce5WsZvVi9d67NJpyfn+Ic2KBDW6aGZP5XBcXMBrN3DM657mf5y7fWxBFER8+fPjw4cOH96PwtAE+fPjw4cOHj6vD57R9+PDhw4ePWwSf0/bhw4cPHz5uEXxO24cPHz58+LhF8DltHz58+PDh4xbB57R9+PDhw4ePW4Rlc9qCIGwXBKFREIQWQRC+t1zv48OHDx8+fPypICxHn7YgCEqgCbgd6AZKgUdEUay76W/mw4cPHz58/ImgWqbXXQO0iKLYCiAIwkvAvcBlnbYgCD6FFx8+fPjw8afGsCiKwdfyC8sVHo8Eui75/+73H5MRBOHPBEE4LwjC+WWywYcPHz58+PBmOq71F5brpP2xiKL4S+CX4Dtp+/Dhw4cPH1fDcp20e4DoS/4/6v3HfPjw4cOHDx/XyXI57VIgWRCEeEEQNMDDwNvL9F4+fPjw4cPHnwTLEh4XRdEhCMK3gH2AEnhOFMXa5XgvHz58+PDh40+FZWn5umYjfDltHz58+PDxp0eZKIr51/ILPkU0Hz58+PDh4xbBY9XjPj4ahUKBWq3GaDTi5+eHRqNhZmaG+fl55ufnsdvtnjbRh49bHn9/f3Q6HVqtFofDwczMDJOTkzidTk+b5uMWJCgoCJVq0a0ODQ0ty3Xkc9peSmBgIPHx8Xzuc59j9erVZGRk8Oabb3LixAnOnj1LXV0dLpfL02b68HHLIggCX/va11i/fj1btmyhubmZt99+m5///Of09PT47i8f14QgCPz93/89sbGxOJ1OHn/8cfr7+2/6+3xinbYgCGi1WhSKP2YAwsLCyMrKQq1WYzAYMBqNvPHGG0RFRZGfn8/KlStRKBTYbDYOHz5MVVUVzc3Nbrc9IiKCjRs3ct9995GZmUlwcDA6nY6ioiJiYmJYvXo1Tz75JDabze22XS8RERHEx8fz4IMP8sYbb1BTU8PIyIinzVqCv78/69evp6ioiLCwMABefPFFGhoa6Orq+pjf9nE5NBoNKpUKjUZDamqqHDk6ceIEMzMzHnOMGo0Gi8VCQUEBaWlpqNVqIiMjiYqKIjw8nP7+fq9z2sHBwahUKpRKJQkJCeTl5ZGbm8tH1SV1dHTQ2trKSy+9hMPh+MjnLhdKpZLg4GDCw8PZvn07e/bsoaqqyu12LCdRUVGsX7+elStXIggC9fX1y3b93PJOW6VSIQgCCoUCPz8//Pz8UCqVqFQqgoKCUKvV8nPj4uJYt24dfn5+mEwmTCYT1dXVZGZmcvfdd7Nx40YAJiYm6Ovrc/tCrVQq8fPzIz09nbVr17Jt2zb0ej2CIOByuYiJiSE8PJyoqCiMRiNOpxOHw3FT3lv6/hQKBQ6H46ZvCIKDg0lLS2Pnzp1cuHCB9vZ2r3LaOp2OsLAwebMUFxeHIAhUVFQwMDDgc9rXgHQdG41GLBYLer0eo9HI6tWrsVgs6HQ62tra6OvrY3x83GM2arVaQkJCsFgsiKKI2WzG398fs9mMIAgeseuDqNVq/Pz80Ov1pKam4ufnh1qtZuXKlWzdupWNGzd+pHOor6+noqKCY8eOMTIywuzsrNsdt1KpJDY2lpUrV7Jz507q6uqoqan5xKQgLBYLSUlJ3H777URFRTEwMMDAwMCyfb5b2mlrNBpCQ0NRqVRYLBYyMzPJyckhICAAo9HIpk2bsFqtS35HuhlnZmYYHR3lqaeeIiYmhuzsbObm5hgYGKC9vZ3S0lL6+vrc+nkCAgJITk7mX/7lX4iNjcVoNAJgs9mw2WzY7XZ0Oh0Wi4WcnByam5sZGBi4Kflt6fvz9/env7+f8+dvrrpsdHQ0MTEx1NXVMTk5eVNf+2ZQUFBAcXExX/nKV9BoNLhcLhQKBaIoeuR0cisTEhJCRkYGn/rUp8jJySE8PJzo6Gj53nM6nQQGBrJnzx5ee+01j9i4sLDA2NgYFy5cQKvVsmLFCo/Y8XHExcWRlZVFcXExO3bsQKfToVQqCQ0NRalUfuxpLi0tjfDwcOx2O88//zzl5eXMzMy4yfrF9dZkMvGlL32JL33pS4yMjBAaGorVamV4eNhtdiwnjz76KBs2bOCuu+7C6XQyODjIuXPnmJ+fX5b3u2WddkhICCkpKXzjG9+QC0n8/f2xWCxySM5sNqNUKpf8nrQIt7a2cuDAAQYHB9Hr9Rw6dIjR0VEmJiYYGRnhwoULbjkFSAVnkZGRFBQUsHPnTqKiotDr9fJzZmZm6Ovr48CBAxQUFLBq1Sq+//3vc+LECUpKSti3b98Nh2LMZjOZmZmEh4dTV1d3U522IAj4+/sTEhJCYGAgGo3mpr32zcLf31+2TUqpeNpZh4WFERoaSl5eHtHR0QQEBBASEgJAX18fhw4dYnJyEp1OR2RkJBUVFfT39zMwMOB2WwMDA4mIiGDNmjWsXbuW6Oho4uLisFgsKJVKxsfHsdls6HQ6/P39WbVqFV1dXZw+fZqBgQG3h6JdLhfz8/OMjIwwNTXl1ve+WhQKBTt37iQvL4+VK1cSHh6OUqlEEAT53x+FKIoIgoDBYGDDhg2cO3eO3t5eLl686KZP8EcUCgUqlUpeo00m0yfGacfGxhIZGYlKpaK9vZ2amhrOnDnjc9ofRKvVEhoaytatWzGbzUvC4JficDhYWFjA6XSiUqnw8/PD5XLR29vLiRMnmJ+flx35yMgI8/PzzM7O0tfX55ZF22AwEB4ezooVKygqKmL9+vX4+/sv2Wy4XC4WFha4cOECERER5OTksHHjRux2O9PT0+zfv/+G7TAajaSkpGC1Wm/6oq9WqwkKCiIsLEwOv3tL9bu0qIWFhREeHo5CocBut8tRl7GxMbfXDigUCkwmE6mpqaSkpLBp0yaSk5MJCgoiICAApVJJd3c3DoeDsbExDAYDiYmJ2O12FAqF25y2IAj4+fkRGBhIUlISSUlJbNu2jXXr1hEQEIBKpcLhcDA8PExTUxPz8/NERERgsViIjo6W/3toaMjtTlu6510ul9flriUEQSA6OpqkpCRSUlKAxbXA6XQucXjSOiUIAjqdTg6hS05drVYTFxdHSEgIJpPJ/R/kfST7JBu9FSnNGhAQwPDwMLOzs5e9RlQqFUajkfDwcKxWKw6Hg6amJhobG+nu7l42+25Zpz06Okp/fz/j4+PodLorOu3JyUmam5uZmpoiKCiI9PR07HY7Fy9e5N1333Wz1R8mKyuLb3zjG9x1111XvKECAwPlvODg4CBVVVUUFhYSHBxMfHz8Tcm/hYaGcs8999z0AhGlUklQUBDr1q1jw4YN7N69m9bWVnp7e2/q+1wver1ejnCsXbsWgIGBAc6fP89f/dVfMTo6umw75ithMBjYtm0bjz/+OKtWrcJqtSIIApOTk5w/fx6LxYLFYuGpp55a8nsWi4W9e/dSUVHhFjt1Oh2JiYl84xvfYOPGjURERCyJEMFiRODgwYP8/d//PWq1mvvvv59nnnkGs9mM1WrFYrF4JH8stVMmJCTIRYfehiiKVFRUEBYWxqpVqwCYn59nbGyMXbt2yZtJyWlrNBpWrlxJSkoKkZGR8s+8JT9/qxAfH09mZiZPPPEE//7v/865c+cum9ILCgpi48aNclqxs7OTZ599lurq6mW175Z12nNzc3R2dvL888+Tk5ODTqejr6+Pe+65h7CwMERRpLe3l5KSEn7+858zOzuLv78/CQkJbNiwgZaWFk9/BKKjo0lPTyc/Px+dTocgCIiiSFNTEz09PfT19fHAAw8Ai5uUtrY2RkdHGR4eZvXq1QiCcNNuSKmY72bf4P7+/nzjG98gLS0Nh8NBb28vc3NzN/U9rpeIiAhSUlJ48sknyczMRKfTAXD69GmOHz8u1wu4q2BGoVAQGhpKamoq3/zmN0lKSkKlUlFXV8fx48dpamqisrISnU5HcnIyX/ziF0lJScFgMADQ1dXl1tB4Wloaa9asYdu2bQQFBaHVauXrR3Iu//3f/8358+cZGBggNDRU/l1POxKp8DIwMBCz2Sw/LhUVNTY2Mjo66tb87wdxuVwcPXqU1tZW9u3bByxGDufn56mvr8fhcMhrBixukPfu3cvjjz/O9u3b8ff3RxAE5ubmuHDhArW1tXR2dnrs80hI3QTeyrZt27j99ttZsWIFiYmJtLe3X9Zph4WF8ZnPfIaoqCh6e3t55ZVXqK2tZWhoaFntu2WdttPpZHx8nFOnTjE1NYWfnx8dHR2yAzebzXR0dHDhwgVOnjyJzWbDYDDQ1NSEQqHwqNOWqmtzc3PJzs4mIiIClUrFwsICMzMzlJWV0dnZydDQEGvWrGFubo6Ojg56enoYGBiQwzVSftBgMDAzM3PdleR+fn5otdolIbWbhUajIS8vj4CAANlpu/vkeiXCwsJkxyNVDE9PT1NbW0t1dbVbNxcKhYKgoCCys7PJy8sjLy8Pp9PJ6OgoZ8+e5dChQzQ0NNDc3ExgYKDcmiiFeB0OB52dncvSF3olYmJiSEtLIyYmRu5wsNlszM3NMTIyQkNDAydOnKClpYW5uTk0Gs2yXGPXg/S9zc3NLUl/WK1WMjIyqKqqksVWPElXVxfDw8M0NjYCi3Y7nU7GxsY+lL5TKBSMj48zMjKyZC2Qro3BwUGvKALVarXyRtMbiYmJIT09neDgYCwWy2Vt9fPzIygoiKysLEwmE3Nzc9TX1zMyMrLs68Yt67QBpqamOHbsGKWlpQiCwOzsLPHx8WzYsIE77riD9957T85bA0xPTzM9Pc1PfvITj9otheX+7u/+juTkZPmENzAwQEVFBf/v//0/dDodKSkpPPfcc9TU1FBaWsrIyAiiKNLV1YXD4SA+Ph4/Pz/S0tJoaWm57haq6OhoYmNj5XzpzUShUGA0GlGpVExPT1NSUsLo6OhNfY/rZcWKFWzevJng4GBg8fqoqKjg0KFDlJSUuNUWvV7PHXfcwVe+8hU5TH/hwgXOnTvH9773PWZnZ+WFODc3l/Xr11NQUAAsnmqHh4c5fPgwtbXum8uzYsUK8vP/KJtss9kYGBiQCxmff/55+vr65PqFyMhIgoKC3GbfR7GwsMDQ0BAHDx5EFEXi4uIAyMjIIDk5mf7+fo4ePer2DpIP4nK5mJmZuarNg0ajYc2aNUs6T2DRaQ8ODt7Qxv5mEh4eTmJiImfPnvV4seflsNvtLCwsIIoiKpXqsqnX6OhoEhIS5Cp+qb7DHRvSW9ppS0hOWRRF5ubm5P8vKipiZmaGkydPes3FERwcTGpqKjt27CAsLAylUsnk5CQTExNcuHCBQ4cOMTExwfDwMKOjo1RXVzM9Pc3ExMSHPoNUjXn//ffz6quvXrfTzszMJDU1FY1Gc1MvOp1OR0BAAGFhYeh0Orkyf2Fh4aa9x/UgtaGsWrWKdevWIYoi8/PzdHd384c//MHt+fbCwkLWrFnDl770JaKjo5menubo0aO8/fbbVFZWMjs7uyREv2XLFjZs2CD/rdrb2/ntb39Le3u7W0+Gb775Jo2NjbS2ttLU1MTg4CC9vb309/czOjrK4OCg7CQEQcBsNn8o5+1JRFHk3XffRa/Xc9999wGeD9tfLwkJCaSkpHDXXXeRlJS0xNFMTk7K14cncDqdcoGvFB2UNsrehMlk4r777mPNmjUEBwfT3d1Ne3s7PT098nMEQUCj0bB161bWr1+PVqulo6ODqqoqTpw4wezs7LLb+Ylw2pdW9vX29tLd3c3k5CSxsbEkJCRgsVi8Qk9YEAQSExPJyckhPz8fvV7PxMQELS0t9Pb2Ul9fT2NjI/Pz89hsNmZnZz8yPyLt7lJSUrBYLNdtV2hoKMHBwSiVSvm9bwZhYWEkJibi7+8vb6hmZmY8+ndQqVTo9Xo5XxUSEoIoigwNDdHa2kpFRYXbQogKhYKIiAjy8vK47bbbyMjIYG5ujr6+Pk6ePElpaSktLS1y7lKtVmMymUhMTCQmJgZYjM40Nzdz+vRpJiYm3Prdtra2MjMzg06no6GhgaGhIYaGhpiamrqsHVardckJ0Bvo6upiaGhoSW74VnHcer0ek8lEWFgYGRkZZGZmkpWVRWBgoBwxs9lsTExM0Nzc7LFQv9TlIhUNazQaObroLUgHoOLiYqKiolAoFFy4cIGenp4l64FOpyMhIYGVK1eSnJyMw+GgvLxcbrd0x/33iXDal7J7924GBweX5Abz8/M5ffq0R/NT0g7toYceoqioiLy8PObm5igvL+e///u/qampweFwoFarrymEpVAoCAkJuaGbICwsTN759vT03LRCio0bN7Jz504CAwMZGBigt7f3Q6dGd+Pv709SUhL/8i//QkJCgvz42bNnOXLkCGVlZW6zRa/X8/nPf5777ruPVatWYbfbaWpqorS0lJ/97GfMz8/LG1KVSkVAQABr1qwhLi6OgIAAAPbs2cPhw4c5efKk2+yWmJyclLszrobk5GS5qtmbuLSg01sicldDQkICBQUFfOMb35DbjqRhFRKSWJSnJExFUWRmZob29nbq6uq8tlLfZDIRExPDPffcg9FopKenh//4j/+QI50SkZGRfOtb3+Kuu+7CYrEwPDzMj370I2pqatxm6yfOaS8sLNDY2Mi//Mu/8E//9E/ExMTwne98h3PnztHX10dPTw8HDx50uwM3GAzExcWRn59PSkoKoijyX//1X5w9e5bS0lJmZmbk9gxPOTVRFGlra7vu8LBerycmJoa4uDiKi4spKioiPT0dQRA4efIkhw4duuIpzF3ExcWxatUqEhMTMRqNzM/PU1tby3vvvcepU6fcZodGoyEwMJCdO3cSFxfH/Pw85eXlPP/883IdhsvlQq1WExwczKOPPsqKFSvIzMwkNjaWqakpqquree2119y6YFwNGzduJDg4eEnRmSAIZGRkEBwcjMvlorm5mZqaGrkK2pNIPdve6LCDg4OJi4sjIyODrKysJVXX8fHxJCUlERERgUajkU/XCwsLjI+P89JLL1FTU0NLSwvz8/MejzR6I0qlkujoaD7zmc+wceNGrFYrZWVllJSUUFlZucRhb9iwgbVr17Jjxw6sVis9PT3s2rXL7TU6nzin7XK5mJiYoKqqigsXLpCVlUVGRgYajYahoSF6enpob2+XdY/dlV81mUzk5eURHh6OVqtlfHyc0tJSLly4wNjY2DW/3qWL4c3EZrNd9jsxGAzyicRisaBWq2Xdd5PJhNFoRK/XExUVRVRUFNnZ2URHR8sysl1dXTQ1NXlsgVYoFAQEBJCens6KFSswGo1yMdKpU6eor693ay5bqVSi0+mIiYmRNw/nzp2jpqaG7u5uoqOjMZvNmM1mYmNjWb9+PVlZWURFRQHQ3d1NdXU1Fy9e9IgC2qUoFAp0Op0sFHTbbbcRHh7+oUrx2NhYTCYTNpuN8vJyGhsbmZyc9ApneakN3mCPSqVCp9ORn59PWloaubm5ZGZmLslVh4SEyAp5lyIpKB47dozm5ma3hW2vFmk2xKUpCXcjrVtSr/Vtt93GypUrUavVciGaxWLBbrfjcrlQqVSkpqaSnZ1NWFgYY2NjtLa2cvr06SWO3R184pw2LDqe3t5enn32WTZv3sy3v/1tioqKUCgUclvK2bNnOXPmDIODg25RRIqOjuZrX/sawcHBTE5OUl1dzYULF657EMWlIb2b6bgNBsOHQu2CIBATE4Ofnx8qlYqioiICAwPx9/dHrVazYsUKsrKyEAQBu93O7OwsVVVVOJ1O7HY7KpWK/v5+Ojo6bpqd14pGo2Ht2rU88MADbN68GYVCQVdXF5WVlfzzP/+z25XPlEolarUaf39/NBoNg4OD/OxnP2Nubo6oqCjuv/9+Vq9eTUJCAomJibLmNCCr9x08eJChoSGPqsspFAq0Wi2xsbFkZmbyhS98gbVr18o1FpIkrHSPzc3NMTw8zM9+9jOampq8Vo3M0xiNRuLi4nj66adJTEyU0yFXQ29vLxUVFezZs8fjUYzLodPpPF7boFAoyMjIoKioiH/4h3+QN0Mul4uoqCgcDgef/exnefPNN2lvb5eld3NzcwEoLS3l8OHD7N692+22fyKdtkR5eTl9fX3U1dXx+c9/npSUFOLj4/nsZz9LTk4OKSkpvPzyywwNDS1ruDw7O5vVq1eTnJyMn58f3d3d8mjC68XlciGKIna7nZaWFiYmJq77tRYWFuSF//HHH+euu+76UMWkNL5QEAR58pjT6aS7u5vBwUHeeOMNLl68SGdnJ729vfT29vLUU09x7733otVqGRoa8pgKmslkIi4ujh//+MdERkai0+nkntbe3l5GR0fd7vicTqccxrRarYSEhLBr1y6cTidKpVIu2lIqlczMzMh9zmazmenpaXp6ejh58qTHdLOVSiVRUVGkp6fzzW9+U5bI9Pf3x+Fw0NbWJusmmEymJSdElUrF2rVrEQSB8fFxr3Qs3sJHCShJP/vgxicxMRGNRkNRURH19fXLLvZxrSQnJ2Oz2TxW8KfRaNiwYQMPPfSQvIGX5GH7+/uxWq0EBgaSnJxMYWEhExMTBAUFkZGRgdlsZnJykv/93//l7NmzHrH/E+20Jycnsdvt2Gw2oqOj5crWtLQ00tPTWVhYoK2tjerqaurr65fNDrPZLE8ekwQz6urqrvlkJwgCQUFBJCcno1AomJmZYWBggPLy8hsS329ubiYgIICYmBg0Gs1lC9tCQ0NZWFhgcnKS3t5euQ2tr6+PwcFBBgcHaW9vl52g0WhEFEWUSiVzc3MfErFwJ2FhYaSnp5OcnLwkJ9jd3U1LSwt2u93tYTqn08nMzAznzp0jJyeHqKgocnNzsdls8iCLlpYWpqamGBkZISkpiZCQEMxmMwMDA/T09HhkrKVUxR4UFERhYSH5+fkUFBQwMTHB9PQ09fX1DA4OMjY2Rn9/P2FhYUt6XaWRmKtWrWJ6eprOzk76+vpwOp0eDUt/sHpcGinqKex2O5OTk5SVlTExMUFUVBSjo6M4nU7ZSV/qtC0WC1arlaioKHQ6nSwR6y0a39PT07IgjMFgWKJC505MJhOhoaFs2rSJ3NxcIiMjWVhYoLOzk4GBAfr6+sjMzCQiIoKoqCiUSiU2mw1/f39MJpMswBQREUFkZCSTk5NLCkbdwSfaacNiOK69vZ2f//znxMTEkJ+fzz/8wz8QFRVFZGQkVquVP/zhD8vqtNVqtZwDHhsbo7Oz87p6+tRqNfn5+Wzfvl2eKFNRUcELL7xwQyeu1157jaqqKoaHh1mxYsVlF6vBwUFaW1upqqqipaWFzs7OK4b2DQYDO3fuJCkpCb1eT1dXl8ccNiyKkdx1113ywiwtztKENE84i4WFBQYGBviHf/gHvvWtb3H33Xfj7+8vn/6PHDnCkSNHuHjxIpOTk3zve9/jtttuIzIykqqqqmXXN74SSqUSs9lMfn4+Tz31FHl5eQC8/fbbVFRUUFFRwYULF7DZbJjNZvl6knT11Wo1FouFBx98EIvFwsjICAcOHPC48Mel1eNKpZK0tDQuXLjgMXtmZmZobW3l7//+72WpYyk6d7mo0OrVqyksLOSxxx5DrVZ7RV7+Unp7e2loaPC4XfHx8RQWFvLkk0+iUqnkzdHLL7/M4cOHmZyc5HOf+xzFxcXk5+cTHh6+JCIgiiL+/v48/fTTnDx5kr/6q7+iu7vbrdoT1+20BUGIBl4AQgER+KUoiv8lCEIA8DIQB7QDnxFF8dorrW4yMzMzXLx4kcHBQf7mb/6GoKAgFAoF2dnZNDU1sXLlSqqrq5d94ZByvsPDw1f9XgqFAr1ezyOPPMK2bdsoLi5mbGyMY8eOsX//fqampm7YbknHXQodXw7pFHilYjUJjUZDfn4+ISEhzM3NcfjwYbfKa0oolUoSEhJYsWIFq1atkj+XVPRVW1vr0cElDoeD2tpann76af71X/8VpVIpT0Cbnp5mamoKpVJJSkoK2dnZpKWlIQgCw8PDHgl5qlQq7rjjDgoLC3nooYcICwtjenqaCxcu8Oabb1JVVcXo6Kgc/v7GN77B2rVr8ff3x2638+abbyKKImFhYRQUFFBYWEhSUhI7duzg/PnzlJaWMj09LUt1tra2uu0Ec+lmThAE1q1bR2trKydPnqSzs9NjuffBwUFZeGlmZkZOi32QkZERmpubefTRR684PMmTTE5OMjQ0hCiKch1EeHg4w8PDbpU1zsrKYseOHQwPD3P27FnOnz/Pvn375Cisy+Xid7/7HVVVVfz5n/85cXFx+Pv7f+h1GhoaqK6upru72+2ptRs5aTuA/yuKYrkgCCagTBCEA8AXgUOiKP6jIAjfA74HfPfGTb0+VCoVBoOBkJAQOSxzqdycy+XCbrfLIzqXm6GhoWty2GazmZCQEKKjo1m/fj1RUVHMzMxw9OhRTp06RUNDw02pDF1YWGBwcPCGXwcWFz2j0YharWZhYYH6+vobyrlfL2q1ms2bN5OVlUVQUBCCIMgphcOHD9PV1eVRdTZJcOajhjhIVfkGgwGtVisPwnG3vKZWqyU1NZXCwkIKCwuJiYmho6NDjhq1tLQwNjaGRqNhxYoV8tx3jUbDwMAALS0tHD16FKfTSWhoKHa7neDgYKxWK7m5uej1eoKCgpidncVutzM1NUVHR4dbnOXU1BStra1ERUXJdRtms5nIyEhSU1Pp6enxmNO22+3yJu6j8FQf9tXicDjkaJs0nEir1d502eSPw+l0Mj09zeHDhyktLaWqqoqampol393Y2Bjj4+OoVCoUCgXT09PU1dUtiQZJnT+eWD+u22mLotgH9L3/31OCINQDkcC9wMb3n/Yb4CgectrSrOT4+Hhuv/124uLiSExMJDAwUA6VSkPh3RW6qa+vv+phJUqlktjYWIqLi9myZQt33nknLS0tHDt2jG9961seDTlfLTabjdLSUo8MvNfr9Xz3u98lODgYrVYLLIpNVFVV8dOf/tTjwyCuBkmUB/7YilRdXe3W3myFQoHVauXhhx+W0x6zs7Nyb/vBgweZnp5Gr9eTnJzMV7/6VXJzc4mNjaWzs5MzZ87w7LPPUlJSgsPhQK/Xc/r0abKzsykoKGDr1q2kpaXx6U9/GoCJiQna29t55ZVX3HKK6e3t5cCBAzzyyCNLTqmxsbFs2rSJEydOeM389ysRGRlJdnb2FaNknkYq9JKQaiPcbW9XVxcnTpxg//79DA8PX3YzpNfrsVqtREdHo9Vq6ezs5D/+4z9ob2+Xn7/cxcsfxU3JaQuCEAesBM4Boe87dIB+FsPnl/udPwP+7Ga8/wfR6XSEhoayY8cOMjIyyM/PJyoqCq1Wi0ajkRdwqShMCsktF1LBiCAIrF69+qpOtOnp6RQXF/OlL32JsLAwTCYTR44c4Y033uC9997zuH73rYBCocDf339JMU5paSl79+5lenraq3pXr8TMzAynTp26rl7+m4WUL/3iF7+ITqejs7OTn/70p5w+fZrx8XFycnIoLi4mJSWF/Px8wsLCGBwc5Oc//zn79u2jvb2dtrY2OZo1NTVFSUkJFy5c4K233uLUqVNkZ2eTnp4OwPHjxzl06JDbwqbT09O0t7d73DELgoBSqWTlypVyOu/jNuZqtZrt27dzzz33sGnTJq8MjcNisavNZqO1tZXw8HACAwP53Oc+x0svveTW+oyKigrq6+sve/9LhzxpJoHRaJTTDvv27WNhYUGOuFxaEOhubthpC4JgBF4D/lwUxckPJO1FQRAu6w1FUfwl8Mv3X+OGPaZCoUCj0RAeHk54eDg5OTls3LiR+Ph4EhMTMZlMKJVKRFFkfHycoaEhObQnjb1zB1arldjYWPLz8+nq6mJubg6HwyFXewYHB+Pv709KSgp5eXlER0cjCAIjIyOUl5fT0NDg0TzstSCFwT7Y8uMOoqOjycrKws/PD4VCIW/KBgYG3JorvVEUCgVms/lD8pTuJC0tjYKCAoKCgrDb7fLJOycnB1EUSU5OlivgExISaG9v58KFCxw5coTq6mpGR0eXFF26XC5mZ2eZnZ1lfHyckpISBgcHuXjxIrA43ay+vt5tfyNpMM/4+Dh6vV7e1AcGBpKSkiK3sS3nRtlqtRIUFCSrCV68eJGenh5Z3OODqNVqOXW2adMmcnJyiIyMlKvJnU4nDofDa65zSQO9vb1dFmpqa2tzuzCJ1MlyOTQaDWlpaaxYsYKMjAwWFhYoLS3l5MmTXjHSVOKGVgJBENQsOuzfi6L4+vsPDwiCEC6KYp8gCOHAzUmUfgxqtZrAwEDWr1/PqlWruOuuuwgPD8fPz09esF0uFy6Xi/b2ds6dO8fu3bs5fPjwsu/oL5VJNBqNpKen89hjj/Hmm28yMDDAzMwMubm5ZGVlUVhYSFpaGlarFZPJRG9vL52dnTQ3N3Pw4EHa2tqW1dabjUqlIjQ0VF4I3cXq1at58MEHl0wuE0VRbk3z5vzfpeh0OnJycjzWIgOwatUqtmzZAvyxHeruu+/Gz88Pg8FARESE/H2KosiRI0c4duwYb7311sc6DZfLRVlZmVs13z9IX18fQ0NDdHV1YTab5Ws1Li6OkJAQIiIimJ+fX1annZCQQH5+Pvfffz9r165l7969HDt2TN7UX4qUc09JSWHFihU88sgjstCRKIosLCzILZbe4rRhsW6mtraWoKAglEolf/jDH9w6s/7jMBgM3H777WzZsoW0tDRGR0f5wx/+wMGDBz1t2hJupHpcAP4XqBdF8d8v+dHbwGPAP77/77duyMKPQJrMEhkZSUZGBo8//jhJSUn4+/tjMBiWnE4aGxupqanh8OHDHD16lLGxMaanp92eF1YoFMTExPDZz36W9evXMzMzw/z8PPHx8RgMBvz8/NBoNHKfbmlpKceOHePYsWNy3/mtgrRRUSgUbhNSUCgUJCUlkZeXx5o1a2QRmOnpad58803Ky8s9Gmq+VjQajZxb8xRTU1OMjY0REBCASqXCaDSSmpqKIAhMTU1x/PhxGhoaaG9vp6ysjIsXLzI+Pu5VDuPjEEWRjo4OgoOD5Znfoii67TN85StfkQWYtFotycnJPPHEE3JF86UolUp5brYkFaxQKLDb7VRWVnLu3DnKy8s5e/asWyuzrwVJGMpbrhGLxUJiYiIPPPAA0dHRjI+P88orr1BbW+t2bfGP40ZO2sXA54FqQRAq33/sb1h01q8IgvBloAP4zA1Z+AGkwpyMjAzCw8OJiIggOjpaFtUPDAxEo9EgiiKjo6OMjIzQ3t5OeXk5zc3NVFZW0t7e7lZnPTY2xuDgIMPDw7Jut6QpLVWHBgQEIAgCCwsLtLa2UlNTQ2lpqVwkd7Mqu92NUql060lbqlz39/fHarUiCALz8/OMjo5y4MAB2trabqmNj8PhYGRkBLvd7jGt5rKyMjQaDStXrsRsNqNWq7HZbHR3dzMwMEBDQwNdXV1ylfjExMQt9R3DohOpr68nPDycjIwMt7+/JI4iaSSEhIRQVFSEzWb70N9coVCQnJxMQEAA/v7+zM3NMTY2xsDAAG+//TZ1dXW0trYyOzvrlRElSWAnJiaGvr4+rygIXbFiBcXFxURGRjIxMUFHRwdHjhxhcHDQ62pfbqR6/CRwpePTlut93Ssh5UdVKhUWi4WdO3dSUFBAcnIy4eHhsoKXtDt2uVy0tbVRVVXFO++8w5kzZzy2mPT09HDx4kWam5vJzMyU5Skl/V3J5rm5OSYmJjh27Bj79u3jrbeWLUjhNtRqNfHx8bK4hjuQtI2l95ybm6O/v5/XXnvtlivgs9lsNDU1LVnYPkracjnYs2cPZ86ckWtETCYTo6OjHDp0iPb2dlknwBsdxNUiiiLnzp0jOjqaLVu2uP07lnLQTqcThUJBaGjoZcdYXpqGgMVN3djYGHV1dVRUVPA///M/Hh9/eyWkNVyS483NzWVubs4rnPbWrVt5+OGHCQoK4sSJE5w4cYK3337b02ZdlltCEU06WW/dulWuMo2KipJ3/VLbgJTP6erqYt++ffzud7+js7OTiYkJj+Z3RkdHOXXqFN/4xjfYuXMna9as4e6775Z/Looir7/+OmVlZZw6dYqWlha3F2gsB+5e+D6JzM/P09TURFtbG4mJiYSFhcnTndwlYzo/P8/AwABvvfWWPJ3J5XLJ4x69JcR5I0jh8ZaWFlpbW4mLi3Nr8d9//dd/kZOTw9atW9m2bdsVaximpqYYHR1lfn6enp4eWltb5aEW/f39y94JcyNotVq2b99OaGionP7zlnWurKyMgIAAnnjiCWpqajh37pynTboit4zTTktLY+XKleTk5BARESFXIE5OTtLf38/c3ByiKFJXV0dbWxvl5eW0trZ6xUACl8vFzMyMXK0ujQeVEEWRkpIS2tvbaW5uZnR09JZeCJ1OJ52dnbKoiTu5tDtgYGCA0NDLdhzeUtjtdurq6ggPDyc0NJScnBxZYWpyctItpyqp4vuTzPj4OOXl5fzmN78hODgYhUIhiw4tdzpNajlbWFhArVaTlJREYmIifn5+8t+6r6+Pnp4eOjo6mJycZHR0lMHBQerq6hgdHfWKE+tH4XK5GB8fR6PRMD4+ztzcnNdEBJqamhBFEZvNxrFjx65aS8MT3FJOOyUlhdjYWGCxEnF+fp6Ojg5KSkoYGhrC5XKxa9cuOjs7ve4CdjqdTE5OyqGX3/zmN542admw2+3U1NSwcuVKedCIu27OSyvEGxsbMRgMzM3NeW1BztVy/vx59Ho9GzdupKioCI1GQ0NDA83NzZ94Z+ouJiYmOHnyJCdPnnT7e0s1L3V1dczOzrJ+/XosFgtms5nu7m6qqqo4c+YM9fX11NbWMjw87LUn6itht9upr69namqK2dnZJX3Pnqa2tpba2lrefPNNT5vysQje8If/uD5tqen9crJ30ohD6Y8vhey84XP9KaPVatHr9ahUKubn55mbm3NbPYEkkajVauU2GIfD4XVVoNeCv78/2dnZ/PCHPyQjI4PZ2VmOHDnCv/7rv3Lx4sVbLlfv48pcKgIldT9IfeIOh8Or+q+vBalVTdLLmJycvKKW+p8QZaIo5l/LL9wSTtuHjz911Go1YWFh3H333dx5552Eh4ejVCr5i7/4C6qqqm6pNjYfPnzIXLPT9k6hWh8+fCzBbrfT29vL7373O/bu3UtjYyOxsbFL2oR8+PDxycd30vbh4xZCEAT8/PxQqVRoNBqmp6dv2XCpDx8+rv2kfUsUovnw4WMRURRv+aI6Hz58XD++8LgPHz58+PBxi+Bz2j58+PDhw8ctgs9p+/Dhw4cPH7cIPqftw4cPHz583CL4nLYPHz58+PBxi+CrHvfxJ41SqSQgIIA77riDhIQEBEHg97//vVdrD/vw4eNPF5/T9vEni16vx2QykZSUxH333Ud+fj52u50jR474nLYPH8uMUqlEr9fj5+eHn58f4+Pjsgy1jyvjc9o+/mS5//772bRpE3fccQcWiwWbzcapU6e8ZlygDx+fVFQqFVFRUTz00EMUFxezceNGnnrqKfbv309nZ6enzfNqPnFOOzAwkMDAQOLj41m5ciUKhYKBgQEGBgaw2+2Iosjp06eZnZ31qUj9CaLVagkMDOSzn/0s69atIzU1FavVytTUFBcvXuT555+np6fH02b68PGJJTg4mLi4OL72ta8RHR2NwWDgzJkz9Pb2+ibWXQWfGKcthVqSkpKIi4tj5cqVbNu2DaVSSXt7O62trdhsNlwuFw0NDTgcDp+y1PsIgoBKpUKr1SKKIgqFArVajSAIqNVq/Pz8sNlsOJ3Oy250RFHE6XQyNzeHw+Hw2vCWSqUiKCiI5ORkPv3pT5OQkIDZbGZiYoLu7m7q6uo4cuSI15y0/f390el0aLVahoaGmJub8200bxLS9a7ValGpVPI/giAAYLPZmJ6eZmpq6padQiUIAhaLBb1ej1KpZGxsjNnZWY/dn4IgoNVqSUhIYOXKldx+++3ynPCzZ8/S09PD3NycR2y7lfjEOO3g4GDuuecevv71r5OUlIROp0OhWCyOz8zMlG88p9NJTU0NZWVlNDY2etJkr0Gr1RIdHU1xcTE2mw2LxUJ8fDwGg4Hk5GTy8/MpKSlhdHSUiYmJD/3+1NQUY2NjHDx4kPb2dgYHBz3wKT4apVJJXFwcDzzwAJs3b2bFihUolUqGhob4r//6L1paWujs7GR8fNzTpsp88YtfZMuWLRQXF/ONb3yDw4cPMzQ05GmzPhFERESwfv16ioqKiIqKIiYmhri4ONRqNS6Xi8rKSnbt2sWvf/1rpqenbznHLQgCOp2OJ598krvvvpvg4GD+9m//ln379tHX1+cRm/z8/Fi/fj3f/OY3WbVqFa+99hovvfQSVVVVzM3NIYriLfc9e4JPjNOempri/Pnz8ozWS+duS7tn6b8fe+wx4uPjefPNN2lsbMThcHjCZK9hw4YN5Ofns2PHDpxOJxqNBpPJhEqlwmQyYTAYyM7OxmazXXZus8PhwGazUVRURE9PDz09PVRVVdHS0kJfX59XzLFWKBQYjUYSExPJzMyUN3QOh4Ph4WGam5vp7u72sJWLSBGOmJgYkpKSMBqNJCUl0dHR4XPaN4AUjbv33nvJycmhsLAQk8mEw+FgbGwMu91OYGAgoaGhpKSkcN999xEaGsozzzzjtWFbaUa1NIMbYGFhAUEQSE1NJTMzk5iYGKamppibm/PYSdZkMhEdHc03v/lNAgMDqa6u5ve//z0XL15kfn7eF0G6Bm7YaQuCoATOAz2iKN4tCEI88BIQCJQBnxdF8cMr/U1mfn6e1tZWenp6iI2NxWg0IgiCHLqVQl+CILB27Vrm5+eprq7m4sWLt6TTlj6PQqHAZDLJVZiw+F3Mz89f9QKfm5tLUVERa9euXbLTdblcOJ1ObDYbJpMJk8kk/8zpdMoOXrIjJyeH4eFh+vv7CQwMRKFQMDc353GnrVQq0el0xMXFERMTQ1hYGACjo6P09PTQ399Pf38/IyMjHrXzUhQKBQaDAbPZjEqlIjQ0lNDQUE+bdc34+/ujVCqXpFp0Oh2weA3Z7Xa6urrcErI1GAwkJiaydetW0tPTiYuLo7e3l+HhYdrb2wEICwsjOTmZpKQkcnNzCQwM5H//93/p7+/3uOPWaDTodDp0Oh1qtRpYvE7CwsIwm83yiNaRkRFsNhuZmZlERUVhNpsZHBzEZrNht9vdbrcgCISEhJCWlkZBQQEXLlygvLyc8+fPe20qDRa/W71ej1arxWKxyNcxwPj4OLOzs0xOTrrdrptx0v4/QD1gfv///wn4D1EUXxIE4efAl4Gf3YT3+UicTicTExPs2rWLoaEhnnjiCbRaLfPz8wwPDxMWFibvRA0GA7GxsRQVFXHs2LFbLretUCgIDw9Ho9FgNpu5++672bRpE+np6QBUVlZSWlrKD37wg6sKN61Zs4bc3Fzgj1EJURSZnJxkeHj4sifQ0dFRRkdHSUhIQK/XYzQaycjIIDg4mODgYLKyshAEgbGxMdra2jwa9rJarSQlJfH3f//3hIeHy4//7ne/Y/fu3Rw+fNirwnLSJK/+/n56e3sJDw/HZDLh7+/vadOumXvuuYfQ0FDUajUJCQmkpaVRWFgIwNjYGB0dHezYscMtG6asrCz+8z//k8jISHp6enjuuef49a9/TV9fn1zHYLVaiY2N5bXXXiMsLIzExESeeOIJ9u7dy/Hjx5fdxishCALx8fGsWbOGdevWERkZiUKhQKlUEh8fT0BAAGbz4hJ84MABampqSEpKIiIiAqfTyeDgIA6HA5XK/cFVjUbDPffcw7333ktbWxu/+tWveO+997zaYSuVSsxmM+vWrWPVqlV88YtfJDAwUD4YPfvssxw9epRXXnnF7bbd0F9QEIQo4C7gGeApYXHF3wx89v2n/Ab4W9zgtCVKSkrQaDTcddddREVF4XQ6mZ2dlRdlySl1d3dz4MCBW8ZhKxQK0tLSiI2NJT4+nnXr1mEwGNDpdISHhxMcHCzftLGxsQwPD1/1a//617/m8OHDREREMDQ0JO/GZ2ZmZMf9QRYWFlhYWMBgMKDVagkKCuKf/umfCAwMRKfTIQgCcXFx5OTkcPz4cY86xcTERIqLiwkNDWV4eJjy8nJeeeUVysvLaW9v9yqHfSkVFRVYLBZWrVpFYGAgQUFBnjbpI9Hr9cTFxREaGkpgYCAxMTHcddddWK1WOT1hMpnk1ITJZCIuLo5nnnmGd999l3fffXfZbEtKSiIzM5OEhASampo4c+YMv//97+np6WF+fl6+Bqampujo6GDfvn0UFBSQmZnJbbfdRmNjo8eddlpaGmvWrGHLli3yPSYIAhqNhv7+furr60lNTSU9PZ34+HiMRiMBAQHMzs5y9OhROjo63B4t0Ol0fOpTn6K4uJigoCCeeeYZysrKPB61+CjCw8OJjY3lU5/6FNnZ2URHR8sOW7p2t27dSkxMDEFBQbzyyivXtN7eKDe67fpP4K8AKW4aCIyLoijFm7uByMv9oiAIfwb82Q2+/4fo7e3l4sWLXLx4EbPZjFKpXLK7FEURQRCYmpqira3Nq0PjgiDIJ2qtVktBQQGpqamkpqayYcMGudhuYmKChYUF+vv7mZycZGhoiLGxsat+n5KSEpqamggJCaG3t1fOW9tsNubm5j62mtpsNhMTE8PCwgIul0suKFGpVHJ0wxMoFAri4uLIzs4mJycHjUZDV1cXJSUlvPXWW4yNjWGz2Txm38fR2dlJc3MzsFhoGRYWhkqlwul0enyjoVKpCAkJQa/Xy6Fas9lMWloaMTExhIaGkpycTEZGBmazWT6hXGq3Wq3G39+fLVu20NzcvKxOOyAggKCgIPR6PZ2dndTX11NTU/Oh59ntdiYnJ6msrCQyMpKcnBxiY2MJCAhAoVB4JPeq0+kICAggKytLDuuPj4/jdDpxOBx0dHRQX19Pd3c3TqeThIQEEhMTUSqViKLI0NAQVVVVDA8Puz08rtFoKCoqIiYmBlEUOXnyJMPDw157ypYKVvPy8ti2bRtRUVHodDqmp6dlx2wymYiJiUGv1zM9Pc3evXtvDactCMLdwKAoimWCIGy81t8XRfGXwC/ff62bugJ1d3fz7LPP8vjjj5OamkpiYqL8M+mkfWlbk7eiVqv5v//3/5KYmCgvIJduQGw2G6Ojo7z99tu0trbS1tbGu+++K7dnXS2Dg4MMDg7S1NR0XXbGxcWxdu1aIiIi5MXZbrdTVlbG/v37PVZkYjKZ+MUvfkFSUhKBgYEMDAzw1ltv8dZbb9Hf3+8Rm66Fzs5OOY+dl5eH0+kkKCiI0dHRyxYEugupde4v//IvKSwsJCIiQn7caDSi1Wrl/F99fT3T09PEx8cDfOh+k9qAJMe/XCiVShYWFujs7OTYsWNUVVVd8blOp5Pjx4+TkpIC/DGXLC3S7iYzM5N7772Xr3zlK1gsFhwOB/v27WNsbIyRkRF+9atfMTo6it1uJz09nW9/+9vcd999WK1WZmZm6O/v59ChQ24vQlMqlZhMJrZv387o6CgVFRX09/d77UFJoVBgsVh46KGH2LFjBwkJCUxNTcnXzJkzZxAEgTvuuINNmzZhMpnIycmRi3bd9blu5KRdDOwUBOFOQMtiTvu/AIsgCKr3T9tRgNuVKsbHxzl16hRjY2MUFhby5S9/mYiIiCWnvqSkJB555BH+/d//3SPFBB+FwWAgMzOT4uJiduzYgcFgAKCjo4Oamhpqa2vp6enBZrMxOztLfX09MzMzzM7Ouq0SU6FQoNVq+exnP8uaNWvIy8uTN0FjY2P86Ec/4uTJk7S1tS27LZcjKiqKjIwM0tLSsFqtOBwOzpw5Q1dXl0cd3vUgCAJKpRKr1UpBQQEnT570WNFcbGwsKSkpfOELXyAvL4/Q0FC0Wu0SO5VKJTU1NfzsZz+jv78ftVpNYGAg/v7+ZGdns23bNqxWK0qlEpfLRU9Pz7K32rW0tDA2NkZNTQ2NjY0fG4mS9AYEQcDf35+0tDSKi4s5dOiQ252O1FPu5+fHwsICIyMjvPTSS/T19cnpK61WS3h4OA888ABZWVkYjUYWFhY4ffo0J0+elDUq3El2djaFhYUYjUYOHjzInj17vLpKPDAwkL/+679mw4YNWCwWKisr+e1vf8uFCxcYGRlhbGwMjUbD0NAQCQkJJCcnExQURGRkJP39/QwMDLjFzut22qIo/jXw1wDvn7T/UhTFRwVB2AV8msUK8seAt27czGvD5XKxsLBAX1+fXIAhhWylnb7BYCA8PHxJa5g3IAgC0dHRrFixgo0bN2IwGJiYmKC/v5++vj7KysqorKyks7OThYUF7Ha729uApNxkcHAwGzduZMWKFSQlJSEIAiMjI7S1tXHo0CG6urqYmppyq22wuMMPCgoiISGBgIAA/Pz8mJqaoru7m/Hx8cuGxC0Wy5ITn81m86p+c4VCgUajkdMlniI2NpaVK1eyadMmrFarHFlxuVxy8dzc3BxlZWXs3buXqakplEolBoOBrKwsgoOD5YXb6XQyPz9PbW3tsvcOj42NMTExQVtbGwsLCx+bXpDao+bn5+Wajbi4OI9E5hYWFpicnMRms6FWq+U6HalQUalUEhISQmpqKqtXryYsLAxRFOnq6qKiosJjVdqRkZFkZWVhs9loa2ujurraq522n58feXl5sl9oamri1KlTlJWVyc+RokhTU1O4XC65I8Kd9+RylBJ+F3hJEIQfAxXA/y7De3wkQUFBbN68maSkJHJycuTpTZfecCMjI1RXV3ukBeKjUCqVPPzww2zYsIG1a9fywgsv8N5777F//35mZmY8bR4KhYIVK1awcuVK1qxZwwMPPCBfsA6Hg4MHD7J7925aW1s9VuQndQdkZ2fLhSMul4upqSlGRkYue0otLi4mPT1dDkd3dHTw//1//59b7b4S0mZTSulIn8kTrF+/njvvvPND7WcOh0OuDK6traW7u3uJhvTIyAhf/OIXycvLIzg4GFgs+urt7eWf//mf6erqWla7r+V0LIoi/f39dHZ20tHRQVJSEv7+/nLFtrvp7u7myJEj8iS6qKgovvKVr/D666+zZ88eTCYTt99+O/fccw+bN2/GZrPR19fHL3/5S9577z3q6urcbjNAdHQ02dnZ1NXVUVdXR2trq0fsuFpEUZQjEjMzMxw/fvwjc9XSpnNiYsKta/NNcdqiKB4Fjr7/363AmpvxuteCn58fISEhfOlLXyIrK4vk5GT5RHi53fHg4CDnzp3zeKhUOkHdeeedcpHZmTNneP311zlw4AD79++XK1w9jdVqJTo6mqeffprIyEgsFsuSXKQoirJj9GSxlMvlIiAggISEBDm36nA4qKmpWdIzvnr1arlA5q677iI9PV3u4Z6fn2fnzp386Ec/oqqqyutSKO5Gq9WSkZFBVlbWkvy03W5nbGyM3//+95SVlckDVz4YzRAEAavVuuR+7O/v5+zZs4yMjHjF9X0pUpGXN+RfJycnaWlpob6+HoPBQFBQENnZ2bS3tzM2NsYXvvAFubNEoVBQUlLCyZMnef311z0SLVIqlbKwS1RUFN/73veorq5e8hyNRkNaWhqzs7PMzMx4TKXtg0jXpsvlkusXpO981apVsuCRyWSirKyMl156ibq6OreuD58YRTS1Wk1QUBCFhYWsWLGCoKCgJc3wH0Rylp7GaDQSFRXFunXr5L7K1157jbGxMZxOJ3V1dddcWLZc+Pv7k5SURF5eHhaLZcmpQzoNBgcHk5yczMjICBMTE0xPT7t15J6kEBUSEkJERAQKhULWmZf0uwVBQK/XEx8fT3Z2tpxrjY2NxWq1Yrfb5RabvXv3Mjc3R3V19VWFVT+p+Pn5kZOTQ2RkpNxaODs7y/j4OHV1dZw6dYqqqqoPnZiVSiV+fn6YTCZCQ0Mxm82Iosj09DR9fX00NTV5rab6pdE5SWjIE0jh8ZGREWZnZ1EoFISEhJCRkYHNZuO2224jICAArVbLwMAA9fX1lJWV0dHR4ZHvVaFQkJSUJLco1tTUMDw8jEKhICgoiJCQEFnLYWZmhtHRUUpKSmRRGE8hialIHRqhoaFkZGQwMzNDbGwsq1evJjAwEFEUqa+vp6WlhRMnTjA6OurWzd0nxmn7+fkRFxdHREQEAQEBH+mwRVEkISGBT33qU9TW1nr0tB0fH8/DDz/MF77wBUZHRzl79iznz59naGjIK3b5lxIVFcX69evRarWXrQJWqVTs3LmTHTt2UF1dTWVlJfX19Rw8eNBtut5KpZKsrCxyc3PJzMwEFsOwg4ODcmucWq0mKSmJjIwMCgoK2LRpE/BHURMpJBYTE8Pjjz9OWloaP/7xj+nv7/d4ZMZTmM1mHnroIRISEmQVvPb2dqqrq3nuuecoKyu77N9Xr9cTFRVFXl4eeXl5REZGIooiTU1NVFRUUFJS4nUpKlg8BGg0GvmzOhwOWR/b3Ug1OrOzs7JEqdVq5c477+Suu+6Snzc7O8vevXs5dOgQpaWlHtsIqdVqNm3ahMVioaOjg97eXmZmZtDpdNxxxx088sgjrF27FrPZjMPhoKenh3/7t39j9+7dsjKdJ1CpVMTFxcnaE5/5zGeAxZN2WloaAA0NDfzd3/0dR44c8Vgx6CfGaU9MTHD8+HF+85vfUFxczF133XXF/J8gCPj5+clSm54kPj6eRx99FJPJRE1NDYcPH2ZqasorTtYfpKenh+PHj8t6xiEhIZjNZtmBS/9WqVSkpqYSGRnJpk2beOSRRzh+/Djl5eXs2bOH6enpZVmoVSoVBoOBsLAwjEYjLpeL8fFxLly4wPnz5+nq6sLPz4/o6GgefvhhNm7cSGJiohx+bG1tZe/evfJJ4bvf/a4sG+lJvKEtcXp6mnfeeUfuA56dneV///d/OX/+PE1NTVfM6W3atIm1a9eyc+dOIiIisNlsdHV18c///M80NjbS1dXldU5bkt2MiIggKioKhUJBd3c3ZWVl8n0pCAImk4mQkBB5cpz0eZYj1Cv1W0uFnR+8Jjo7O2lsbOTll1+msbHRo5K8Go2Gbdu2UVVVxbFjx+Tolr+/P4899hgNDQ0cOHCA2tpakpKSSEhI4Otf/zp2u51Tp05RW1vrEbtdLheTk5NotVp0Oh2pqalyLQn8sYe/ra3NoxGBT4zTdjgcjIyMcP78eWCxclGr1cpfuBQ2jYiIkNtSvCE8rlarMZvNKBQKeSNhtVpxuVxepxo0MTFBS0sLx44dIyYmhoiICKxWK2q1Wm7rsVgsGAwGDAYDRqMRURSJi4vD6XRiNBoZGBigqqpqWSredTodwcHBZGdnExoaitPppL29naqqKsrKyrDb7SQkJJCZmcmqVatk4YS2tjbOnz9PXV0dlZWVpKWlySEyaTzjn/pQg/n5eS5cuIDZbKatrY25uTlKS0tpbGy8bIeARqPB39+f/Px88vPz5Z7nkZERmpqauHDhAj09PV51jUvRIo1GQ0hICFarVd6wGY1GwsPDycjIkFuwrFYr4eHhhIaGEhcXh16vR6PRLIvTllosP9jP7nK5GBsbo7q6mtOnT9PY2Mjw8LDHIkJqtRqj0UhYWBilpaWy4ItWq8Xf3x+9Xk97ezsnT56ktraW3t5eJiYm2LlzJ8nJyQwPD3vMac/NzXHixAm5XkdS+DMajcCidHN/f/8S1UhP8Ilx2hInTpygvr6e+vp6TCaT3NKlUqlYvXo1jz32mPxH8AYmJydpbGwkKyuLzMxMQkJC6OrqorKy8rrFTpYLSW9cWrytVqtcXBQQEMD27dspKioiMzNTHtYi5QXXrFlDZmYmKSkp/OAHP2B4ePimhxqDg4PJzc3liSeewGQyMTMzw759+3j33XcpLS0lKCiIrVu3sn37djZs2MDCwgLDw8McP36c559/nqamJmJiYrj33ntZs2YNs7OzNDY2UldX59H2r0tbFT2VU5+bm+PkyZOcPHnyqp5vsVgoLCzkwQcflAvXnE4nnZ2d7Nmzh76+Pq9y2PBHhxMYGEhGRoZclAhQWFhIdHQ01dXVhIeHExYWhsViwWq1YjAYmJmZISkpiXPnznHq1Kmb+neS+t/T0tKWaOfD4ndaVlbG888/z+uvv37T3vN6sVgsxMTEyJsLKTIRHBxMREQE5eXlnDp1itLSUgC5OLSgoIC0tDRMJpPHPsfg4CBf/epXSUpKIiwsjNDQUL7zne+Qn58v23r+/PklXRGe4BPntGFRXOXMmTMoFIolodve3l78/Px46KGH8Pf3Jzk5mYiICOx2u9v6iRMSEnA6nfT09OBwODh//jx//ud/zgMPPEBeXh6FhYV8//vf58033+S5556jq6vLK09409PTcnGXQqFApVJRWVlJfn4+q1at4s477yQqKoqAgAD5d7RaLXl5edx7772YTCZ27959U22yWq1yxTgsng5/85vf0NfXh7+/P8888wyrVq2S+21ffPFFDh8+zJkzZ9Dr9WzcuJEf/vCHREREMDc3x1tvvcVzzz1HQ0PDTbXzepmbm6OlpcVj4xWvhYyMDP75n/+ZqKgoWU7zueee4+TJkxw4cMDtDlvqpzUajcTHxxMSEkJMTAyZmZlyjYafnx9Wq5WwsDAMBgMWi0X+favVKgucSKIwUmtjf38/ExMT9PT0MDQ0dNM3VgUFBTzwwANs3759yf0EeN0Map1Oh9VqlUWWuru7cblcDA8PMz8/z+Dg4IcGEE1MTPDaa6/x/e9/n+TkZA9Z/ke6uroYHR1lfn5+yb22f/9+Dh065EHLFvlEOm2Hw8HExMSSxxQKBQsLC/j7+8shc51Oh0ajWXaBFbVajdVqJS0tjeTkZCYnJykpKZFDQzU1NXKoef369SQlJREVFSUPV/BGpy0Vx1wahhsfH0elUjE9PY1KpaK4uJi0tDQsFot8WjCZTERGRhIVFXXTbfLz85Nz7NPT07L4hMvlwmq1kp2dLadNpArb8vJyhoeH5dxrZmamrPl97NgxmpqavEZkRZr97W054A8SGxtLamoqycnJskMRRZG6ujoaGxvdJgYkVQNHR0cTGhqKv78/gYGB8gCIiIgIUlNTUavVuFwuFAoF/v7+BAcHyxEiSbFN6u/v7OzE5XJhs9k4ffo0bW1tDAwMMDMzw8TExE1rXbtUTCc3N5fi4mJCQkIQRZHx8XGmpqbkcZzuWMOuFqVSKZ+ypR5mQO6AmZmZ+dB3JMnLAvJ4UU9is9nk9Kl08BNF8YoaD+7mE+m0L4dKpSIxMZGHHnoIWFRIcjgc8oCL5cRsNrN69Wp++MMfEhoaSmNjI2q1mt27dzMxMcHU1BSHDx8mOjoaWHQ+er0eg8HgFUVI14I0rOXgwYP8xV/8BZ/+9KfJz89f8jmsVuuyTKxSq9Xo9XoEQaC7u5vy8nK5Zzs+Pp7Q0FBZP/r555/n+PHjdHR0EBsby0MPPcTGjRuBxR31kSNHPDJ276OQhsJ485ATgDvvvJNNmzYtCelLFeM9Pe5TNVar1cTGxvKFL3yBdevWERUVtWSzKN33drudubk5ZmdnZcUxlUqFKIo4HA6OHDnCqVOnOHfuHDU1NW7ZREv1LTt37mTz5s0UFBQgiiIDAwN0dnbS0NBAVlYWOTk5SwayeBNSO5coitjt9ituNp1OJyMjI17VmaFWq+WhIJLT9hb+ZJz2ww8/zNatW+X8YEdHB++++65bpDb/9m//lqKiIuLi4ti1axdnzpxhz549SypB7733XvnGrKyspKqqitbWVq9r+7panE4nvb29tLS0yDkhiaGhoWUXU/jgTSaKorxBU6lUJCcnU1RUhNlsxmg0olKpKC8v5yc/+Qk9PT1LRFh8XB1SIVdcXJzc2iWKIg0NDbz55pvU1ta69aTyzW9+k+LiYoqLi2Up2xMnTjA4OEhvby+1tbXY7Xamp6cZHR1lcHCQ7du3881vfpPIyEjm5ubo6+vjJz/5Cd3d3UxMTLgt6hUcHMzq1av58pe/THR0NE6nkxdeeIFz585RVlZGXl4eAQEBZGZm0t7e7pZ2yqtFilJc7YFDp9OxatUqtFqt16R+JHU0qRi1rq5O1s7wNLec0xYEAbVaTXJyMnq9noGBAQYHB68YlpKqsvPy8khJSZEvpNHRUaqrq5mdnXXbgA1JtS01NZXbb78dWAx52mw2NmzYQHx8PNPT05SWltLU1MTU1JTHd3hSmCg6OhqXyyUvZFfznalUqg8ppomiyNjY2LKMspNC4i6XS+4UCAwMJCAggICAAHmgiVqtJjU1laioKCwWC6IoUlZWRlVVFeXl5czPz3vlZklqP/HG6IsUio6IiJBnaouiSF9fHw0NDZw5c4aJiQm3hvbT09PJysrCYDBQXl4uD9wZHR1laGiIixcvysI709PThISEoFKp5IlgbW1tlJSUyE7RXQt2QEAASUlJ3HbbbURHRzMzM0NFRQXHjx+noaGBvr4+IiIi5Gt3bGzMqxTlLk2JfByS0FFOTg5zc3MfSmt6AikvHxsbi1qtZnx8nKqqKgYHB71iU3HLOW2FQoHZbOYzn/kMMTExHDx4kMOHD1/x5CaFmdatW0d6errsOAYGBigtLXVLSObs2bNoNBqSkpK45557uPfee+WfTU1NMTAwQGxsLPPz8/T19fH6669TX1/vFTei9P3dc889LCws0N3dzZ49ez42RCuNb/xg7loqwluOCsy+vj4qKipwOBxERERgMBhIT0/HYrGQmJiITqeTNxLr16+X7RkbG+P06dMcPnzYq0/Y0obDG522RqMhNDSUTZs2kZeXR2xsLKIoUl5eLledu7v4LDExkejoaPr6+vif//kfzp8/f1n9ayl//JWvfIXbbrsNq9Uqn8p/9atfudVhw+IEwg0bNvDYY49hMBg4ceIEP/3pTzl9+jRqtZrQ0FB27txJbGwsLpeLoaEhr6nEl9bXSztHPgqlUonFYuH222+nv7/fYzrplxIYGEhSUhJbtmxBr9fT29vL/v37aW1t9YqIxi3ntIODg/nLv/xL7rjjDoKDg0lKSgIWy/HHxsYYGBiQc1nh4eGkpqZyxx13kJiYiJ+fH6Io8vrrr3P48GGGhobccsp+5513aGhoYGxsjISEBCIiImSFHZfLhclk4vz585SUlPDOO+9QUVHhFcNBrFYrmzZt4stf/jK5ubm8+uqr7Nq16yNPS3q9nqCgIB544AF27tzJihUr5Bt3YWFBVtGqr6+/6fa6XC7sdjvz8/PodDpMJhO/+tWv5OIYqR9ewmazMTY2xqFDhzhy5AgVFRU33aabidlsZv369bz33nteMw9c6h/+8pe/TGFhIRs2bCAgIICLFy+ya9cudu3aRXd3tzwVyZ1IOempqSna2toum09Xq9UUFBTw+c9/nu3bt+Pv74/T6eSHP/yh3IfuToctCAIrV64kPT0dg8FAbW0tZ8+e5cSJE9jtdnl2/aUT1m5mAdyNMjY2RmtrK4ODg8TGxrJt2zbeeuuty36HKpWKT3/60xQVFREZGcmLL77IqVOnPGD1IoIgYDAY+PrXv84dd9xBUlISe/bs4fTp0+zfv99rNka3nNOWTkbT09MEBweTkJDAli1bSExMZHBwkM7OTvz8/EhNTSU0NFSeNKPT6XA4HExPT3P69GlZ09sd4efJyUk6OjrkauSgoKAPOYiuri4aGhqor69nenra47kTQRBYsWIF+fn5ZGdnExISgsFgQKFQyC080ncnnQBTUlJISEggPj6ejRs3EhcXJ4tTOJ1OpqamOHXqFL29vctSTDU3N8fg4CAnT54kMTGR8PBwoqKiluz2h4aGGBsbY3JykvHxcQYGBjh06BCdnZ1eEfq6FEm6cnx8HKPRiFKplP/tLUgdAdnZ2eTm5hIaGsrs7Czd3d0cP36ctrY2j4xnhUVZT5vNhr+/PxkZGfJYRWnM5fz8PGlpaaxevZrVq1ejVqvp6OigoaFB7sf1RNGfTqfDz88PhUIhF8jNzc3Ja9ltt92GVqtlfHyc4eFh6urqvKKqGRYrxkdHR6mqqsJkMrFhwwYaGxvp7+9nampKFl4ymUykp6dTUFBAaGioV3RqaLVa8vPz5UEnQ0NDVFZWUlZW5rFr+HLcck57enqagwcPEh0djVarJTMzk8ceewybzUZ/fz8dHR34+fmRmZmJXq9f4mAmJibo6Ohgz549tLe3uzVvOTw8zHvvvee297tRBEHgnnvuoaioSA5xSxKhs7OzzM/Pyydu6RR73333sXXrVvLy8jAYDEteTxoX+Nvf/nbZxAmkEXm//OUvWb9+PevWrcNsNi9xcg0NDdTU1NDc3ExHR4csT+mNzM7OMjY2Rm9vLzExMYB3SJpeijSoJzs7m9TUVGDxWm9sbOTAgQMetU2S/YyOjubTn/40k5OTmEwmmpqaGBkZYWhoiC984QvExsYSEhLChQsXOHTokDy5yVN1DZfmhEVRRKFQoNPpWLlyJXfccQcPPfQQY2NjNDc3c/78eQ4dOuQ1J22bzcbw8DDvvvsu999/Pw8++CDt7e2cOXOG1tZWYmJiiIqKIjExkccffxy73c7Fixf5l3/5F49PezObzTz88MNkZGSgVCo5c+YMBw8epLy83GM2XY5bzmnPz89TWVlJZWUls7OzvPbaazz22GPEx8cTGRlJaGgogiDIQv8S09PT7Nmzh7/+679mYGDA63tdPY1UxR4WFsbq1asRBIG7776boqIi6urqZMcNyKIVq1atwmAwyGE7qc91bm6O5557jgMHDlBSUrKspxeHwyEPrv/Zz36GVqtd8nObzcbCwoLcguLpiMZH4XQ6WVhYkCVUTSYTRUVFHDhwwK2tU1ciLCyMhIQE7r//fsLCwuSajL/7u7+T5YQ9yXe/+11SUlJ4+OGHuf322+XJf4WFhfLozdraWt566y3Onz/PqVOn5CieJwsR/f395U3v+Pg4BQUF8j8Wi4WFhQV+/OMfU1ZWRkNDg9e1AC4sLLBr1y76+/u57bbbePLJJ3nyySdxuVyyPPDU1BT79u1j79691NbWygWkniIpKYkVK1awc+dO9Ho9/f397Nu3j4GBAY/ZdCVuOactleJXVVXR19fH5OQkKSkpsmCDv78//v7+REREAH/Mcx45coSTJ09+SI3Hx5Xp6OiQB2nExMRgNBrl6IXNZlty0vbz8yMgIACFQiGfELq7u+nv76e1tVXWGnZHCHpubs7rQt3XiySoEhMTg0qlwmw2o1J5x22bmppKXl4eBQUFmM1mpqamqKyspKGhgd7eXk+bJ6uTHThwgImJCfz9/T8UqZCGfDQ1NdHZ2ekVm/mZmRn5+o2KikIURXQ6HYGBgYyNjdHR0UFFRQWtra1eExa/FFEUmZqaoqGhQdYdl4RKAFmB8ty5c9TX1191N8pyIOWx8/Pz2bBhAyaTibq6OqqqqqipqfGqsLiEd9z918GZM2fkXKqkcgSLO6bExERZ7s9utzMzM8Pzzz9PZWWlBy2+tRBFkZaWFgIDA4mPjycwMFAWewkJCVky/UZy0tLMYUlMobKyknPnznHw4EEuXrzolQuMtyO1HqWlpWE2m72melzSk9+yZQtr167F5XLR29vLoUOH3KJ9cLUMDw/z9ttv8/bbb3valKtmYGCA0dFRnE6nPF5Wau2qra3l6NGjXutQLqW1tZXW1lb27dvnaVOuiFKpJCQkhDvuuIN7770XURQ5fPgw+/fvp6qqytPmXZZb1mnD4oW8sLDA7t27lwwGUalUSxSCRFFkdHTUK3bRtxJSiOjs2bMcOnSIxMRE4uLi6OrqIiYmRq7clwaJALS1tdHV1UVnZycXL16UFby8se/5VmBoaIif/exn6HQ6wsPD2bdvn8fb0vz8/IiNjSUwMFCWnTxy5AilpaWcOXOG6elpj9p3KyNFBnp7e+nt7eWxxx5jYWGBrq4ujh8/zunTpzl+/DgzMzNeKW98K6FSqQgICODxxx8nNzcXQRA4evQoBw4c4PTp054274rc0k5b4pMSCvU2nE4ns7OzLCwscP78eTo6OggKCmJkZITAwEBCQ0OBxXCe1KI2NDTEyMgIo6OjjI2NeUXr2q2M3W6nu7ub3bt34+/vT319PZOTkx61SavVkp2dTUpKCqGhoXJYvKysjL6+Pq+So7wVkUbgCoLA/Py8LPPZ1NTk0Wr8TxpSS3BRURFarZaOjg7eeustj8/L/jg+EU7bx/Ihhbpramo8bcqfJFKL46uvvuppU2T0ej15eXlkZmYSERFBV1cXJSUllJaWum0YyCeZ+fl5Ojs76ezs9IqpUp9EFAoFiYmJFBcXs2bNGtra2rhw4QIvvviiVzts8DltHz58XCPT09OcOHGCzZs3I4oizzzzDGfOnPHKSlsfPj6ISqUiMzOT+++/nw0bNvCLX/yCEydOUFNTw/z8vMeloz+OG3LagiBYgGeBLEAEHgcagZeBOKAd+IwoimM38j4+fPjwHmw2G83NzbzwwgtotVrKy8vdOkzDh48bQZrvffToUTo6Oqiurqa1tZWBgQGvd9gAwo0YKQjCb4AToig+KwiCBtADfwOMiqL4j4IgfA+wiqL43Y95He//pnz48OHDh4+bS5koivkf/7Q/ct1OWxAEf6ASSBAveRFBEBqBjaIo9gmCEA4cFUUx9WNey+e0ffjw4cPHnxrX7LQVH/+UKxIPDAG/FgShQhCEZwVBMAChoihKI7f6gdDL/bIgCH8mCMJ5QRA8L53kw4cPHz583ALciNNWAXnAz0RRXAnMAN+79Anvn8Ave4oWRfGXoijmX+suw4cPHz58+PhT5UacdjfQLYriuff//1UWnfjA+2Fx3v+358a2+PDhw4cPH58grttpi6LYD3QJgiDlq7cAdcDbwGPvP/YY8NYNWejDhw8fPnz4AG68T/vbwO/frxxvBb7E4kbgFUEQvgx0AJ+5wffw4cOHDx8+fHCDLV83zQhf9bgPHz58+PjT45qrx32KaD583CIEBwdz5513EhYWxujoKPv37/dpffvw8SfGn6TTViqVaLVa9Ho94+PjOBwOr1bCUSqVqNVqgoKCEAQBURQZHh7GbrfjdDo9bZ4PN2G1WrnzzjvJyMigt7eXnp4eJiYmvMZpKxQK/Pz85LGtKpUKg8EALA4+mZ2dxeFw4HQ6fRP3fPi4Tv4knXZUVBQbN27kwQcf5Dvf+Q7d3d1ePTknIiKCrKwsfvOb32AymZiYmODzn/88tbW19Pb2eto8H25Co9EQGRlJZGQksbGxREVF8bnPfY4LFy54xeYtICCAdevWodFosFqtJCUl8cgjj+ByuaitreXNN9+ks7OT3t5e6urqfI7bh4/rwOud9qXzsRcWFnA4HDd0swuCQFpaGtnZ2WRmZrJu3TrOnz9PVVWV12knKxQKVq1axYoVKygoKMBgMKBWqzEajTz44IMkJCRQW1vL2bNnffOqbwJms5ni4mKKiorw8/Pj7NmzHD9+nOHhYU+bhtFoJDg4mLi4OHQ6HUqlkoiICPz8/FAoFB5x2kqlEj8/PwIDA1m3bh2pqamsWbMGjUaDn58f/v7+WK1WALKysjAajYyMjNDf309ZWRm1tbVUVlYyPz/vdfeeNyFFLVJSUkhNTSUrK4uIiAgUCsWSCOHc3ByvvfYac3Nz8ho5NTXF1NQUg4M3t/M2MDCQ2NhYioqKCA8PJzg4GEEQAJbY5HA4qKuro7u7m/7+fpqbm+VojBQt9P3trw2vd9pmsxl/f39CQ0MZHh5mfHz8hhdRaRZ0eHg4q1evZmxsjPr6eubn52+S1TeO5Jxzc3MpLCyksLAQpVIp/6y4uBij0YjZbKaiogJRFD1+2lKpVOh0OnQ6HSaTCVi8gWdmZpiYmPCq7/eDCIKA2Wxmw4YNfPrTn0av12MymWhtbWV2dpbZ2VmP2qfX6/H39yc4OBilUonL5UKpVMoLpbu4dBNtNBqxWCwkJCSwfft2srKySE9PR61Wo1As7SaNiIggIiKCmZkZRkZGCAkJwWq1Mjc3R29vL7Ozs9hsNq8J9cPiNaFSqQgPD2dmZobJyclljQ5I72c2m+V7HRa/c61Wy+rVqyksLGT9+vUkJSWhVCqXOMjp6WmGhoaYnJyUv8eBgQEGBgYYHx/HbrfflDSgTqcjOjqawsJC7rvvPhITE4mJibms015YWODUqVM0NjZy8eJFFAoFer0eQRDo6upifn5ePnBMTk4yPz/vVdeAN+L1Tvu2225jy5YtPPHEE7z88sscPHiQ3//+9zf0ms3NzeTm5qJUKvniF7+IIAiUlZXR0dHhNbu+pKQk1q5dy7333ktKSgrx8fHyz5RKJcnJySQlJVFUVMQbb7xBf38/09PTHrNXoVAQHh7Ohg0buO2223jkkUfQaDTMzs6yZ88efvrTn3L69GmP2fdx+Pn5ERkZyaOPPkpAQAAajYbHHnuMsbExjh07xr59+zxqn06nw2g0olIt3rJjY2OcPn2akZERt0ZZJOe7cuVKVq5cSXJyMkVFRSgUiqvaQOj1evR6PVFRUdx99908/fTT/OQnP6GyspILFy7Q1NTk8c2nhNFoJCIigl27dvHuu+/ywgsv0NTUtGxrhMFgIDIykscff5yAgAD5cWkzvGPHDtnhAR9ywAaDgaeffnrJYz09PdTV1fHUU0/R09Nzw2uEIAgUFxdz991389hjj2EwGOQN2uU2BGq1mg0bNrBhwwbsdjttbW1oNBoEQWBsbGzJSfvXv/41586do6am5oZs/KTj9U5bEASUSiUqlYqCggL0ej0NDQ20tLQwMTFxXa9pNBrRarUIgiAXzUj/7Sm0Wi1ms5mQkBAefvhh4uLiiI6OJi4uDrPZjCiK8rxipVJJUFAQCoXiqhfL5SQuLo7U1FQ++9nPYjabUalUvPjiixgMBvz9/Vm7di0HDx6kubmZoaEhj9p6ORQKBbm5uRQVFWG1WlGr1TidTqampuQcrKftKywspLi4GFi8J8bHxzl16hRTU1NuLaKMjIwkLy+Pz3/+8wQGBmIymeQT/8LCApOTk4iieFmbpGtDpVLJ16yfnx/33HMP/v7+zM/Pc/HixWV32oGBgSiVSpxOJ+Pj41d8v+zsbO688078/PxwOBzLPms5LS2Nb33rW+Tl5aHT6eTHpTVQq9V+KIJxKdLzLiUoKIicnBx++MMfUl5eTm1tLe++++4N2WkymTCZTOj1+o9dfy5dV9VqNZGRkfJnsFgsSzZADz/8MKmpqfz85z+nr6+Pubm5G7Lzk4rXO+1LiY2NRaVSsWbNGqamppibm7vmUIpCoSA0NBSLxYIoigiCgEajwWQyecT5Sbvo8PBwwsPDSUxM5P777yckJAS9Xo9arUYQBBwOBx0dHTidTjQaDYGBgW639YMoFAosFgvp6emsXbuWtWvXMjAwQHt7OwcOHMBqtRIXF8f27dsJDAxEr9d72uQPoVAoMBgMZGVlsXLlSnkzNzs7S1tbG93d3R7NaavVakJCQlixYgVZWVny4/Pz87S3t2Oz2dxqj9lsJiwsjNTUVPR6vez8pqammJiYoK2t7bLdGIIgEBcXR0BAAAEBAfK9plAoSE9Pp7u7e0ledDlQKpXo9XrS09NRKBSMjIwwNTV1Waet1WpJSEhg3bp1cmh8uZ1IQEAA+fn5JCQk4OfnJz/ucrlwuVyMjY2hVquXrAsfhSiK6PV6tFotd955JxaLBb1ez4EDB647lywIAlFRUQQFBclRn6tFoVDIaTNA7iyQyMvLw2QyUVpaSn19PUNDQwwNDXlN9BMWP79arV7ymMPhcKuNt5TT1mg0xMbG8qMf/QilUsnBgwdpaGi46t9XqVTo9XoeeOABcnNz5Ys+NjaWrVu30tzc7PaCrrCwMPLz83nwwQfJyMggIyNDzlV9MDf04osvMjc3h8lkIicn50O7anciObsvfOEL7Nixg1WrVvHcc8+xf/9+SktLmZycJDo6mtzcXERRxGazeeXO2c/Pj9zcXB599FHWr18vXxPd3d386Ec/4syZM4yOjnrMvpCQEL7zne+wc+dOYmNjPR5VgcXc6eHDhyksLCQgIIDR0VH27NnD2bNneeWVV5ibm/uQI1QoFDz66KNs27aNRx55xCN2BwUFsXbtWn74wx8yNTXFK6+8Qm9v74c2/gqFgoyMDPLz88nLy+PnP/85ZWVlcqRruejv72fv3r089thjS5y23W5namqKn/70p8THx1NUVER8fPzH3v/StSIIAjqdjtTUVGw2G0lJSXR0dFxXqFyhUPCtb32LhISEa/7dj8Pf359Vq1bxwgsvUF5ezunTp3nmmWeuuLFyNwqFQj5gXUp/f/+S3Pxy4/VOe3JykuHhYURRlMPBBoOBjRs3olKp5GKGq/mj6nQ6IiIiSEpKIjQ0VD5pAx5ZDFetWkVRUREPP/wwUVFRcqhRqgq91GlL/dnSY5eG9j1BbGwsOTk5PPzwwwwPD/O73/2OF198kZ6eHmZmZhBFkaioKFJTU5mcnGRsbOy60xnLhclkIjY2li996UvExcXJ10BbWxvV1dVUV1czMzPjMfuys7PJy8tj69atco++p6mvr2dwcJCSkhLeeecd9Ho9s7Oz1NfXMzAwwOzs7GXvRafTSUdHx4ccnyiKLCwsMDs7K4fWl4uAgADWr19PR0cHdXV17N69+7KOSxAEVq5cSUxMDADnz5+nq6tr2eySGBgY4MCBAzQ0NCw5zTmdTmw2G+fPnycgIICjR49y33330dHRQUdHB0888QQBAQEYDIYlOe9LPw8spgUDAgLkNMX1IIoiu3fvJjc3l+TkZGCxvqK/v58zZ85gt9tRKpXExMSQmppKbGwssBihMRgMH5mGlB6XKuXHx8cJDAy86vVdeo2cnBxGR0ev+2+m0WgwGAykpqaSk5MjF38qlUrMZjNpaWlLnv/2229TVVXltpodr3faY2NjcrtAQEAAOp0OtVpNWloaMzMzhISE0NfXd1V/VEn8Qa/Xy20HADabjfHxcbeFONRqNSaTiVWrVrF27VoKCgoQRRGXy8XCwoKc87u04ET6uTeIwCiVSqKjoykoKCA6OpqGhgaOHTtGTU2NvNuU8leJiYksLCwwPz/v9lDuxxEdHU1OTo58YpS+4/b2dhoaGhgYGPBIJauUdlixYgWFhYUkJCQgiiLz8/NLrltPMDQ0xNjYGC6Xi8bGRlQqFXa7nZGRkSt+VzqdjoCAADk8eymiKNLT00NPTw+Dg4PLen0bjUbS09MZGhqivb2dlpaWyz5PoVAQFRUlF4P19fUxOTm5bHZJ2Gw2BgYGqK+vX7KeSZ0hQ0ND6HQ62tvbsVqtNDU10dzcTF5eHikpKURHR182BSWtHxMTE4yMjNxQFbnL5eLMmTPyRhxgeHiYrq4uDh48yMLCAkqlkqSkJIaGhuRNWnR0NMHBwfJm4YN1ORLSuhcQEEBYWBgREREMDg5edeeJIAjo9fpriiJI1fkajQaNRoPFYiEoKIjCwkIKCgoIDQ3F6XTKUc4VK1ag1WrliOjExAR2u93ntCWqqqro6uoiJCSET3/602RnZwOQmpqKVqtl8+bNvPfee1dVLLSwsMDo6Ci9vb2YzWaCg4MB6Ozs5PDhw24TewgNDWXjxo18+9vfJjo6WnbWMzMzTE1NYTAY0Ol0cs5HKoK59BQjOXBP5HtMJhNFRUU89thjnD17lnfeeYd33nlHdthKpZKwsDCKiorYtm2bV5wQL8fnPvc5HnjgARITE4HF73RmZoZ9+/Zx9OhRj20yTCYT99xzD1/96lfJy8tDrVbLBVopKSkesUni0hTH1UQhVCoV6enpPPLII9xxxx2EhYUt+bnD4eDZZ5/l2LFjnD179qbbeylms5nc3Fx27979sWJK0slVwh3XsNlsJj09nd27d19xkzAzM8PMzAz/+Z//CSxuMJ5++mk++9nPcv/99xMUFPShsLmUntq1axf79+/n/Pnz122jKIrs2rWLXbt2feTzzp8/z0svvST//7p168jJyZEL7AICAvizP/szjEYjGo3msq9htVrZsmUL7e3tVy1+5XK5qKiouOqTuUKhwGq1kpmZSVxcHHFxcWRkZBAfH09eXh4As7OzNDY2Ul1dLdcUpaWlyfn5e++9F5fLxbPPPntV73mjeL3TdrlcTE1N8c4775CWlkZoaKhcsGKxWLj77rupq6tjfHz8Y3tppV1UaGgo/v7+bvoES8nOzqaoqIivfvWrREVFodVqAWhtbeX48eO8/vrrrF+/nqKiIjZt2gRAdXU1x48f58SJE8TExBAUFOQR2wVBQKvV8oMf/IDMzEyGhob41a9+RV1d3ZKbxGQy8Rd/8ResX78eo9HI4cOH6e/v94jNH0VQUBARERHA4mebmJhg3759nD17lqampmV9b6nXOTo6Wj6FSlX40dHRxMfHExMTw/j4OG+88QaVlZUEBgbyox/9aFntulmYTCbi4+PZvHkzOTk5rF+/nqCgIPz8/OS01MzMDMPDw5w7d46Ojo5ltUfSe5AcxEedNBUKBWvWrCE6OlruHXdHTnVwcJBjx45dcR3TaDRotVr8/f352te+JofCAwICSEtLIyEhYcmpVRRF7Ha7nL7avXs39fX1y/45LkdtbS2dnZ3yhkISL9LpdPKBLCkpSW63FAQBo9FIXl7eEud/NSwsLFx1JMFisfDjH/+YhIQEgoOD0ev1GI1GdDodgiDQ0NBAbW0tzz//PF1dXRiNRiYmJggNDZWLl0tLS93apub1ThsW/wgXL16kp6eHkZER+YSs1WpJT08nMDAQnU73oYtdEARZmQkWF+mUlBTMZvMVd3fLhVqtxmKxkJeXR0FBATk5OTidThwOB1NTU9TW1lJSUsLRo0dltaHp6WlGR0e5cOECJ0+epLe3F51Ox+joKNPT027PZ2u1WoKDg1m/fj1Op5PGxkYqKysZHR2VbxK9Xk9YWBjFxcVERUUBUFZW5tFirg+iUqmIiIggMDAQg8GAKIrMzs4yMDDA6dOn6ejoWPb8u0ajwWw2k5eXR0REBMHBwSQmJpKenk5oaCgOh4PBwUHa29s5cuQIzc3NckTA27FYLMTExFBcXMymTZtITk5eojMgiiJ9fX1ymLqzs3PZv28p7OpyuZa0Sl5ucRcEgdDQUHQ6HWNjY8zOzmK326/4/JvF/Pz8FSOGSqVSdiwRERHcfvvtciRAr9djsVgwm83y86WQel9fH83NzRw9epTm5mZGRkaWzf6PYmxsTA6nw+J33NnZKUvzZmVlERISsuQwJYridc2FuJboo5+fn5zmM5lMzM/PMzo6ytjYGG1tbVRUVHDhwgVOnz6NRqNZkrp0uVzY7XZZX8Bd3BJOW1LVam1tpampSS4E8PPzIzExkcjISKxW64cuSKkgYtu2bcBi8VRBQYFHTtnBwcFs27aNp556Sq68tNvtDA0N0dTUxM9+9jOam5vl54+Pj9PQ0MDrr7/O6dOnOXv2LE6nk9bWVtnJp6amuvUzREdHs3btWuLj43n33Xf5xS9+wfDw8JKqybS0NNauXSu3J7W1tfGHP/zBY4vF5TCbzXz7298mPT1dfqylpYUzZ87w3HPPuSWPbTQaiY+P53vf+x5RUVHyNTk3N8fIyAh79+7lrbfeoqamht7eXoKCgpY4Pm+muLiYLVu28M1vfvOyFc4ul4tXXnmFiooKKisr6ezsXPbKWymfOzU1Jecw1Wr1ZfO70mZ/bm6Orq4uenp6mJ2dRa1We6TGQaVSYTQaefLJJykoKCA3N/djf0daM9944w327t3LwYMHvaIeRkLaKM/NzaFUKikrK2PFihVLDiKDg4O8/PLLy95yKUUkpqenaWho4I033qC8vJyWlhaGh4eZnZ1FFEUefPBBNmzYwBe/+EX8/PyYn5+nv7+fXbt2ceHChWW18VJuCact0d3dzcWLF+X/l3a+MTExxMXF0dLSQlxcHKGhocTExMhtDvfffz+AHFq6NETW29vL4OAg09PTy5ofDg8P53Of+xwRERFyZeju3bspKSnh0KFDtLe3yxdHQ0MDvb297N69m+bm5iUCEHa7nfn5ebnP0p354piYGDZv3szw8DAXL15cUjAjCAL+/v5s2bKFBx98ELVaTWlpKWfOnPnIIiV3s2nTJjn1EBoaysLCAl1dXbzwwgscP36chYUFt9QJjI2NUVtby9e//nXCwsKWOO3p6WlZiGZ6ehqn0/mhBVdqP/GWeoHw8HDCwsLIzMzkc5/7HCkpKbLDlu5TaZPa3NzMgQMH5OiZu9p5RkdHOX78OGvWrCEuLo6wsDBeffVVent7L3vSNxgMxMbG8sMf/pCysjIqKys9MqNAKlwtLi6+6ra/qakpfvGLX7B3716qq6u9ymFfipQi+j//5/8QGRkpr40dHR3U1tZSUVGxrB0cU1NT/PKXv2R+fp6pqSna2toYGhpaogNiMBjIzs7mwQcfJD8/Hz8/P8bHx6mqquJ//ud/qKurc+vAqVvKaff29tLS0sLg4CAWi0WupE1JSWFmZoaFhQUSEhIICQkhLi4OPz8/YmNjP7Z4R6PR4O/vz8jIyLIsIDqdjsDAQJKSkuSQW21tLcePH5dPG5ciVV0uLCwwMTGx5IaTwl5SIZo7Q+QBAQGkpqbS29tLX1+fbJt0MlmxYgXZ2dkkJCQwMDBATU0NZWVl2Gw2jy8agiBgMBhIT0+X85V6vZ6FhQWam5uprq6msbHRbQuy3W5nYmKCkpISLBYLRqMRWKwglkJ0lyKlUux2OyqVCo1GQ0hIyIeEHtyNNDQkMzOTlJQU8vLy5DYZ+GPlsiiKdHV10dTURElJCS0tLQwMDLi1pW5iYoKysjI5JbFhwwaGhoZoa2ujs7MTURRRKpWyXKwkuhQVFUVzc7PHrmGptVPqnLkapHVCup68EbVaTWxsLFlZWaSmpi4ZgNLd3U1rayuDg4PLWiC8sLBAaWkpExMTTE5OfqglMSgoiMjISG677TYyMzMJDw/HbrdTW1vLuXPnOHPmDGNjY27V97ilnHZlZSXDw8OsW7eODRs2EBkZCcD9998vn6YlLm2ZutJCLAiCnE9Zu3YtXV1dy+K0IyMjZSehUqk4deoUDzzwwBWffzVFW9Lnc+dJKzQ0lBUrVvAf//EfNDc3y9+VRqMhODiY//f//h9paWmo1WpeffVVXn31Vc6ePetxhw2LIca0tDTy8/MpKCggMDAQQRAYHR3l3Xff5eLFix7RbpdOnx8n72qz2ZienmZkZITAwED8/f0pKirirbfecpOlH0ZSGIuKiuJrX/saa9asITIyUv57SxsNyWm/+uqrHtVx7+np4fe//z1JSUkUFhayadMmNm/eTHt7OyUlJSwsLGAymQgJCcFisaBWq/Hz86O0tJRz5859aHPtLiTn29LSgkajkWtFPsila4HJZOLxxx9nYmKCiYkJjxWgXQmlUonFYuHhhx9my5YtS+oFXC4XJSUlnDp1ivHx8WW1Y2Fh4SOr6SVf8+UvfxmdTofdbmdwcJD//M//pKSkxCMFtreU04bF3fLLL78s57KvxLU4ioCAANLT02+6wphWqyU8PJxvf/vbrFmzRq42vBEndmk+brkLYyQEQSArK4uYmBicTifnzp2jvb0dpVJJbm4uW7ZsYcOGDaxcuRK73c7Fixf51a9+RUdHh8enY0lotVp27NhBRkYGFotFHg3Y0tLCiRMnvCrnfjlmZ2cZGxujr68Po9HI1NQUFRUVHv1+N23aRH5+Pg888ACxsbEYjUZEUaSlpYXOzk4qKys5dOiQHHqWojOeQpJb/clPfsJ7773H2rVrWbNmDUajUW69lCbUCYLA8ePHeeutt3j77bc9qpkvtar+27/9GzExMUuctk6n41vf+taHUiVSK9PKlSsZHR2loaHBKzbPCoWC7Oxs0tLS2Lp1K2vWrJFFbJxOJ8PDw+zdu5fXX3/doxsNvV7PihUreOSRRygsLJQjpC0tLbz44otUVFR4TN74lnPadrudnp6ey4Z8LlU4k8QopDDypRNlYmJiloSZNBoNRqPxpoeajUYjq1atIi8vj6SkpOtWIboUs9lMREQEVqsVPz8/t/USa7VaVCoVoigSERGBy+UiISGBlStXUlxcTF5eHhaLhZaWFpqamuSTqzfoBlutVmJiYsjLyyM0NFT+2zc2NnL27Fl6e3u9UmL1UpxOp1zPIFWtLnfo8HJIldc6nY6MjAxWr17NypUrEQQBu93O6Ogo58+fp6GhgYqKCk6fPu3WfN9HIVUjS8WcCwsLTE1NERAQINcUWCwWwsLCiImJobe3l3PnztHd3e3RmgxJNa6mpoa+vr4ldT1arVZWHgsNDSUsLEyujNdoNCQmJjI8PIzVamVqasrt18ulSDPg8/PzWb16NWvXriUyMlKuep+enqa7u5sjR47Q2tq6pNrcnZhMJiIiIti4cSMZGRmEhYWxsLBAdXU1FRUVnD171qN1OjfkRQRB+AvgCUAEqoEvAeHAS0AgUAZ8XhTFZf900i7y0t3k0NCQnE89fvw4o6OjzM/P881vfnPJ6LvlIjw8nK9//eukp6fLi8KVJiBdLbGxsRQXF5Oamoqfn5/bwjOTk5Ny+PjJJ5+UT/wBAQHyQANBEKitrWX//v1eoxcMi73xGzdu5I477kCj0cjf/8svv8wrr7zi9afsyyE5SHd/x9LACqmla82aNcDidT01NUVVVRU///nPqa2t9dii+3GIosjg4CCDg4McOXJkyc9SUlJYt24deXl5jI6O0tzcfE19v8uJNBv7gxw9epSdO3dyzz338PDDDy9Jm+Xl5REUFMSuXbuoq6vz6LWu0+nYsmULX/jCF1i3bh2wdD3s7u6mrKyM3/3udx6zEZDTJ9/97nfR6XQ4nU4GBwf55S9/ycmTJ+np6fGofdfttAVBiASeBDJEUZwTBOEV4GHgTuA/RFF8SRCEnwNfBn52U6z9CJxOJzMzM4yOjtLe3s7hw4c5fvy4nBOZnp5Gr9cTERHBl7/85eU257KIosjLL7/MoUOHrvl3pek6O3bs4KGHHkKtVsuFEzabbVkXb1EU6ejoYO/evTgcDjZs2IDL5WJycpLTp09TWFjImjVraGxsZP/+/bz33nte4bAFQSA4OJg1a9awfft2OVIwPT3NwYMHvdqxfBx2u93tBTCwqJefn5/PE088QUREhNwr3NTUxPnz5/n3f/93WltbParZfiMYDAYiIiI8pul/Pdjtdurr6zEYDOzYsQOTySR3yHh69LBWq+X+++9n9erVpKWlERsbu2Tghs1mo62tje9+97tMTU15VM9Bq9WyatUqvvrVr1JcXIxer2d0dJTGxkaeeeYZqqqqvGKDf6PxWhWgEwTBDuiBPmAz8Nn3f/4b4G+5iU5bkprs7++np6cHrVbL3NwcExMTtLa20tvbS0dHB2fOnKGqqmpJcZHVasVgMCwJ2YqiiE6nIzg4GI1Gg0KhWNaQbk9PzzVPC1Kr1RiNRtatW0dubi6xsbE4HA5aWlooKyu74pCGm8n8/DwdHR2cPHkS+OPfYWJiQi42Ki0t9ZqZ2R/8zqKjo1EoFAwODsqytT09PW53eh+Hn58farV6SSrl0oKu1tZWIiMjsdlsKBQKuVhKoVAQHx+PKIqMjo4yPDx8U68JhUKBXq+XQ+IZGRlySHx4eFhu72toaLhit4A0D1qpVJKYmIhOp/uQyJHD4WBoaIjBwcFlL0L6IIIgYDabiY6OZm5ujrm5Oa+7Pi6HpH/d19f3ob7zkZERurq6GBsbc1s4V2r/DAoKIjw8nNWrV7NmzRpSUlLkynyn08nExASdnZ1cuHCBkpISuTvCE0gjZyUFP6m1rq6ujjNnzlBZWSmnWD3NdTttURR7BEH4V6ATmAP2sxgOHxdFUbrSu4HLVosJgvBnwJ9d6/s6HA76+/spLS3F5XIRGRlJV1cXzc3NvPnmm7S2tl5TcY4gCISEhJCbm4u/vz8zMzPL2iIxOzt7Ta8v3QBxcXH85V/+JTExMej1eoaHh3nvvffk8K47wnfd3d10d3fLkQK9Xs/TTz9NTEwMGo2GP/zhDzQ1NXnFQmcymYiLi+P73/8+MTExcjqkqqqKI0eO8Itf/MIr8u0fxGq1YrVa5Tyfy+XCZrMxOTmJ0+nknXfeITw8HJfLhcViwWKxyPKWX/rSl7Db7Zw5c4bDhw/f1NOuSqUiOjqajRs3snnzZvnx2dlZqqqqeO6556iqqrritS0IgqxcqNfrefLJJ2UJVwlJEOTgwYPs3buX8vJyt0Zs1Go14eHhrFq1Sg6fz83NeUVo/OOQoowfnO1cXV3NyZMnaWpqcpvDUSqVpKSksGHDBgoLC0lJSSE8PByLxSI/Z35+nsbGRt5++21KSko8vtGPj49nzZo1PPXUU7I8rCiKvPLKK+zZs2fZx7JeCzcSHrcC9wLxwDiwC9h+tb8viuIvgV++/1pXfVdIJ7zf/va3vPrqq7KqkbSwXc9uUtK/1Wg0N72C/NLWLEEQ+NrXvkZUVNSHcmkfRKVSYTAY+M53vkNOTg6pqalERkYyNzdHU1MT3//+96mtraWrq8sji0p4eDipqal85jOfob29nd/+9rdcuHDBa8KikpZ3XFyc3AMNi2Fcb2lDg8W/c0BAACEhIWzbto1169aRmJgoT2uSipDm5uZQKBQEBgbKE8lSU1MZHBzE4XCgVquJioqio6OD+fl5Tp8+fdP+FuHh4aSkpPA3f/M3ZGRkEBgYKP/MZrPR1dWFv78/ISEh8uk4NjaWO++8c0lodsuWLYSGhmK1WgkKCkKtVi+532w2G//6r//KiRMnqKmpcXuKxWg0Eh4eTnJyMrt27aK5udnrJtN9EIVCQUREBPfeey87duwgODh4SZGtdO1IRbjuQDoIrVixgs2bN6PRaFCr1fLgkpKSEioqKvjtb3/LwMCAWyaoXQmFQkFwcDB3330327dvx2g0Mjs7S3d3N88++yxHjhyhr6/PY/ZdjhsJj28F2kRRHAIQBOF1oBiwCIKgev+0HQXc9Ky9y+W6rtyCNE1LUhSTdGSVSiUajYasrCwcDgcXL168KYv67OwsdXV1xMfHYzAYUKlUhISEkJqaKg8DmZ2dZXR0lJmZGcxms5zvkZz22rVrSUhIIDQ0lJaWFrq6urh48aLcs+6pRSUuLo6NGzdiNpvp6enhzJkzzMzMeMUpGxbDzCaTCT8/PzkcNzIyQm9vL/39/R532iqVCp1OR05OjjwcpLCwkKysLFk1z26343A4cDqdjI2NYbPZmJ2dXRJJkq5fQRAYGRlhYGBAduQ3C0nMRxpfeGnoXqPREB0dTVFRETExMbKQUVRUFJs2bZJzw4IgkJubi9VqxWg0Xja/KggCNptNDk27E0EQCA8PJygoCI1GI4u/LPd1Ig0xSk9PlydgweK4y/b29iWpBumkLKUYAgMDCQsLIyMjg7Vr15KZmYlarZa/c0+N9FWpVHIa71I99IWFBVpaWqirq6O6ulqOinpqzZDmY99xxx3k5+cTFxfH/Pw8NTU1VFdXc/r0aY+N5/0obsRpdwKFgiDoWQyPbwHOA0eAT7NYQf4Y4Dn1hw/gcDiYmZlhbGyMwMBALBaL3CamUqn41Kc+BSzqZd+MXf7w8DCvv/462dnZ+Pv7yye+1NRUnn76aXmWcFlZGW1tbaSnp8ujLJVKJWq1mpycHGBRbu+1116jvLycuro6WcHJU+Tn5/OVr3yFmZkZKioq2Ldvn9c4bFh02pc6B4fDQV1dHa2trV4xcUyn0xEeHs7XvvY1CgoKSEpKApYutJOTk3J4trOzk4GBgSXtPpejvb2d48eP31SnNzExQXd3N9XV1YSHh8snbWnS3tatWykoKJDHx0qfT1JF+6gCqEurh0VRxGKxLHFe7uJSLQKXy0V5eTmdnZ3L/r4mk4msrCz+6q/+aonuxNmzZ3nhhReW1CZMTk7Ka5VOp6OwsJANGzZw2223ERQU9KFhG5f+4040Gg33338/cXFxSx632WwcO3aMsrKyJWm0D04mc5e9Wq2WyMhIfvCDH8iR1p6eHt544w2OHDlCWVmZW+y4Vm4kp31OEIRXgXLAAVSwGO7eDbwkCMKP33/sf2+GoTeD2dlZurq6+MEPfsAdd9zB5z73OSwWC0qlEpVKxfbt22lqamL37t03ZdGbmpri7NmzPPPMM6xbt45vfetb8kSe/Px8RFEkNzeXzZs3Y7fb8fPzQ6/Xyxex0+mkpqaG2tpaKisreeWVV+T+Uk9KKkp9oXq9np/85CecO3fO6/qcCwsL+cpXviKHCmdnZ9m1axc1NTUeUT77IOnp6RQXF7Njx44l4fuGhgYaGxt544035KlMUo+20+n82F2/w+GQp1LdLBYWFujp6eEf//EfGRkZYePGjRQXFy95jjQDXspbXm2lssPh4De/+Q1jY2NMTk7y4osvekS0QhAEkpKSCAsLQxRFGhsb3ZLHlGRTY2NjiY2NlR+PjIxk69atssN2uVzU1NSgUChkMRiDwYBer0en030orTc3N8f+/fs5d+4ctbW1y/45JKTiyICAAHnssITBYODRRx/lgQceYHx8nKNHjzI/P7/kgDQ6OsqRI0e4ePEiU1NT1z3p62qQJuuFhob+/+3de0yc553o8e/DwHAfYLjbGAw2iS9guzbGsR37WCd1k0YbpSeJojp/NKtErXqSrXrSHrVZVd3sqlppz1ZNulX2bJTjjdJLWjvNaZS26XFo0thO44CxMWAu9mBzx9zBgGEG5vKcP+adN1wNtjEzwO8jvWL8zu2Znx/4zftcsVqtdHV18f3vf5/z588HfVrXzdzR6HGt9UvAS9NONwLFd/K6d0ugf7C+vh6bzUZ6ejoPPvggcXFxREREYLPZSEpKIjk5mY6OjjuuKD6fj7GxMerr61FKsWbNGu69914yMjLMkb7R0dEzttSbmJgwB5aUlJRw+fJlHA4Hvb29QZ1KFViw4f777ycnJ4exsTHOnz/PtWvXgt7cPFlg5GpaWhpKKTo6OnA4HFRVVdHX1xcSA9ACMx4CU7ecTicdHR2cPXuWuro6ysvL6ezsZGRkJOjlDdTJrq4uLl++bP6xS0pKMv8wBxZdmY3H48HlclFeXj5j33uv18uHH37IyMgIY2NjdHV1LXlzZGDvgYKCApKTk80BaEvRchSYkhWYBRAQGRmJzWabshRzIMaBLXKnxzuwKYvW2pyOWVtbS3Nz85L9ftpsNrKysoiKiprxRSKwShv4B1zO1tc+MjJCYmIi165dY3h4mOvXr1NXV0dfX99dm54ZWM64ubmZqqoqenp6Qnosw7JbEW0xtLW1mX8kCwsLWbduHRaLBa21OeWjs7Nz0RJkW1sbPT09XLx4ka985Svs37+f7OzsGVcjgUX+BwcHaWtro76+ntdeey2ofdeTWSwWYmNjOXLkCOnp6XR3d1NWVhYScxcDAgNz7HY7VqsVpRQ1NTWUlJRQXl4eMk347e3tWCwWmpqacDqd9Pb2cuLECU6fPk1jY2Owizcrj8dDfX094eHh5Obmsn37dnNcSGDAGWD2oQYSxdjYGL29vbz66qvmDnaTjY6OLnm/62SxsbFkZ2dz4MABc4GgpfxyHOgOmb5r3/RlSWfbmnV6M/jQ0BA+n4/u7m5OnDhBW1vbkg70SktLY8uWLWa9mPyZJq9WGRUVxc6dO2d8FoDDhw8D/gR+6dIljh49SkVFxaIn7cDF0ejoKFevXuXChQtcuXIl6F+S57Mqkzb4m2EqKyt55ZVX2Lt3L3v27OGzzz7jo48+or6+ftH/48bHx+nu7uaXv/wln376KRUVFTzxxBOsW7eOtLQ0WltbuXjxIqdOnaKtrY22tjZzKb9QWKgE/CsF7d+/n4KCAk6cOMEbb7xBf39/SMxdDLBYLBQUFJgbtMDnS4CGUmvA0NAQdXV1PP744+aXNafTGbI7MgXU1tZy9epVPvnkE4qLi8nKyiItLY0nn3yS7OxstNZ8+OGH5nK24E/K3d3dnDlzhhs3bsz44hTs/xer1Wpe1fb391NXV7dkddrlcnHt2jVOnDjBjh072L17t7mvwEK43W5OnDhhLsn66quvmttK9vb2LvmX1MbGRnp7e7Hb7ezZs4fCwkJSU1Ox2Wzm7+Ncny0wELGnpwer1crExARKKb797W9z5swZvvnNby5qWbu7uxkaGuKRRx5hZGQkJFq1FmLVJm2fz4fL5eLChQuMjIzQ2NiIw+HA4XCY+1ovtsCGBa2trXz66af4fD5zAElfX5+5h+zg4CADAwMhdQULkJGRQXFxMePj43R2dtLQ0BAyV64BPp+PlpaWKUvY9vX1LUp3x2IKzGQItekk85mYmDCnWFZWVtLS0kJCQgIul4uUlBTAPxe+q6vLvKIeHx83/yiGWn0B/1Sv9evX4/V6aW1t5cyZM0vWRD8+Pk5XVxd/+ctfzLnKX/jCF8wEN11XV5e5WEqgDr3//vvmqPsrV67gdDqD9kXf4/EwPDxstsDV1dWxfv160tLSzIFydrudpKQk1q5dOyWBOxwOOjo6qKurIzo62lyTY9OmTXdl8Gjgi7LD4WBiYiIk6+ZsVm3SBv83/KqqKqqqqpb0fQcHByktLaW0tHRJ3/dOZWVlceDAAXp6emhvb1+SKTG3yuv1cvbsWfbu3Utvby9RUVE0NzfjcDhCrqzLVWC+7eXLl81zwdpuczEkJiaydetWJiYmuHz5Mn/605+WrDvK5XKZLWu1tbXcuHGDdevWmSPvp7t06RIlJSXU19ebLUgnT54MqZYkr9dLeXk55eXlhIeHU1BQQFZWljnroLCwkHvvvRe73T4laX/66aeUlpZy6tQp4uPj8Xg8dHZ2snfv3ru2m53P51t2Sxmv6qQtbk1kZCSxsbEcO3aMqqqqkGm2n83Ro0c5duwY4O9TXS4rW4ng0FrT3t5OV1cXo6OjQakrjY2NvP766/z617+ec1DfxMQE4+PjU0ZUh9o84sk8Hg+1tbVcunTJ/Ezvvvsu4eHhUwbegX9wZuDzhYWFmX39JSUl8rs7iSRtsWA1NTUcPXqUTz75hJaWlmAX56YCA0yEmE93dzenT5/m6tWr1NfXBy1BBDY9Wmn11u123/IYgckXBKE+zmOpqVD4BnMry5gKIYQQK8R5rXXRrTxh+ew/J4QQQqxykrSFEEKIZUKSthBCCLFMSNIWQgghlglJ2kIIIcQyIUlbCCGEWCYkaQshhBDLhCRtIYQQYpmQpC2EEEIsE5K0hRBCiGVCkrYQQgixTMiGISEkLCwMi8VCXFwcFouFsLAwoqKi8Hg8TExMmLv5aK0ZHR1dFhu2CyGEWDyStEOIzWYjKyuLr33ta6Snp5OUlMShQ4dobW2lpqaGiooKc+P2t99+m+Hh4ZDelk8IIcTikqQdQjIzMzl48CCHDh0iKSmJyMhIoqOjWbt2LTExMeTl5aG1xu12s337ds6fP091dTVlZWWy36wQQqwC8yZtpdQbwN8APVrrAuOcHTgOrAeagSe11oNKKQX8G/AwMAb8rda6YrEK6395/8+oqCgiIiKIiIhY8PPdbjejo6N4PJ7FKtKiSklJYdeuXWzatInY2Fi01kxMTBAeHk5ycjLJycnmYzdv3kxiYiJKKcrLy6fsPyuEEGJlWsiV9pvAq8AvJp17EfhIa/0vSqkXjX9/H/gykG8ce4D/MH4uCqvVSkREBFarlaKiInJzc8nPz1/w8x0OBx988AEdHR0hmbizsrI4fPgwUVFRAPh8PhwOB9evX2d4eNh8XEREBPfddx8FBQVorXnjjTckaQshxCowb9LWWp9WSq2fdvpR4JBx++fASfxJ+1HgF9rfVluqlEpUSmVqrTtvt4AbNmxg3bp1rF+/nqysLFJSUli7di12u524uDji4+MX/FpFRUXs2LGDH//4x3R2duJyuW63WHdNoDVhdHSUvr4+fvazn9HV1cXIyAgAUVFRpKSksGXLFuLj49m4cSMHDx6ksbGRnp4eRkdHg1l8IYQQd9Ht9mmnT0rEXUC6cXst0Dbpce3GuRlJWyn1DeAb871Rfn4+u3fvZvPmzaxfv5709HTWrFkD+K9EXS4XXq8Xr9eL2+2e8fywsDCsVisJCQlkZWWRlZXF8ePHGR0dDcmkHTA+Ps7g4CCnTp3i2rVrOJ1OwsLCSElJIS8vD6/Xi81mIz09nf379xMTE0NDQwOXLl0KdtGFEELcJXc8EE1rrZVStzwKSmv9OvA6wM2e/9hjj/Hss8+itTb7eDs7/d8BRkZGqKysZGhoiOvXr9Pa2jrj+fHx8eTk5PDUU09ht9vJzs7mi1/8IuHh4Xz00Ue3Wuwl4/F4cDqdDA4O4nQ6UUoRHx/Pzp07OXjwIBkZGVitVhITE/nhD39IRUUFpaWlvPDCCzIVTAghVqjbTdrdgWZvpVQm0GOc7wDWTXpclnHutg0NDdHZ2Ynb7aa0tJT6+npKS0vRWuPxeBgaGsLtduN2u3E6nVOem5iYyMaNG9m6dSthYWFm0/PAwAA3bty4k2LdddHR0aSmprJp0yZaW1txuVx85zvfobCwkI0bNxIREYFSyhw13tzcTHV1tYwiF0KIFex2k/bvgaeBfzF+vjfp/N8ppY7hH4A2dCf92QBNTU2UlZXhdrs5efIk1dXVfPbZZzd9jlKKiIgIUlNTSUxMJD09nfDwcCYmJhgZGaGnp2fKwK5QZLVasdls7N69m4yMDJxOJw888AA5OTmkpqYCmIuu9Pb24nA4aGhoCHKphRBC3FWBZue5DuA3+Puk3fj7qJ8FkoGPgAbgQ8BuPFYB/w5cBS4CRfO9vvE8vZhHdHS0zs3N1cePH9cOh0N7PB7t8Xh0S0uL/u1vf6s3bNigIyMjF/U9F+M4cuSIbmtr0xMTE2aZZzu8Xq/2er26t7dXV1ZW6ueee05v3rw56OWXQw455JDjlo5zC8mRk4+FjB4/MsddD8zyWA08P99r3g1KKaxWK/n5+RQWFvLkk0+ya9cukpKSzGbxxsZGfvWrX9Hb2zvroLVQEijzdIODg3R0dPDyyy/T39/P8PAwzc3N9Pf3L3EJhRBCLLUVsyJaVFQU2dnZ7Nmzh6KiIg4cOEB8fDzh4Z9/RK01Xq+XnJwcM2mPjY0xOjrKyMhIUJcEjYyMJDIykrCwmXu4aK3x+Xy0tLTQ2tpKQ0MDJSUlDA0NMT4+LnO0hRBilVgxSTsjI4NnnnmGI0eOmFPCptu8eTPf/e53GRsbw+fzobXm4sWLVFZWUlZWRnt7e9BGXq9Zs4bU1FQiIyNnvcp2Op389Kc/5dy5c9TX14d8n7wQQojFt2KSdmRkJGvXrsVqtc7ZtJyYmMi2bdumXJkWFBSwc+dOtm7dyk9+8hNu3LixZCOww8LCsNvt5Obm8txzz7Ft2zbi4uJQSuF2uxkfH6eqqorKykrKy8s5efIk169fZ2xsbEnKJ4QQIrSsmKQ9Pj5OZ2cnNTU1JCQkzLg/IiKC6OhoEhIS6O3tBfwjtLOzs83pYLt27aKhoYGOjjuapTavwIIvGRkZ5Obmsnv3bu677z4yMzPNtdQHBwdpbW3l9OnTnD9/noqKCjo6OmQOthBCrGIqFOb13s7iLNPFxMSQk5NDQkLClH5s4/Wx2+3k5+ezb98+fve736G1Ji0tje9973vY7XaUUpw/f54333yTo0eP3tV+4piYGNLT03niiSfYv38/jzzyyIzHfPzxxxw7doy33nprxvxzIYQQK8J5rXXRrTxhxVxpu1wumpubsVgsU5rHbTYbubm5fOlLX6K/v59XXnkFh8OB1+vFarXS0dHB/fffz+HDh7nnnnvIz88nKyuLtra2u3JVm5OTw+7du3nmmWfIz88nMTFx1seNjIzQ3t4ug8yEEEKYVkzS9vl8M65I4+LisNls3HPPPbjdbrq6urh06RL9/f34fD6UUpSVlREbG0teXh45OTnm1fg777yzqEnbYrEQExNDUVER+/btY9euXSQmJmKxWPD5fAwMDJgLqoB/4ZTx8XFZ4UwIIYRpxSTt2eTm5lJcXMzTTz/Nyy+/TE1NjdmfDf6pVC0tLVRWVpKamsqBAwc4dOgQ+fn5vP/++4s6lzsmJoYNGzbw/PPPU1hYiN1uNyfLe71eLly4QGpqKjt27Fi09xRCCLGyrOik/fDDD7Nt2zasVivXrl2jr69v1sf19/dTV1eHy+UiISHB7ONeTBs3buRHP/oRBQUF2Gw2tNZcuHCB2tpa/vrXv5KSkkJxcTHbt29f9PcWQgixMqzopJ2RkcGaNWuw2+1s3rwZn89HU1MTg4ODZtN3RkYGmZmZpKenY7FYsFgsMway3am0tDTy8vIoLCwE/Jt71NXVUVtbi8PhoKKigoceemhGf7wQQggx2YpO2jExMSQnJ5OXl8fXv/51ysrKeO+99zh37hxOpxOLxUJRUREFBQXs2LGD6OjoKTtnLZYdO3awe/duMjMzuXz5MiUlJbz00ktMTEzg9XrRWvPss8+Snp4+5b2lP1sIIcRkKzppt7S0kJ2dzZYtW9i+fTsbNmzgwQcfpLu7G4/Hg1KK9evXExsbS3R0NFarldOnT/PHP/4Rl8t1x+8fGRlJZmYmTz31FMXFxYSFheHz+XC73YyNjZl92haLhX379rFp0yYArl+/zsDAAIODg5K4hRBCmFZ00q6urjZHhq9bt47k5GQSEhJIT083R48nJCSY630HBqWVlpbi8Xju+P0tFgtxcXFkZmaa22n29vaao9djY2Ox2WxkZmaSkpJCdHQ0Wmva2tro6OgwHyeEEELACk/aH3zwAZ2dnSQlJfHYY49ht9uJiIiYMTdaa43b7eb06dOcOnWK0tLSRUmWgWle8fHxxMTEANDQ0EBraythYWGkp6ezadMmDh48iM1mIywsDK/XS2VlJbW1tbS3t99xGYQQQqwcK2ZFtLlER0eTlJREamoqGzduZO/evWaCDOjr66O1tZU//OEPDAwMMDo6uijvnZmZyeOPP863vvUt8vLyUErR2dlJe3s71dXVbNy4kdTUVNasWYPNZqO/v5+GhgZeeOEFmpqaGBgYWJRyCCGECEmrd0W0ubhcLrq7uxkcHGR4eBiXy0VMTMyUUdrDw8P09vbS1dW1qHOzPR4P/f39jI2N4Xa7sVqt2O12wsPDiYqKIi0tjbi4OGJjY/H5fHR3d3Pu3Dna29tlFy8hhBAzrPikHVi8xOl00tTURFNT05K99+joKFVVVVy7ds3st46MjCQtLY20tDSzfFprnE4nV69e5YMPPmBoaGhR+tSFEEKsLCu+eTyYlFKEh4ezfft2Dhw4wIsvvkhiYuKUeeDd3d3U1dVx/Phxc29vWb5UCCFWBWkeDyWBAW5NTU1YLBaOHj1KbGzslP70oaEhOjo6OHv2LF1dXYsy1UwIIcTKJFfaQgghRHDc8pV22PwPEUIIIUQomDdpK6XeUEr1KKVqJp37sVLqklKqWin1rlIqcdJ9f6+UuqKUuqyUevAulVsIIYRYdRZypf0m8NC0c38GCrTW2wAH8PcASqktwFeBrcZz/rdSyrJopRVCCCFWsXmTttb6NDAw7VyJ1jowJ6kUyDJuPwoc01qPa62bgCtA8SKWVwghhFi1FqNP+xng/xm31wJtk+5rN87NoJT6hlLqnFLq3CKUQQghhFjx7mjKl1LqB4AHeOtWn6u1fh143XgdGT0uhBBCzOO2k7ZS6m+BvwEe0J/PG+sA1k16WJZxbj59wKjxU9xcChKnhZA4LYzEaWEkTvOTGC3M5Djl3OqTbytpK6UeAr4H/Bet9diku34P/Fop9TKwBsgHzs73elrrVKXUuVudr7YaSZwWRuK0MBKnhZE4zU9itDB3Gqd5k7ZS6jfAISBFKdUOvIR/tHgk8Gdj441SrfU3tda1Sqm3gTr8zebPa629t1s4IYQQQnxu3qSttT4yy+n/vMnj/xn45zsplBBCCCFmCqUV0V4PdgGWCYnTwkicFkbitDASp/lJjBbmjuIUEmuPCyGEEGJ+oXSlLYQQQoibkKQthBBCLBNBT9pKqYeMzUWuKKVeDHZ5QolSqlkpdVEpVRlYOU4pZVdK/Vkp1WD8TAp2OZfaHJvYzBoX5fczo35VK6V2Bq/kS2uOOP2jUqrDqFOVSqmHJ923Kjf7UUqtU0p9rJSqU0rVKqW+bZyXOjXJTeIkdWoSpVSUUuqsUqrKiNM/GedzlVJlRjyOK6WsxvlI499XjPvX3/QNtNZBOwALcBXIA6xAFbAlmGUKpQNoBlKmnftX4EXj9ovA/wp2OYMQl4PATqBmvrgAD+NfZlcB9wFlwS5/kOP0j8D/nOWxW4zfv0gg1/i9tAT7MyxRnDKBncbtePybIG2ROrXgOEmdmvq5FRBn3I4Ayox68jbwVeP8a8B/N24/B7xm3P4qcPxmrx/sK+1i4IrWulFrPQEcw7/piJjbo8DPjds/B74SvKIEh55lExvmjsujwC+0XymQqJTKXJKCBtkccZrLqt3sR2vdqbWuMG6PAPX490yQOjXJTeI0l1VZp4x6ccP4Z4RxaOC/Au8Y56fXp0A9ewd4QBkLoMwm2El7wRuMrFIaKFFKnVdKfcM4l6617jRudwHpwSlayJkrLlLHZvo7o1n3jUndKxInwGia/AL+qyOpU3OYFieQOjWFUsqilKoEevBvZX0VuK4/3x1zcizMOBn3DwHJc712sJO2uLn7tdY7gS8DzyulDk6+U/vbU2TO3jQSl5v6D2ADsAPoBH4S1NKEEKVUHPB/gf+htR6efJ/Uqc/NEiepU9Norb1a6x34998oBjYt1msHO2nf7gYjq4LWusP42QO8i/8/vzvQFGf87AleCUPKXHGROjaJ1rrb+IPiA/4PnzdXruo4KaUi8Ceit7TWvzNOS52aZrY4SZ2am9b6OvAxsBd/N0pgFdLJsTDjZNyfAPTP9ZrBTtrlQL4xqs6KvxP+90EuU0hQSsUqpeIDt4EvATX44/O08bCngfeCU8KQM1dcfg98zRjxex8wNKnJc9WZ1vf63/DXKfDH6avGSNZcFrjZz0pg9B/+J1CvtX550l1SpyaZK05Sp6ZSSqUqpRKN29HAYfz9/x8DTxgPm16fAvXsCeAvRsvO7EJgpN3D+EchXgV+EOzyhMqBf0R9lXHUBmKDv6/jI6AB+BCwB7usQYjNb/A3w7nx9w09O1dc8I/k/Hejfl0EioJd/iDH6ZdGHKqNPxaZkx7/AyNOl4EvB7v8Sxin+/E3fVcDlcbxsNSpBcdJ6tTUOG0DLhjxqAH+wTifh/9LyxXgt0CkcT7K+PcV4/68m72+LGMqhBBCLBPBbh4XQgghxAJJ0hZCCCGWCUnaQgghxDIhSVsIIYRYJiRpCyGEEMuEJG0hhBBimZCkLYQQQiwT/x/ArHUNtvT3yQAAAABJRU5ErkJggg==\n"
          },
          "metadata": {
            "needs_background": "light"
          }
        }
      ],
      "source": [
        "import torchvision\n",
        "from torch.utils.data import DataLoader\n",
        "\n",
        "mnist_train = datasets.MNIST(\"data\", train=True, download=True, transform=transforms.ToTensor())\n",
        "\n",
        "mnist_test = datasets.MNIST(\"data\", train=False, download=True, transform=transforms.ToTensor())\n",
        "\n",
        "bs = 32\n",
        "train_dl = DataLoader(mnist_train, batch_size=bs)\n",
        "test_dl = DataLoader(mnist_test, batch_size=100)\n",
        "\n",
        "dataiter = iter(train_dl)\n",
        "images, labels = next(dataiter)\n",
        "viz = torchvision.utils.make_grid(images, nrow=10, padding=2).numpy()\n",
        "fig, ax = plt.subplots(figsize=(8, 8))\n",
        "ax.imshow(np.transpose(viz, (1, 2, 0)))\n",
        "# ax.grid(None)"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "779GU7DKdVbn"
      },
      "source": [
        "Using PyTorch's built-in functions, we can easily define any model like multi-layer perceptrons. After training, we just care about the average test accuracy, so let's write a function to compute the accuracy over the test set."
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 36,
      "metadata": {
        "id": "2Led8bErdZrw"
      },
      "outputs": [],
      "source": [
        "def get_test_stat(model, dl, device):\n",
        "    model.eval()\n",
        "    cum_loss, cum_acc = 0.0, 0.0\n",
        "    for i, (xb, yb) in enumerate(dl):\n",
        "        xb = xb.to(device)\n",
        "        yb = yb.to(device)\n",
        "\n",
        "        xb = xb.view(xb.size(0), -1)\n",
        "        y_pred = model(xb)\n",
        "        loss = loss_fn(y_pred, yb)\n",
        "        acc = (torch.max(y_pred.data, 1)[1] == yb).sum()  # accuracy(y_pred, yb)\n",
        "        cum_loss += loss.item() * len(yb)\n",
        "        cum_acc += acc.item() * len(yb)\n",
        "    cum_loss /= 10000\n",
        "    cum_acc /= 10000\n",
        "    model.train()\n",
        "    return cum_loss, cum_acc"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 37,
      "metadata": {
        "id": "ohxuCpR9eLGt",
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "outputId": "2e1347f7-356c-48db-ee54-f3caac3319d2"
      },
      "outputs": [
        {
          "output_type": "stream",
          "name": "stdout",
          "text": [
            "Using cuda\n",
            "0\n",
            "Test loss: 0.41829101026058196  Test acc: 88.88\n",
            "1\n",
            "Test loss: 0.3321799498796463  Test acc: 90.55\n"
          ]
        }
      ],
      "source": [
        "dim_x = 784\n",
        "dim_h = 100\n",
        "dim_out = 10\n",
        "\n",
        "model = torch.nn.Sequential(\n",
        "    torch.nn.Linear(dim_x, dim_h),\n",
        "    torch.nn.ReLU(),\n",
        "    torch.nn.Linear(dim_h, dim_out),\n",
        ")\n",
        "\n",
        "learning_rate = 1e-2\n",
        "epochs = 2\n",
        "\n",
        "optimizer = torch.optim.SGD(model.parameters(), lr=learning_rate)\n",
        "\n",
        "# Using GPUs in PyTorch is pretty straightforward\n",
        "if torch.cuda.is_available():\n",
        "    print(\"Using cuda\")\n",
        "    use_cuda = True\n",
        "    device = torch.device(\"cuda\")\n",
        "else:\n",
        "    device = \"cpu\"\n",
        "\n",
        "# we need to tell pytorch to move the model to gpu\n",
        "model.to(device)\n",
        "\n",
        "loss_fn = torch.nn.CrossEntropyLoss()\n",
        "\n",
        "model.train()\n",
        "for epoch in range(epochs):\n",
        "    print(epoch)\n",
        "    for i, (xb, yb) in enumerate(train_dl):\n",
        "\n",
        "        # We also need to transfer the data to the target device\n",
        "        xb = xb.to(device)\n",
        "        yb = yb.to(device)\n",
        "        xb = xb.view(xb.size(0), -1)\n",
        "\n",
        "        # Forward pass\n",
        "        y_pred = model(xb)\n",
        "        loss = loss_fn(y_pred, yb)\n",
        "\n",
        "        # Backward pass\n",
        "        model.zero_grad()  # Zero out the previous gradient computation\n",
        "        loss.backward()  # Compute the gradient\n",
        "        optimizer.step()  # Use the gradient information to make a step\n",
        "\n",
        "    test_loss, test_acc = get_test_stat(model, test_dl, device)\n",
        "    print(\"Test loss: {}  Test acc: {}\".format(test_loss, test_acc))"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "nSTuUZlRgeUD"
      },
      "source": [
        "### Dynamic network"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "Ji_qMWsOgbbj"
      },
      "source": [
        "To showcase the power of PyTorch dynamic graphs, we will implement a very strange model: a fully-connected ReLU network that on each forward pass randomly chooses a number between 1 and 4 and has that many hidden layers, reusing the same weights multiple times to compute the innermost hidden layers.\n",
        "\n",
        "By Justin Johnson: https://github.com/jcjohnson/pytorch-examples/blob/master/nn/dynamic_net.py."
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 38,
      "metadata": {
        "id": "D5dRsCN9gnvU",
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "outputId": "aa545540-d221-452a-f97a-83d8774f2285"
      },
      "outputs": [
        {
          "output_type": "stream",
          "name": "stdout",
          "text": [
            "0 51.36301040649414\n",
            "10 49.24879837036133\n",
            "20 41.896018981933594\n",
            "30 32.24109649658203\n",
            "40 13.806378364562988\n"
          ]
        }
      ],
      "source": [
        "import random\n",
        "\n",
        "\n",
        "class DynamicNet(torch.nn.Module):\n",
        "    def __init__(self, D_in, H, D_out):\n",
        "        \"\"\"\n",
        "        In the constructor we construct three nn.Linear instances that we will use\n",
        "        in the forward pass.\n",
        "        \"\"\"\n",
        "        super(DynamicNet, self).__init__()\n",
        "        self.input_linear = torch.nn.Linear(D_in, H)\n",
        "        self.middle_linear = torch.nn.Linear(H, H)\n",
        "        self.output_linear = torch.nn.Linear(H, D_out)\n",
        "\n",
        "    def forward(self, x, verbose=False):\n",
        "        \"\"\"\n",
        "        For the forward pass of the model, we randomly choose either 0, 1, 2, or 3\n",
        "        and reuse the middle_linear Module that many times to compute hidden layer\n",
        "        representations.\n",
        "        Since each forward pass builds a dynamic computation graph, we can use normal\n",
        "        Python control-flow operators like loops or conditional statements when\n",
        "        defining the forward pass of the model.\n",
        "        Here we also see that it is perfectly safe to reuse the same Module many\n",
        "        times when defining a computational graph. This is a big improvement from Lua\n",
        "        Torch, where each Module could be used only once.\n",
        "        \"\"\"\n",
        "        h_relu = self.input_linear(x).clamp(min=0)\n",
        "        n_layers = random.randint(0, 3)\n",
        "        if verbose:\n",
        "            print(\"The number of layers for this run is\", n_layers)\n",
        "            # print(h_relu)\n",
        "        for _ in range(n_layers):\n",
        "            h_relu = self.middle_linear(h_relu).clamp(min=0)\n",
        "            if verbose:\n",
        "                pass\n",
        "                # print(h_relu)\n",
        "        y_pred = self.output_linear(h_relu)\n",
        "        return y_pred\n",
        "\n",
        "\n",
        "# N is batch size; D_in is input dimension;\n",
        "# H is hidden dimension; D_out is output dimension.\n",
        "N, D_in, H, D_out = 64, 1000, 10, 1\n",
        "\n",
        "# Create random Tensors to hold inputs and outputs, and wrap them in Variables\n",
        "x = torch.randn(N, D_in)\n",
        "y = torch.randn(N, D_out).requires_grad_(False)\n",
        "\n",
        "# Construct our model by instantiating the class defined above\n",
        "model = DynamicNet(D_in, H, D_out)\n",
        "\n",
        "# Construct our loss function and an Optimizer. Training this strange model with\n",
        "# vanilla stochastic gradient descent is tough, so we use momentum\n",
        "criterion = torch.nn.MSELoss(reduction=\"sum\")\n",
        "optimizer = torch.optim.SGD(model.parameters(), lr=1e-4, momentum=0.9)\n",
        "for t in range(50):\n",
        "    # Forward pass: Compute predicted y by passing x to the model\n",
        "    y_pred = model(x)\n",
        "\n",
        "    # Compute and print loss\n",
        "    loss = criterion(y_pred, y)\n",
        "    if t % 10 == 0:\n",
        "        print(t, loss.data.item())\n",
        "\n",
        "    # Zero gradients, perform a backward pass, and update the weights.\n",
        "    optimizer.zero_grad()\n",
        "    loss.backward()\n",
        "    optimizer.step()"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "FX-mRC_5iNvs"
      },
      "source": [
        "### CIFAR10 classification"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "KVftofphh-ng"
      },
      "source": [
        "We will finish with an example on [CIFAR10](https://www.cs.toronto.edu/~kriz/cifar.html), highlighting the importance of applying transformations to your inputs.\n",
        "\n",
        "Example is lifted from: https://github.com/uoguelph-mlrg/Cutout/blob/master/train.py."
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 39,
      "metadata": {
        "id": "9iAw2caKiW97"
      },
      "outputs": [],
      "source": [
        "import torch.nn as nn\n",
        "import torch.nn.functional as F\n",
        "\n",
        "\n",
        "class Net(nn.Module):\n",
        "    def __init__(self):\n",
        "        super(Net, self).__init__()\n",
        "        self.conv1 = nn.Conv2d(3, 6, 5)\n",
        "        self.pool = nn.MaxPool2d(2, 2)\n",
        "        self.conv2 = nn.Conv2d(6, 16, 5)\n",
        "        self.fc1 = nn.Linear(16 * 5 * 5, 120)\n",
        "        self.fc2 = nn.Linear(120, 120)\n",
        "        self.fc3 = nn.Linear(120, 10)\n",
        "\n",
        "    def forward(self, x):\n",
        "        x = self.pool(F.relu(self.conv1(x)))\n",
        "        x = self.pool(F.relu(self.conv2(x)))\n",
        "        x = x.view(-1, 16 * 5 * 5)\n",
        "        x = F.relu(self.fc1(x))\n",
        "        x = F.relu(self.fc2(x))\n",
        "        x = self.fc3(x)\n",
        "        return x"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 40,
      "metadata": {
        "id": "A3p2SQViibIc"
      },
      "outputs": [],
      "source": [
        "def get_data(data_normalize=False, data_augment=False):\n",
        "    train_transform = transforms.Compose([])\n",
        "    test_transform = transforms.Compose([])\n",
        "\n",
        "    if data_augment:\n",
        "        train_transform.transforms.append(transforms.RandomCrop(32, padding=4))\n",
        "        train_transform.transforms.append(transforms.RandomHorizontalFlip())\n",
        "\n",
        "    train_transform.transforms.append(transforms.ToTensor())\n",
        "    test_transform.transforms.append(transforms.ToTensor())\n",
        "\n",
        "    if data_normalize:\n",
        "        normalize = transforms.Normalize(\n",
        "            mean=[x / 255.0 for x in [125.3, 123.0, 113.9]],\n",
        "            std=[x / 255.0 for x in [63.0, 62.1, 66.7]],\n",
        "        )\n",
        "        train_transform.transforms.append(normalize)\n",
        "        test_transform.transforms.append(normalize)\n",
        "\n",
        "    train_dataset = datasets.CIFAR10(\n",
        "        root=\"data/\", train=True, transform=train_transform, download=True\n",
        "    )\n",
        "\n",
        "    test_dataset = datasets.CIFAR10(\n",
        "        root=\"data/\", train=False, transform=test_transform, download=True\n",
        "    )\n",
        "    train_loader = torch.utils.data.DataLoader(\n",
        "        dataset=train_dataset, batch_size=128, shuffle=True, num_workers=2\n",
        "    )\n",
        "\n",
        "    test_loader = torch.utils.data.DataLoader(\n",
        "        dataset=test_dataset, batch_size=128, shuffle=False, num_workers=2\n",
        "    )\n",
        "    return train_loader, test_loader\n",
        "\n",
        "\n",
        "def test(net, loader):\n",
        "    net.eval()  # Change model to 'eval' mode (BN uses moving mean/var).\n",
        "    correct = 0.0\n",
        "    total = 0.0\n",
        "    for images, labels in loader:\n",
        "        with torch.no_grad():\n",
        "            pred = net(images)\n",
        "\n",
        "        pred = torch.max(pred.data, 1)[1]\n",
        "        total += labels.size(0)\n",
        "        correct += (pred == labels).sum().item()\n",
        "\n",
        "    val_acc = correct / total\n",
        "    net.train()\n",
        "    return val_acc"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 41,
      "metadata": {
        "id": "-GR395vyihIU"
      },
      "outputs": [],
      "source": [
        "def train_model(train_loader, test_loader, epochs=5):\n",
        "\n",
        "    net = Net()\n",
        "    optimizer = torch.optim.SGD(net.parameters(), lr=0.1, momentum=0.9)\n",
        "    criterion = nn.CrossEntropyLoss()\n",
        "    train_accs = []\n",
        "    test_accs = []\n",
        "\n",
        "    net.train()\n",
        "\n",
        "    for epoch in range(epochs):\n",
        "        print(epoch)\n",
        "\n",
        "        xentropy_loss_avg = 0.0\n",
        "        correct = 0.0\n",
        "        total = 0.0\n",
        "\n",
        "        for i, (images, labels) in enumerate(train_loader):\n",
        "\n",
        "            net.zero_grad()\n",
        "            pred = net(images)\n",
        "            xentropy_loss = criterion(pred, labels)\n",
        "            xentropy_loss.backward()\n",
        "            optimizer.step()\n",
        "\n",
        "            xentropy_loss_avg += xentropy_loss.item()\n",
        "\n",
        "            # Calculate running average of accuracy\n",
        "            pred = torch.max(pred.data, 1)[1]\n",
        "            total += labels.size(0)\n",
        "            correct += (pred == labels.data).sum().item()\n",
        "            accuracy = correct / total\n",
        "\n",
        "        test_acc = test(net, test_loader)\n",
        "        print(\"Test acc: \", test_acc)\n",
        "        train_accs.append(accuracy)\n",
        "        test_accs.append(test_acc)\n",
        "    return train_accs, test_accs"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 42,
      "metadata": {
        "id": "MADQb81UijK7",
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 205,
          "referenced_widgets": [
            "9cd42a082e2c486c9103936a5feeb9dd",
            "89121e8de88149819740e86d77dacb65",
            "79e5d20fc6184a8a89321fcfc9b9bb0a",
            "7a9e7b8ff5f043a48e351ade72dee629",
            "96cbc923b2e94a069e5428e492c3c87d",
            "1259ed5bc6b54c49b24409eb45d3b4c7",
            "d429bce01e794576adaf1f7cbe2f117c",
            "49e17d4e1a9a48b4aac9c897987e817f",
            "0722ef4bbe454f8fa63bb6a8a26c10f8",
            "5a6e379606814bec945f65da30bb573d",
            "c3714160355d475b8c5f24835518100c"
          ]
        },
        "outputId": "9021a2ff-1fa1-4558-857c-e67b2653d8cd"
      },
      "outputs": [
        {
          "output_type": "stream",
          "name": "stdout",
          "text": [
            "Downloading https://www.cs.toronto.edu/~kriz/cifar-10-python.tar.gz to data/cifar-10-python.tar.gz\n"
          ]
        },
        {
          "output_type": "display_data",
          "data": {
            "text/plain": [
              "  0%|          | 0/170498071 [00:00<?, ?it/s]"
            ],
            "application/vnd.jupyter.widget-view+json": {
              "version_major": 2,
              "version_minor": 0,
              "model_id": "9cd42a082e2c486c9103936a5feeb9dd"
            }
          },
          "metadata": {}
        },
        {
          "output_type": "stream",
          "name": "stdout",
          "text": [
            "Extracting data/cifar-10-python.tar.gz to data/\n",
            "Files already downloaded and verified\n",
            "0\n",
            "Test acc:  0.3399\n",
            "1\n",
            "Test acc:  0.3675\n",
            "2\n",
            "Test acc:  0.3762\n"
          ]
        }
      ],
      "source": [
        "train_loader, test_loader = get_data(data_augment=False, data_normalize=False)\n",
        "train_accs, test_accs = train_model(train_loader, test_loader, epochs=3)"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 43,
      "metadata": {
        "id": "UnYCP-REikzs",
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "outputId": "13697152-b3de-4ec1-8024-09753961d1c7"
      },
      "outputs": [
        {
          "output_type": "stream",
          "name": "stdout",
          "text": [
            "Files already downloaded and verified\n",
            "Files already downloaded and verified\n",
            "0\n",
            "Test acc:  0.3602\n",
            "1\n",
            "Test acc:  0.4164\n",
            "2\n",
            "Test acc:  0.4418\n"
          ]
        }
      ],
      "source": [
        "train_loader, test_loader = get_data(data_augment=False, data_normalize=True)\n",
        "normalize_train_accs, normalize_test_accs = train_model(train_loader, test_loader, epochs=3)"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 44,
      "metadata": {
        "id": "aRwFghLzimSJ",
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 295
        },
        "outputId": "6eb21aa1-b268-4e51-942b-417b87372fb0"
      },
      "outputs": [
        {
          "output_type": "display_data",
          "data": {
            "text/plain": [
              "<Figure size 432x288 with 1 Axes>"
            ],
            "image/png": "iVBORw0KGgoAAAANSUhEUgAAAY4AAAEWCAYAAABxMXBSAAAAOXRFWHRTb2Z0d2FyZQBNYXRwbG90bGliIHZlcnNpb24zLjUuMywgaHR0cHM6Ly9tYXRwbG90bGliLm9yZy/NK7nSAAAACXBIWXMAAAsTAAALEwEAmpwYAAA9jklEQVR4nO3de5zN1frA8c9jEkOkUioUQsLIZZpKEbrpEDqllA66SaWkk1KdU9LlVzjlVMpRp1RHcbqJLkeuRXIZGvcwBjWo3C+5jZnn98f67pk9Yy57j/2dPZfn/Xrtl/1d38t+9p5tnllrfddaoqoYY4wxoSoX7QCMMcaULJY4jDHGhMUShzHGmLBY4jDGGBMWSxzGGGPCYonDGGNMWCxxGJMHEflaRHpHOw5jihuxcRymNBGRfUGblYBDQLq3fbeqjivieGYB5wOnq+qhonxtY/xiNQ5TqqjqCYEH8DNwbVBZZtIQkeP8jkVE6gBtAAW6+P16OV7b9/dnyi5LHKZMEJF2IpIqIo+KyK/AOyJykoh8ISJbRWSn97xW0DmzRORO73kfEZkjIiO8Y9eLyDUFvGwvYB4wFsjW5CUitUXkU++1t4vIa0H77hKRVSKyV0RWikhLr1xFpH7QcWNF5NljeH8ni8g7IrLZ2z/RK18uItcGHVdeRLaJSIvwPnVTWlniMGXJ6cDJwNlAX9z3/x1v+yzgAPBanmfDhcBqoDowDPi3iEg+x/cCxnmPq0WkBoCIxABfABuBOkBNYLy3rzswxDu3Kq6mst2n9/c+rjmvCXAa8LJX/h5wa9BxfwK2qOqPIcZhSjtVtYc9SuUD2ABc4T1vBxwGKuZzfHNgZ9D2LOBO73kfIDloXyVcE9TpeVzrUiANqO5t/wQM9J5fDGwFjsvlvCnAgDyuqUD9oO2xwLOFeX/AGUAGcFIux50J7AWqetsfA49E++dpj+LzsBqHKUu2qurBwIaIVBKRf4nIRhHZA3wHVPNqBLn5NfBEVfd7T0/I49jewDequs3b/oCs5qrawEZVPZLLebWBdaG9naOE8/5qAztUdWfOi6jqZuB74HoRqQZcg6s1GQOAdaCZsiTnLYR/Bc4FLlTVX0WkOfAjkF/zU4FEJBa4EYjx+hsAKuB+aZ8P/AKcJSLH5ZI8fgHOyePS+3E1nYDTgdSg7XDe3y/AySJSTVV35fJa7wJ34n5H/KCqm/J6v6bssRqHKcuq4Nr9d4nIycBTEbpuN9wtwI1xzUPNgfOA2bi+iwXAFuAFEaksIhVF5BLv3LeAh0WklTj1ReRsb18ScIuIxIhIR+Cywr4/Vd0CfA287nWilxeRtkHnTgRaAgNwfR7GZLLEYcqykUAssA1399P/InTd3sA7qvqzqv4aeOA6pnvi/uK/FqiPu2U4FbgJQFU/Ap7DNW3txf0CP9m77gDvvF3edSYWEMdI8n9/f8H1w/wE/A48GNihqgeAT4C6wKchv3NTJtgAQGNMrkTkSaChqt5a4MGmTLE+DmPMUbymrTtwtRJjsrGmKmNMNiJyF67z/GtV/S7a8Zjix5qqjDHGhMVqHMYYY8JSJvo4qlevrnXq1Il2GMYYU6IsWrRom6qemrO8TCSOOnXqkJiYGO0wjDGmRBGRjbmVW1OVMcaYsFjiMMYYExZLHMYYY8JiicMYY0xYLHEYY4wJiyUOY4wxYbHEYYwxJixlYhyHMcaUFQcPwqpVsHw5LFsGTz0FlStH9jUscRhjTAmUng7r1rnkEEgSy5fD2rWQkeGOOf54+MtfIC4usq9ticMYY4oxVdi0KXtyWL4cVq50tQsAETjnHGjaFG680f0bFwf160P58pGPyRKHMcYUEzt3Zk8QgX937co65owzXFK4996sBHHeeZFvjsqPJQ5jjCliBw64GkPOJLF5c9YxJ57oEsNNN7nk0LSpe5xySvTiDrDEYYwxPjlyBJKTj65BJCe7JiiAChVcjeHyy7NqEE2bQq1argmqOLLEYYwxx0gVfvklq/8hkCBWrYJDh9wx5cq5Poe4OLjllqwaRP36cFwJ+03sa7gi0hH4JxADvKWqL+Rx3PXAx8AFqpooIlcCLwDHA4eBQao6wzt2FnAGcMA7/SpV/d3P92GMMQHbt+feD7FnT9YxNWu6pHDFFdn7IWJjoxd3JPmWOEQkBhgFXAmkAgtFZJKqrsxxXBVgADA/qHgbcK2qbhaRpsAUoGbQ/p6qagtsGGN888cfufdD/Ppr1jHVqrmk0LNn9n6Ik06KWthFws8aRwKQrKopACIyHugKrMxx3DPAi8CgQIGq/hi0fwUQKyIVVPWQj/EaY8qgtDQ39iFngkhJyeqHqFgRGjeGq6/O3g9x5pnFtx/CT34mjprAL0HbqcCFwQeISEugtqp+KSKDyN31wOIcSeMdEUkHPgGeVQ38eLNduy/QF+Css84q/LswxpQKqvDzz0c3Mf30Exw+7I4pVw4aNoQWLaBXr6waxDnnQExMdOMvTqLWJSMi5YCXgD75HNMEVxu5Kqi4p6pu8pq4PgH+AryX81xVHQOMAYiPjz8qsRhjSq+tW4+uQaxYAXv3Zh1Tu7ZLCh07ZtUiGjVytQuTPz8TxyagdtB2La8soArQFJglrq53OjBJRLp4HeS1gM+AXqq6LnCSqm7y/t0rIh/gmsSOShzGmNJv3z6XEHLezfTbb1nHnHyySwq9emXvhzjxxOjFXdL5mTgWAg1EpC4uYfQAbgnsVNXdQPXAtne31MNe0qgGfAkMVtXvg445DqimqttEpDzQGZjm43swxhQDaWmwevXRtYj167OOiY2FJk3gT3/K3g9x+ullsx/CT74lDlU9IiL9cXdExQBvq+oKERkKJKrqpHxO7w/UB54UkSe9squAP4ApXtKIwSWNN/16D8aYopWRARs3Zp+TadkylzTS0twxMTFw7rlwwQVw221ZCaJuXeuHKCqSS79yqRMfH6+JiXb3rjHFyW+/Hd3EtGKFa34KOPvs7LWHuDiXNCpUiF7cZYmILFLV+JzlJWy8ojGmpNm71yWEnHczbd2adUz16i4pBNcgmjSBqlWjF7fJmyUOY0xEHD7sbm3NOf33hg1Zx1Su7BJCly5ZndRxcXDaadYPUZJY4jDGhCUjw3VK5+yHWLPGTeoHbu6lRo3goovgzjuzahF16rixEqZks8RhjMmVquuHyNnEtGIF7N+fdVzdui4pdOuWVYNo2NCtPmdKJ0scxhh2787eDxFIFNu3Zx1z2mkuKdx1V1aCaNwYqlSJXtwmOixxGFOGHDrkpvrOeTfTzz9nHXPCCS4xXHdd9gFzp50WvbhN8WKJw5hSKD3dTdKXc8Dc2rVuH7i1qBs1gksvzX7L61lnWT+EyZ8lDmNKMFXYsuXofoiVK93ypODuVqpXzyWFG27IShINGrjkYUy4LHEYU0Ls2nV0E9OyZbBzZ9Yxp5/ukkK/ftn7ISpXjlrYphSyxGFMMXPggBsPkbMWkZqadUzVqi4xdO+evR+ievW8r2tMpFjiMCZKVN3Yh5wD5taudWMlwN3Set550K5d9n6I2rVtwJyJHkscxkTBypXQty987839LAL167ukcNNNWTWIBg3cYDpjihP7ShpThA4ehOefhxdecOMfRo50dzWddx5UqhTt6IwJjSUOY4rIrFlw992ueerWW+Gll+DUU6MdlTHhs7u1jfHZjh1wxx3Qvr2by+mbb+D99y1pmJLLEocxPlGFDz5wg+zefRcGD3ad4FdeGe3IjDk21lRljA9SUuCee1ztIiEBpk6F88+PdlTGRIbVOIyJoLQ0GD7c3RE1dy68+qr715KGKU2sxmFMhCxc6GaOXbIEunZ1SaN27WhHZUzk+VrjEJGOIrJaRJJFZHA+x10vIioi8UFlj3nnrRaRq8O9pjFFZe9eGDDALVq0dSt8+ilMnGhJw5RevtU4RCQGGAVcCaQCC0VkkqquzHFcFWAAMD+orDHQA2gCnAlME5GG3u4Cr2lMUZk8Ge69FzZtcn0azz8PJ54Y7aiM8ZefNY4EIFlVU1T1MDAe6JrLcc8ALwIHg8q6AuNV9ZCqrgeSveuFek1jfLV5s5tptksXqFbNjQAfNcqShikb/EwcNYFfgrZTvbJMItISqK2qX4Z4boHXDLp2XxFJFJHErVu3Fu4dGJNDRgaMHu1Gen/xhathLFoEF18c7ciMKTpRu6tKRMoBLwF/9eP6qjpGVeNVNf5UG2llImDFCmjTxjVJxce7MRmPPWZra5uyx8+7qjYBwd2DtbyygCpAU2CWuGk+TwcmiUiXAs7N75rGRNzBg/Dcc/Dii24683ffhb/8xWanNWWXn4ljIdBAROrifrn3AG4J7FTV3UDm6gEiMgt4WFUTReQA8IGIvITrHG8ALAAkv2saE2kzZ7r5pdaudcniH/+wqUKM8a2pSlWPAP2BKcAq4L+qukJEhnq1ivzOXQH8F1gJ/A+4T1XT87qmX+/BlF3bt8Ptt0OHDm6N7qlT4b33LGkYAyCqGu0YfBcfH6+JiYnRDsOUAIH5pR580C3V+vDD8Pe/25TnpmwSkUWqGp+z3EaOG+MJnl/qwgthzBho1izaURlT/NhcVabMS0uDYcPc/FI//OCmCvn+e0saxuTFahymTFuwwM0vtXQpdOvmkkatWtGOypjizWocpkwKnl9q2zY3v9Rnn1nSMCYUVuMwZc6kSXDffW5+qXvvdaO/q1aNdlTGlBxW4zBlxubNcP31bsrzatXcOhmvvWZJw5hwWeIwpV5GBrzxhptf6quv4P/+DxYvds1UxpjwWVOVKdWWL4e+fd3dUpdf7iYorF8/2lEZU7JZjcOUSgcPwt/+Bi1awJo1btT31KmWNIyJBKtxmFJnxgw3v1RyMvTq5eaXql694POMMaGxGocpNbZvh9tuc01Sqq6G8e67ljSMiTRLHKbEU4X//AcaNXL/PvaYWyvjiiuiHZkxpZM1VZkSbd06N7/U1KnuLqkxYyAuLtpRGVO6WY3DlEhpaW5hpaZNYd48t973nDmWNIwpClbjMCXO/PnuFtulS+G669z8UjVzXXneGOMHq3GYEmPPHnjgAbj4YtcR/tlnbo4pSxrGFC2rcZgS4fPP3fxSmze7f597zqYKMSZarMZhirVNm+DPf3ZTnp98ctZ6GZY0jIkeSxymWMrIgNdfd/NLff21m19q0SK3Mp8xJrqsqcoUO8uWuc7vefPcWIzRo+Gcc6IdlTEmwNcah4h0FJHVIpIsIoNz2d9PRJaJSJKIzBGRxl55T68s8MgQkebevlneNQP7TvPzPZiic+AAPPEEtGzppgt5/323/rclDWOKF99qHCISA4wCrgRSgYUiMklVVwYd9oGqjvaO7wK8BHRU1XHAOK88DpioqklB5/VU1US/YjdFb/p06NfPJYzevWHECJsqxJjiys8aRwKQrKopqnoYGA90DT5AVfcEbVYGNJfr3Oyda0qhbdtcoghMDzJtGowda0nDmGNy+DDMng1Dh7rnEeZnH0dN4Jeg7VTgqK5NEbkPeAg4HuiQy3VuIkfCAd4RkXTgE+BZVT0q4YhIX6AvwFlnnVWY+I2PAvNLDRwIu3fD44+7adBjY6MdmTEl0JEj8OOPbmroGTNc0jhwAETg2mvd+gIRFPXOcVUdBYwSkVuAvwG9A/tE5EJgv6ouDzqlp6puEpEquMTxF+C9XK47BhgDEB8fn1tNxkTJunWuWWraNDe/1JtvuqlDjDEhysiAFSuyEsW337q/wMD9Z7rrLujQAdq2hZNOivjL+5k4NgG1g7ZreWV5GQ+8kaOsB/BhcIGqbvL+3SsiH+CaxI5KHKb4SUtza2M8/TQcf7ybX6pfPyhnN4Ubkz9V1wEYSBQzZ8LWrW5f/fpw000uUbRrBzVq+B6On4ljIdBAROriEkYP4JbgA0Skgaqu9TY7AWuD9pUDbgTaBJUdB1RT1W0iUh7oDEzz8T2YCJk3z91iu2yZG9D3yis2VYgx+fr5Z5cgAskiNdWV16wJ11zjEkX79hCFpnjfEoeqHhGR/sAUIAZ4W1VXiMhQIFFVJwH9ReQKIA3YSVAzFdAW+EVVU4LKKgBTvKQRg0sab/r1Hsyx27PH3WI7ahSceSZMnAhdc/ZYGWPgt99g1qysRJGc7MqrV3dJIvCoX9/1XUSR5NKvXOrEx8drYqLdvVvUJk6E/v3d/FL9+8Ozz9pUIcZk2rnT9U0Emp6We125Vau6JqdAomjSJGrtuSKySFXjc5ZHvXPclD6bNsH997vZa5s1g08+salCjGHfPrdoTKBGsXix67uIjYU2beDWW12iaNECjivev5qLd3SmRElPd9ODPPZY1kJLAwdC+fLRjsyYKDh40HXuBRLF/Pnuttny5d3aAE895RJFQgJUqBDtaMNiicNExLJl7g7A+fPhyivhjTdsqhBTxhw5AomJWYni++9d8ihXDuLj4eGHXaK45BKoVCna0R4TSxzmmBw4AM88A8OHQ7VqblDfLbdEve/OGP9lZLhlKAOJ4rvvYO9et69ZM3eveWAsxYknRjfWCLPEYQpt2jT3f2PdOujTx80vdcop0Y7KGJ+owurVWYli1iy3FCVAw4bQs2fWWIpTT41mpL6zxGHCtnUr/PWvbvba+vXdBIUdcpssxpiSbsOGrEQxYwZs2eLKa9d2U3kExlLUqhXVMIuaJQ4TMlWXLB56yM1u8MQT7mHzS5lSY8uW7IPu1q935aedln0sRb16Zbo9tsDEISLXAl+qakYRxGOKqeRk1yw1fbq7IWTMGJtfypQCO3ZkH3S3apUrr1bNNTkNHOgSRePGZTpR5BRKjeMmYKSIfIIb/f2TzzGZYiQtzfVdDB3q5pd6/XW4+26bX8qUUHv3upljA4kiKclVpStXdmMpbrvNJYrmzSEmJtrRFlsFJg5VvVVEquLWxRgrIgq8A3yoqnv9DtBEz7x57hbb5cvh+uvd/FJnnhntqIwJw4ED8MMPWYliwQI34Oj446F1azfjZocOcMEFrsyEJKQ+DlXdIyIfA7HAg8B1wCAReUVVX/UxPhMFe/a49TFef93Np/b559ClS7SjMiYEaWmwcGFWopg7Fw4dcrWHhAQYPNgliosvts65YxBKH0cX4DagPm768gRV/V1EKgErAUscpchnn7l5pbZscdOGPPssVKkS7aiMyUN6OixZkn0sxR9/uP6I5s3dl7lDB9cMZV/kiAmlxnE98LKqfhdcqKr7ReQOf8IyRS011SWKiRPh/PNdAklIiHZUxuSg6jqwg8dS7Nzp9p13nhtQ1KEDXHaZDSryUSiJYwiwJbAhIrFADVXdoKrT/QrMFI30dDc9yOOPuxkTbH4pU6youltig8dS/Pab21e3rlvcJTCW4owzohtrGRJK4vgIaB20ne6VXeBLRKbILF3qFleaPx+uusolkHr1oh2VKfM2bco+lmLjRld+xhlwxRVZiaJu3ejGWYaFkjiOU9XDgQ1VPSwidvtBCXbggLu9dsQItxyxzS9lomrbtuxjKVavduUnn+wSxCOPuGRx7rn2JS0mQkkcW0Wki7diHyLSFdjmb1jGL1OnuoF8KSnulvXhw60p2BSxPXtcJ3YgUSxZ4spPOMH1TfTt6xJFs2Y2YKiYCiVx9APGichrgAC/AL18jcpEXPD8Ug0auP+v7dtHOypTJuzf726LDSSKxETXuVaxopti/LnnXKJo1co610qIUAYArgMuEpETvO19vkdlIkYV3nvPJY09e+Dvf3cd4RUrRjsyU2odPuwG2gUSxQ8/uLLjjnNLQT7+uEsUF11kX8QSKqQBgCLSCWgCVBSvjVFVh/oYl4mAtWtds9SMGW6Q7JgxbvliYyIqPd0tgxro0J4929UyRKBlSxgwwCWKSy91zVGmxAtlAOBooBLQHngLuAFYEMrFRaQj8E8gBnhLVV/Isb8fcB/uTq19QF9VXSkidYBVgNdLxjxV7eed0woYixvF/hUwQFU1lHjKisOHs+aXqlDB3S3Vt681F5sIyciAFSuyahTffuumSwb3l8kdd2SNpTjppOjGanwRSo2jtao2E5Glqvq0iPwD+Lqgk0QkBhgFXAmkAgtFZJKqrgw67ANVHe0d3wV4Cejo7Vunqs1zufQbwF3AfFzi6BhKPGXFDz+4+aVWrIAbboB//tPmlzLHSNVNjxxIFDNnuk4zcOsD33hj1i2yNWpEN1ZTJEJJHAe9f/eLyJnAdiCUkTYJQLKqpgCIyHigK26aEsDNgRV0fGUg35qDiJwBVFXVed72e0A3LHGwe7drOn7jDbemzKRJbp0ZYwrll1+yD7pLTXXlNWtCx45ZieLss6Mbp4mKUBLHZBGpBgwHFuN+ub8Zwnk1cXdgBaQCF+Y8SETuAx4CjgeC15GrKyI/AnuAv6nqbO+aqTmuWTO3FxeRvkBfgLPOOiuEcEsmVTc9yP33u/mlHnjArQFu0/KYsPz+e/ZBd8nJrrx69ewLGNWvb2MpTP6JQ0TKAdNVdRfwiYh8AVRU1d2RCkBVRwGjROQW4G9Ab9wUJ2ep6navT2OiiITVrauqY4AxAPHx8aWyDyQ11c3h9vnnbn6piRPd7NDGFGjXLtc3EUgUy5e78qpV3QJGgckBmzSxzjFzlHwTh6pmiMgooIW3fQg4FOK1NwG1g7ZreWV5GY/rv8j2Oqq6SETWAQ2984MX9y3omqVSerqb8vzxx93zYcPgwQftFniTjz/+gDlzshLF4sWukzs21s0ce+utLlG0aOFumzUmH6F8Q6aLyPXAp2HevbQQaCAidXG/3HsAtwQfICINVHWtt9kJWOuVnwrsUNV0EakHNABSVHWHiOwRkYtwneO9KGPTui9d6jq/FyyAq692fRo2ZY85yqFDbiWuQKKYP9+tVVG+vFuL4sknXaJISHC33hkThlASx924PogjInIQN3pcVbVqfiep6hER6Q9Mwd2O+7aqrhCRoUCiN4VJfxG5AkgDduKaqQDaAkNFJA3IAPqp6g5v371k3Y77NWWkY3z//qz5pU4+GT74AHr0sOZm4zlyBBYtykoUc+bAwYOumSk+3o0A7dDBjdSuVCna0ZoSTsrCEIj4+HhNTEyMdhiF9s03biDf+vVw++1ufqmTT452VCaqMjJg2bLsYyn2eis5N2uW1Zndti2ceGJ0YzUllogsUtX4nOWhDABsm1t5zoWdTORt3QoPPeRmr23Y0N300q5dtKMyUaEKa9ZkH0uxfbvb17Ah9OzpEkW7dnDqqVEN1ZR+oTRVDQp6XhE3PmMR2W+dNRGkCu++61oX9u61+aXKrI0bs4+l2LzZldeu7QbpBMZS1KqV/3WMibBQJjnMNoxMRGoDI/0KqKxbs8Y1S82c6Zqjx4yBxo2jHZUpEr/+mn0sRUqKKz/ttOxjKerVs84tE1WFue8uFTgv0oGUdYcPu76LZ55xNYvRo93dU3YLfSl24AD8739ZiWKlN6lCtWquyenBB12iaNzYEoUpVkLp43iVrKlAygHNcSPITYTMnesmIVyxArp3d/NL2fLJpdzkyW64/8aNULmyG0vRp49LFM2bQ0xMtCM0Jk+h1DiCb0c6Anyoqt/7FE+Zsns3PPaYG4tRu7b7XdK5c7SjMr5av95NMz55shuV/fXXLlkcb6sxm5IjlMTxMXBQVdPBzXorIpVUdb+/oZVeqvDpp+4Pzt9+cy0SzzxjSxWUaocOuUE4zz7rahMjRriJxWy4vymBQmlBn44bbBcQC0zzJ5zS75dfoGtXN+V5jRpuQO/LL1vSKNWmTnVjK/72N3c31E8/uVvmLGmYEiqUxFExeLlY77kNPQ1Tejq88orr55w+3XWEL1zoBvWaUmrTJrjpJrjqKlfNnDIF/vtfu33WlHihJI4/RKRlYMObrfaAfyGVPkuWuOmBBgxwq2cuXw4PP2xzyZVaaWnw0kvQqJFbGOWZZ9wo76uuinZkxkREKL+6HgQ+EpHNuHmqTgdu8jOo0mL/fnj6afjHP+CUU2x+qTJh9my4917310GnTq6aWa9etKMyJqJCGQC4UEQaAed6RatVNc3fsEq+KVPgnnvcTTR33OGmPrf5pUqx33+HRx5xQ/7PPtstjtKli/2VYEqlApuqvBX6KqvqclVdDpwgIvf6H1rJ9PvvbmmDjh3dHZazZsFbb1nSKLXS09391Oee66qUjz/uBvJ17WpJw5RaofRx3OWtAAiAqu4E7vItohJKFd55B847z/V/PvkkJCXBZZdFOzLjm4UL4aKLXNNUy5ZusZTnnrNpy02pF0riiBHJ+tNJRGJw64Mbz5o1bgzX7be7u6aSklzfhk1KWErt3OnaIS+80N059eGHMG2a6ww3pgwIJXH8D5ggIpeLyOXAh5SRxZMKcviwG8/VrBn8+KObkPDbb21SwlIrIwPGjnXNUm++6W6T++knu+PBlDmh3FX1KNAX6OdtL8XdWVWmff+9m19q5Uq48UYYOdLmlyrVli51TVLffw+tW7tF388/P9pRGRMVBdY4VDUDt773BtxaHB2AVf6GVXzt2uVaKS69FPbtgy++gAkTLGmUWnv3utW0WraE1avh7bfdLbeWNEwZlmeNQ0QaAjd7j23ABABVbV80oRUvqvDJJ25+qd9/h4ED3RrgNlVIKaXq7nJ46CHYssVVL59/3m6PM4b8axw/4WoXnVX1UlV9FUgP5+Ii0lFEVotIsogMzmV/PxFZJiJJIjJHRBp75VeKyCJv3yIR6RB0zizvmkne47RwYiqMn392t+R37+5qFgsWuIHBljRKqdWr3SjvHj3cD3z+fLdAiiUNY4D8E8efgS3ATBF50+sYD7kH0Lv7ahRwDdAYuDmQGIJ8oKpxqtocGAa85JVvA65V1TigN/B+jvN6qmpz7/F7qDGFKz3drY3RuLFbZ2fECJc0WrXy6xVNVO3fD088AXFx7lbbUaNc0rjggmhHZkyxkmdTlapOBCaKSGWgK27qkdNE5A3gM1X9poBrJwDJqpoCICLjveusDHqNPUHHV8ZbMEpVfwwqXwHEikgFVT0U4vuKiGuvdcslXHON6wutU6coX90UqUmT3DTnGzdCr15uqH+NGtGOyphiKZTO8T9U9QNv7fFawI+4O60KUhP4JWg71SvLRkTuE5F1uBrHA7lc53pgcY6k8Y7XTPX34DEmkXbbbTB+PHz5pSWNUmv9evcXQteuru3x22/dtCGWNIzJU1grWqvqTlUdo6qXRyoAVR2lqufgktHfgveJSBPgReDuoOKeXhNWG+/xl9yuKyJ9RSRRRBK3bt1aqNi6d3ezYtst+qXQoUNuEE7jxm5emBEj3GCctm2jHZkxxV5YiSNMm4DaQdu1vLK8jAe6BTZEpBbwGdBLVdcFylV1k/fvXuADXJPYUbwEF6+q8aeeemph34MpjaZOdf0Yf/+7q22sWmULKxkTBj8Tx0KggYjUFZHjgR7ApOADRKRB0GYnYK1XXg34EhgcvL65iBwnItW95+WBzsByH9+DKU2CF1YCW1jJmELybSkhVT0iIv2BKUAM8LaqrhCRoUCiqk4C+ovIFUAasBN3BxVAf6A+8KSIPOmVXQX8AUzxkkYMbgnbN/16D6aUSEtz62IMGQJHjriFlQYNggoVoh2ZMSWSqGq0Y/BdfHy8JiYmRjsMEw3BCyt17uwSSN260Y7KmBJBRBap6lELXPvZVGVM9Pz+O/Tu7Tq79+6Fzz+HyZMtaRgTAZY4TOmSnu4G3Zx7rpvuPLCwUpcu0Y7MmFLDtz4OY4rcwoVuBspFi+Dyy+G112yNDGN8YDUOU/Lt2JG1sNLmzW7U5tSpljSM8YklDlNyZWS49XoDCys9+KBbWMlGbRrjK2uqMiVT8MJKl1zi+jWaNYt2VMaUCVbjMCXLnj3ZF1Z65x347jtLGsYUIatxmJJB1S21+NBD8OuvtrCSMVFkicMUf6tXw333wfTpbjGUzz+3NTKMiSJrqjLF1/79bhxGXJy7xfb1121hJWOKAatxmOIpeGGl3r3dwkqn+b5KsDEmBFbjMMVL8MJKVaq4ju+xYy1pGFOMWOIwxcOhQ27W2uCFlRYvhjZtoh2ZMSYHa6oy0ffNN9C/P6xdCzfeCC+9BDWPWmXYGFNMWI3DRE9qqksUV1/ttqdMcbfcWtIwplizxGGKXlqaa4pq1MhNdf7MM7BsWdbKfMaYYs2aqkzR+u47N1XIihW2sJIxJZTVOEzR+O03d1vtZZfBvn22sJIxJZglDuMvW1jJmFLHmqqMfxYscM1SgYWVRo1yCcQYU6L5WuMQkY4islpEkkVkcC77+4nIMhFJEpE5ItI4aN9j3nmrReTqUK9pioEdO6BfP7joouwLK1nSMKZU8C1xiEgMMAq4BmgM3BycGDwfqGqcqjYHhgEveec2BnoATYCOwOsiEhPiNU20BC+s9NZbtrCSMaWUnzWOBCBZVVNU9TAwHugafICq7gnarAyo97wrMF5VD6nqeiDZu16B1zRRsmSJG+V9++0ucSxe7AbyVa0a7ciMMRHmZ+KoCfwStJ3qlWUjIveJyDpcjeOBAs4N6ZredfuKSKKIJG7durXQb8IUYM8eGDjQTXe+Zo0trGRMGRD1u6pUdZSqngM8Cvwtgtcdo6rxqhp/6qmnRuqyJkDV9V00agT//CfcdZdbN6NPHygX9a+VMcZHft5VtQmoHbRdyyvLy3jgjRDODeeaxg8//eQWVpoxwxZWMqYM8vNPw4VAAxGpKyLH4zq7JwUfICINgjY7AWu955OAHiJSQUTqAg2ABaFc0/jojz/cOIxmzVwfhi2sZEyZ5FuNQ1WPiEh/YAoQA7ytqitEZCiQqKqTgP4icgWQBuwEenvnrhCR/wIrgSPAfaqaDpDbNf16D8ajmrWw0s8/28JKxpRxoqoFH1XCxcfHa2JiYrTDKJlSUlzC+PJLaNrU1TJsjQxjygQRWaSq8TnLrRfT5O7gQTdrbZMm8O238I9/2MJKxhjAphwxufnmG9f5nZxsCysZY45iNQ6TJTUVund3CyuJuARiCysZY3KwxGGyL6z0xRfw7LNuYaUrr4x2ZMaYYsiaqsq64IWVrr3WDeazNTKMMfmwGkdZ9dtv0KtX9oWVJk2ypGGMKZAljrImPT1rXYzx4+GJJ2xhJWNMWKypqiyZP981Sy1eDFdcAa+9ZmtkGGPCZjWOsmDHDrj7brj4Yvj1V1fT+OYbSxrGmEKxxFGaZWTA22+7BPHvf7vpz21hJWPMMbKmqtJqyRLXLDV3LlxyiZsqxNbIMMZEgNU4Sps9e9ySrS1bwtq1trCSMSbirMZRWgQWVvrrX10/Rr9+8NxzcNJJ0Y6szElLSyM1NZWDBw9GOxRjQlKxYkVq1apF+fLlQzreEkdpsGoV9O9vCysVE6mpqVSpUoU6deog1pdkijlVZfv27aSmplI3xHFc1lRVkv3xBzz2GJx/vi2sVIwcPHiQU045xZKGKRFEhFNOOSWsGrLVOEoiVVerGDDAFlYqpixpmJIk3O+r1ThKmpQUN6fUdddB1aqu43vsWEsaxpgiY4mjpDh4EIYOtYWVTET86U9/YteuXezatYvXX389s3zWrFl07tw5Iq8xa9Ys5s6dm+u+SZMm8cILL0TkdYJNnDiRlStXRvy6kTB27Fj69+8PwOjRo3nvvffCvkbOz7Sw1zlW1lRVEkyZ4jq/k5Pd4L1//MPWyDDH5KuvvgJgw4YNvP7669x7770Rf41Zs2Zxwgkn0Lp166P2denShS4+zI82ceJEOnfuTOPGjSN63SNHjnDccZH7ddmvX79CnZfzMy3sdY6V1TiKs8DCSh07QrlyMHWqu+XWkkbJ8eCD0K5dZB8PPpjvSw4fPpxXXnkFgIEDB9KhQwcAZsyYQc+ePQGoU6cO27ZtY/Dgwaxbt47mzZszaNAgAPbt28cNN9xAo0aN6NmzJ6oKwPTp02nRogVxcXHcfvvtHDp0KNu1ABITE2nXrh0bNmxg9OjRvPzyyzRv3pzZs2dnizH4r+8+ffrwwAMP0Lp1a+rVq8fHH38MuF+Sbdu2pVOnTpx77rn069ePjIwMAE444YTMa3388cf06dOHuXPnMmnSJAYNGkTz5s1Zt25dttfM63VUlUGDBtG0aVPi4uKYMGFC5uu3adOGLl260LhxY2bNmsVll11G165dqVevHoMHD2bcuHEkJCQQFxeX+XqTJ0/mwgsvpEWLFlxxxRX89ttvR/2MhgwZwogRI9i8eTPNmzfPfMTExLBx48Zcr5HbZxq4DkBSUhIXXXQRzZo147rrrmPnzp0AtGvXjkcffZSEhAQaNmx41M+iMHxNHCLSUURWi0iyiAzOZf9DIrJSRJaKyHQROdsrby8iSUGPgyLSzds3VkTWB+1r7ud7iIq0NBg+PPvCSkuXuokJjSlAmzZtMn85JCYmsm/fPtLS0pg9ezZt27bNduwLL7zAOeecQ1JSEsOHDwfgxx9/ZOTIkaxcuZKUlBS+//57Dh48SJ8+fZgwYQLLli3jyJEjvPHGG3nGUKdOHfr168fAgQNJSkqiTQFNqlu2bGHOnDl88cUXDB6c9atiwYIFvPrqq6xcuZJ169bx6aef5nmN1q1b06VLF4YPH05SUhLnnHNOSK/z6aefkpSUxJIlS5g2bRqDBg1iy5YtACxevJh//vOfrFmzBoAlS5YwevRoVq1axfvvv8+aNWtYsGABd955J6+++ioAl156KfPmzePHH3+kR48eDBs2LM+YzzzzTJKSkkhKSuKuu+7i+uuv5+yzz871GgV9pr169eLFF19k6dKlxMXF8fTTT2fuO3LkCAsWLGDkyJHZygvLt6YqEYkBRgFXAqnAQhGZpKrBDZA/AvGqul9E7gGGATep6kyguXedk4Fk4Jug8wap6sd+xR5VtrBS6TJyZJG/ZKtWrVi0aBF79uyhQoUKtGzZksTERGbPnp1ZE8lPQkICtWrVAqB58+Zs2LCBKlWqULduXRo2bAhA7969GTVqFA8WUPsJVbdu3ShXrhyNGzfO9hd6QkIC9erVA+Dmm29mzpw53HDDDRF9nTlz5nDzzTcTExNDjRo1uOyyy1i4cCFVq1YlISEh29iGCy64gDPOOAOAc845h6uuugqAuLg4Zs6cCbhxPDfddBNbtmzh8OHDIY2N+P7773nzzTeZM2dOoa6xe/dudu3axWWXXQa4n0/37t0z9//5z38G3Hdjw4YNoXxU+fKzxpEAJKtqiqoeBsYDXYMPUNWZqrrf25wH1MrlOjcAXwcdVzrZwkomQsqXL0/dunUZO3YsrVu3pk2bNsycOZPk5GTOO++8As+vUKFC5vOYmBiOHDmS7/HHHXdcZhNSYUfLB79moGkMjr5NNLAdXB7Oa+b1OnmpXLlynueXK1cuc7tcuXKZn9P9999P//79WbZsGf/6178KjG/Lli3ccccd/Pe//81sggv3GgUJxBnKzzMUfiaOmsAvQdupXlle7gC+zqW8B/BhjrLnvOatl0WkQi7nICJ9RSRRRBK3bt0aTtxFKz09a10MW1jJREibNm0YMWIEbdu2pU2bNowePZoWLVoc9Yu4SpUq7N27t8DrnXvuuWzYsIHk5GQA3n///cy/buvUqcOiRYsA+OSTT8K+dn4WLFjA+vXrycjIYMKECVx66aUA1KhRg1WrVpGRkcFnn312TK/Zpk0bJkyYQHp6Olu3buW7774jISGh0DHv3r2bml4/5LvvvpvvsWlpaXTv3p0XX3wxszaX3zXyen8nnngiJ510UmYTZfDPxw/FonNcRG4F4oHhOcrPAOKAKUHFjwGNgAuAk4FHc7umqo5R1XhVjT/11FN9ifuYzZ8PCQlw//1utPeyZa4/o1KlaEdmSrg2bdqwZcsWLr74YmrUqEHFihVz7Wc45ZRTuOSSS2jatGlm53huKlasyDvvvEP37t2Ji4ujXLlymXf0PPXUUwwYMID4+HhiYmIyz7n22mv57LPPcu0cD9UFF1xA//79Oe+886hbty7XXXcd4PpmOnfuTOvWrTObjgB69OjB8OHDadGixVGd43m57rrraNasGeeffz4dOnRg2LBhnH766YWKF1zHd/fu3WnVqhXVq1fP99i5c+eSmJjIU089ldlBvnnz5jyvkd9n+u677zJo0CCaNWtGUlISTz75ZKHfQ4FU1ZcHcDEwJWj7MeCxXI67AlgFnJbLvgHAmHxeox3wRUGxtGrVSouVbdtU+/ZVFVE980zVCRNUMzKiHZWJkJUrV0Y7hFJh5syZ2qlTp2iHUWbk9r0FEjWX36l+1jgWAg1EpK6IHI9rcpoUfICItAD+BXRR1d9zucbN5Gim8mohiKtzdwOWRz50n2RkuAWVci6sdOONtrCSMabE8O2uKlU9IiL9cc1MMcDbqrpCRIbistgkXNPUCcBHXtvrz6raBUBE6gC1gW9zXHqciJwKCJAERGcETLiSktzdUj/8AJde6iYkjIuLdlTGFFvt2rWjXbt20Q7D5MLXkeOq+hXwVY6yJ4Oe5zkwQVU3kEtnuqp2iGCI/tuzB558El59FU45xc0r1auX1TCMMSWWTTnil8DCSg895G61tYWVjDGlhCUOP6xaBffdBzNnQny8G49ha2QYY0qJYnE7bqnxxx8weLBbWOnHH+GNN2DePEsaxphSxRJHJKjCxInQuDG8+CL07AmrV7vmqaD72o0pLo51WvUnn3ySadOmRTyukSNHsn9/8Zwkok+fPpkTI955552Fmr597NixbN68OXO7sNeJNkscxyolBTp3dgsrnXgizJ4N77xjCyuZYu2rr76iWrVqRyWOUA0dOpQrfJh006/EEYlpNoK99dZbhZq6PWfiKOx1os36OArr4EG3XOvzz0P58vDSS27NjPLlox2ZKUYefNDdiR1JzZvnP3fi8OHDqVChAg888AADBw5kyZIlzJgxgxkzZvDvf/+bcePGUadOHRITE7NNq37llVfSqVOnzGnVly9fTqtWrfjPf/5z1FQlffr0oXPnztxwww3UqVOH3r17M3nyZNLS0vjoo49o1KgRQ4YMYd26dSQnJ7Nt2zYeeeQR7rrrLmbNmsWIESP44osvAOjfvz/x8fHs2bOHzZs30759e6pXr545aWBAXq+zY8cObr/9dlJSUqhUqRJjxoyhWbNmma+fkpLCWWedxbnnnsv69etJSUnh559/5uWXX2bevHl8/fXX1KxZk8mTJ1O+fHmGDh3K5MmTOXDgAK1bt+Zf//rXUe+/Xbt2mdOiB0ZoHzhwgMOHD7N+/fpcr/HJJ5+QmJhIz549iY2N5YcffuCaa65hxIgRxMfH8+GHH/L888+jqnTq1IkXX3wRcFPIDxgwgC+++ILY2Fg+//xzatSocWxfomNkNY7C+N//oGlTeOop6NbNDeIbONCShikW/JhWvSDVq1dn8eLF3HPPPZnrQwAsXbqUGTNm8MMPPzB06NBsf23n9MADD3DmmWcyc+bMo5JGfq/z1FNP0aJFC5YuXcrzzz9Pr169Mo9fuXIl06ZN48MP3TjidevWMWPGDCZNmsStt95K+/btWbZsGbGxsXz55ZeAS2QLFy5k+fLlHDhwIDPB5aZLly6Z06Kff/75PPzww3le44YbbiA+Pp5x48aRlJREbGxs5nU2b97Mo48+yowZM0hKSmLhwoVMnDgRgD/++IOLLrqIJUuW0LZtW9588838fhRFwmoc4fjlF5cgPvkEGjZ0CyvZGhkmH1GYVd2XadUDkwvmJXja7uA1M7p27UpsbCyxsbG0b9+eBQsWUK1atUK/t9xeZ86cOZmTK3bo0IHt27ezZ88ewP1iD/4Ffc0111C+fHni4uJIT0+nY8eOgJsWPTDd+MyZMxk2bBj79+9nx44dNGnShGuvvTbfuIYNG0ZsbCz33Xdfoa6xcOFC2rVrR2BevZ49e/Ldd9/RrVs3jj/++Mx+p1atWjF16tSwPjM/WOIIRVqa+w3w9NNu2pDnnoO//hUq5DoxrzFRlXNa9WbNmvk6rXrwOTmPz21a9OBp2KFw06KHGlde06KXK1eO8uXLZ8YXmBb94MGD3HvvvSQmJlK7dm2GDBlSYHzTpk3jo48+4rvvvst8P+FeIz/BcUZqWvRjZU1VBfn2W9eo/MgjcPnlbsrzxx+3pGGKtUhPq15Yn3/+OQcPHmT79u3MmjWLCy64gLPPPpuVK1dy6NAhdu3axfTp048pnjZt2jBu3DjA3RVWvXp1qlatWqh4A7/gq1evzr59+zLvosrLxo0bue+++/joo48yazb5XSOv95eQkMC3337Ltm3bSE9P58MPP/R1WvRjZTWO/Nx9N4wZA3XquEF8BVRXjSku2rRpw3PPPcfFF19M5cqVQ5pW/ZprrqFTp04RjaNZs2a0b9+ebdu28fe//50zzzwTgBtvvJGmTZtSt25dWrRokXl837596dixY2ZfRyiGDBnC7bffTrNmzahUqVKBa2Dkp1q1atx11100bdqU008/nQsKGIM1duxYtm/fTrdu3QC3FOxXX32V5zX69OlDv379MjvHA8444wxeeOEF2rdvn9k53rVr15wvV2yIhrAKVkkXHx+viYmJ4Z84fDjs3u1qGLZGhgnRqlWrQmoSKu2GDBnCCSeckNlhbIq33L63IrJIVeNzHms1jvzks7CNMcaUVZY4jDG+GDJkSLRDMD6xznFjfFAWmoBN6RHu99UShzERVrFiRbZv327Jw5QIqsr27dupWLFiyOdYU5UxEVarVi1SU1PZunVrtEMxJiQVK1bMHPQZCkscxkRYYACeMaWVNVUZY4wJiyUOY4wxYbHEYYwxJixlYuS4iGwFNhby9OrAtgiGEykWV3gsrvBYXOEprXGdraqn5iwsE4njWIhIYm5D7qPN4gqPxRUeiys8ZS0ua6oyxhgTFkscxhhjwmKJo2Bjoh1AHiyu8Fhc4bG4wlOm4rI+DmOMMWGxGocxxpiwWOIwxhgTljKdOESko4isFpFkERmcy/4KIjLB2z9fROoE7XvMK18tIlcXcVwPichKEVkqItNF5OygfekikuQ9JhVxXH1EZGvQ698ZtK+3iKz1Hr2LOK6Xg2JaIyK7gvb58nmJyNsi8ruILM9jv4jIK17MS0WkZdA+Pz+rguLq6cWzTETmisj5Qfs2eOVJIlKIJTWPKa52IrI76Gf1ZNC+fH/+Psc1KCim5d736WRvn5+fV20Rmen9HlghIgNyOca/75iqlskHEAOsA+oBxwNLgMY5jrkXGO097wFM8J439o6vANT1rhNThHG1Byp5z+8JxOVt74vi59UHeC2Xc08GUrx/T/Ken1RUceU4/n7g7SL4vNoCLYHleez/E/A1IMBFwHy/P6sQ42odeD3gmkBc3vYGoHqUPq92wBfH+vOPdFw5jr0WmFFEn9cZQEvveRVgTS7/H337jpXlGkcCkKyqKap6GBgP5Fwdvivwrvf8Y+ByERGvfLyqHlLV9UCyd70iiUtVZ6rqfm9zHhD6fMg+xpWPq4GpqrpDVXcCU4GOUYrrZuDDCL12nlT1O2BHPod0Bd5TZx5QTUTOwN/PqsC4VHWu97pQdN+tUD6vvBzL9zLScRXJdwtAVbeo6mLv+V5gFVAzx2G+fcfKcuKoCfwStJ3K0R985jGqegTYDZwS4rl+xhXsDtxfFQEVRSRRROaJSLcIxRROXNd71eKPRaR2mOf6GRdek15dYEZQsV+fV0HyitvPzypcOb9bCnwjIotEpG8U4rlYRJaIyNci0sQrKxafl4hUwv3y/SSouEg+L3FN6C2A+Tl2+fYds/U4SjARuRWIBy4LKj5bVTeJSD1ghogsU9V1RRTSZOBDVT0kInfjamsdiui1Q9ED+FhV04PKovl5FVsi0h6XOC4NKr7U+6xOA6aKyE/eX+RFYTHuZ7VPRP4ETAQaFNFrh+Ja4HtVDa6d+P55icgJuGT1oKruieS181OWaxybgNpB27W8slyPEZHjgBOB7SGe62dciMgVwBNAF1U9FChX1U3evynALNxfIkUSl6puD4rlLaBVqOf6GVeQHuRoSvDx8ypIXnH7+VmFRESa4X5+XVV1e6A86LP6HfiMyDXPFkhV96jqPu/5V0B5EalOMfi8PPl9t3z5vESkPC5pjFPVT3M5xL/vmB8dNyXhgattpeCaLgKdak1yHHMf2TvH/+s9b0L2zvEUItc5HkpcLXAdgg1ylJ8EVPCeVwfWEqGOwhDjOiPo+XXAPM3qjFvvxXeS9/zkoorLO64RrrNSiuLz8q5Zh7w7ezuRveNygd+fVYhxnYXrs2udo7wyUCXo+VygYxHGdXrgZ4f7Bfyz99mF9PP3Ky5v/4m4fpDKRfV5ee/9PWBkPsf49h2L2IdbEh+4uw7W4H4JP+GVDcX9FQ9QEfjI+4+0AKgXdO4T3nmrgWuKOK5pwG9AkveY5JW3BpZ5/3mWAXcUcVz/B6zwXn8m0Cjo3Nu9zzEZuK0o4/K2hwAv5DjPt88L99fnFiAN14Z8B9AP6OftF2CUF/MyIL6IPquC4noL2Bn03Ur0yut5n9MS72f8RBHH1T/ouzWPoMSW28+/qOLyjumDu1km+Dy/P69LcX0oS4N+Vn8qqu+YTTlijDEmLGW5j8MYY0whWOIwxhgTFkscxhhjwmKJwxhjTFgscRhjjAmLJQ5jCinHzLpJkZyZVUTq5DUjqzHRZlOOGFN4B1S1ebSDMKaoWY3DmAjz1mEY5q3FsEBE6nvldURkhmSto3KWV15DRD7zJvBbIiKtvUvFiMib3noL34hIrHf8A5K1Hsv4KL1NU4ZZ4jCm8GJzNFXdFLRvt6rGAa8BI72yV4F3VbUZMA54xSt/BfhWVc/Hrf2wwitvAIxS1SbALuB6r3ww0MK7Tj9/3poxebOR48YUkojsU9UTcinfAHRQ1RRvIrpfVfUUEdmGm88rzSvfoqrVRWQrUEuDJqv0psqeqqoNvO1HgfKq+qyI/A/Yh5shdqJ6k/8ZU1SsxmGMPzSP5+E4FPQ8naw+yU64OYhaAgu9mZuNKTKWOIzxx01B//7gPZ+Lm2UZoCcw23s+HbcEMCISIyIn5nVRESkH1FbVmcCjuJlZj6r1GOMn+0vFmMKLFZGkoO3/qWrgltyTRGQprtZws1d2P/COiAwCtgK3eeUDgDEicgeuZnEPbkbW3MQA//GSiwCvqOquCL0fY0JifRzGRJjXxxGvqtuiHYsxfrCmKmOMMWGxGocxxpiwWI3DGGNMWCxxGGOMCYslDmOMMWGxxGGMMSYsljiMMcaE5f8BZytD7Mi85ucAAAAASUVORK5CYII=\n"
          },
          "metadata": {
            "needs_background": "light"
          }
        }
      ],
      "source": [
        "fig, ax = plt.subplots()\n",
        "epochs = 3\n",
        "ax.plot(range(epochs), train_accs, c=\"red\", label=\"without input normalization\")\n",
        "ax.plot(range(epochs), normalize_train_accs, c=\"blue\", label=\"with input normalization\")\n",
        "ax.legend()\n",
        "ax.set_xlabel(\"Epochs\")\n",
        "ax.set_ylabel(\"Accuracy\")\n",
        "ax.set_title(\"Train Accuracy\")\n",
        "fig.show()"
      ]
    }
  ],
  "metadata": {
    "colab": {
      "provenance": [],
      "toc_visible": true
    },
    "kernelspec": {
      "display_name": "Python 3",
      "language": "python",
      "name": "python3"
    },
    "language_info": {
      "codemirror_mode": {
        "name": "ipython",
        "version": 3
      },
      "file_extension": ".py",
      "mimetype": "text/x-python",
      "name": "python",
      "nbconvert_exporter": "python",
      "pygments_lexer": "ipython3",
      "version": "3.8.6"
    },
    "accelerator": "GPU",
    "widgets": {
      "application/vnd.jupyter.widget-state+json": {
        "4e110a04fa804028ac51cb93e91b3918": {
          "model_module": "@jupyter-widgets/controls",
          "model_name": "HBoxModel",
          "model_module_version": "1.5.0",
          "state": {
            "_dom_classes": [],
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "HBoxModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/controls",
            "_view_module_version": "1.5.0",
            "_view_name": "HBoxView",
            "box_style": "",
            "children": [
              "IPY_MODEL_d619b748aea0474084b1bf6bf02106d9",
              "IPY_MODEL_60609425f5694effaeaccce1e8fbfcef",
              "IPY_MODEL_47432ea2dc86471d93cf86efcce67706"
            ],
            "layout": "IPY_MODEL_85836827ca4d47498a72d309772bf7ac"
          }
        },
        "d619b748aea0474084b1bf6bf02106d9": {
          "model_module": "@jupyter-widgets/controls",
          "model_name": "HTMLModel",
          "model_module_version": "1.5.0",
          "state": {
            "_dom_classes": [],
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "HTMLModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/controls",
            "_view_module_version": "1.5.0",
            "_view_name": "HTMLView",
            "description": "",
            "description_tooltip": null,
            "layout": "IPY_MODEL_328c6754a97b42318a08e64b5d96c8f3",
            "placeholder": "​",
            "style": "IPY_MODEL_7a23052817d5472f9dc8a156b65285d0",
            "value": "100%"
          }
        },
        "60609425f5694effaeaccce1e8fbfcef": {
          "model_module": "@jupyter-widgets/controls",
          "model_name": "FloatProgressModel",
          "model_module_version": "1.5.0",
          "state": {
            "_dom_classes": [],
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "FloatProgressModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/controls",
            "_view_module_version": "1.5.0",
            "_view_name": "ProgressView",
            "bar_style": "success",
            "description": "",
            "description_tooltip": null,
            "layout": "IPY_MODEL_94ad419e14e84460b6f87ad87dd554db",
            "max": 9912422,
            "min": 0,
            "orientation": "horizontal",
            "style": "IPY_MODEL_d87c3e20cffb45499b60578d0c61472b",
            "value": 9912422
          }
        },
        "47432ea2dc86471d93cf86efcce67706": {
          "model_module": "@jupyter-widgets/controls",
          "model_name": "HTMLModel",
          "model_module_version": "1.5.0",
          "state": {
            "_dom_classes": [],
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "HTMLModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/controls",
            "_view_module_version": "1.5.0",
            "_view_name": "HTMLView",
            "description": "",
            "description_tooltip": null,
            "layout": "IPY_MODEL_e28753ae72d24f228eaed71d5b6cc914",
            "placeholder": "​",
            "style": "IPY_MODEL_f3d4797e9fbf42669609893ba72f8559",
            "value": " 9912422/9912422 [00:00&lt;00:00, 20270267.65it/s]"
          }
        },
        "85836827ca4d47498a72d309772bf7ac": {
          "model_module": "@jupyter-widgets/base",
          "model_name": "LayoutModel",
          "model_module_version": "1.2.0",
          "state": {
            "_model_module": "@jupyter-widgets/base",
            "_model_module_version": "1.2.0",
            "_model_name": "LayoutModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "LayoutView",
            "align_content": null,
            "align_items": null,
            "align_self": null,
            "border": null,
            "bottom": null,
            "display": null,
            "flex": null,
            "flex_flow": null,
            "grid_area": null,
            "grid_auto_columns": null,
            "grid_auto_flow": null,
            "grid_auto_rows": null,
            "grid_column": null,
            "grid_gap": null,
            "grid_row": null,
            "grid_template_areas": null,
            "grid_template_columns": null,
            "grid_template_rows": null,
            "height": null,
            "justify_content": null,
            "justify_items": null,
            "left": null,
            "margin": null,
            "max_height": null,
            "max_width": null,
            "min_height": null,
            "min_width": null,
            "object_fit": null,
            "object_position": null,
            "order": null,
            "overflow": null,
            "overflow_x": null,
            "overflow_y": null,
            "padding": null,
            "right": null,
            "top": null,
            "visibility": null,
            "width": null
          }
        },
        "328c6754a97b42318a08e64b5d96c8f3": {
          "model_module": "@jupyter-widgets/base",
          "model_name": "LayoutModel",
          "model_module_version": "1.2.0",
          "state": {
            "_model_module": "@jupyter-widgets/base",
            "_model_module_version": "1.2.0",
            "_model_name": "LayoutModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "LayoutView",
            "align_content": null,
            "align_items": null,
            "align_self": null,
            "border": null,
            "bottom": null,
            "display": null,
            "flex": null,
            "flex_flow": null,
            "grid_area": null,
            "grid_auto_columns": null,
            "grid_auto_flow": null,
            "grid_auto_rows": null,
            "grid_column": null,
            "grid_gap": null,
            "grid_row": null,
            "grid_template_areas": null,
            "grid_template_columns": null,
            "grid_template_rows": null,
            "height": null,
            "justify_content": null,
            "justify_items": null,
            "left": null,
            "margin": null,
            "max_height": null,
            "max_width": null,
            "min_height": null,
            "min_width": null,
            "object_fit": null,
            "object_position": null,
            "order": null,
            "overflow": null,
            "overflow_x": null,
            "overflow_y": null,
            "padding": null,
            "right": null,
            "top": null,
            "visibility": null,
            "width": null
          }
        },
        "7a23052817d5472f9dc8a156b65285d0": {
          "model_module": "@jupyter-widgets/controls",
          "model_name": "DescriptionStyleModel",
          "model_module_version": "1.5.0",
          "state": {
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "DescriptionStyleModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "StyleView",
            "description_width": ""
          }
        },
        "94ad419e14e84460b6f87ad87dd554db": {
          "model_module": "@jupyter-widgets/base",
          "model_name": "LayoutModel",
          "model_module_version": "1.2.0",
          "state": {
            "_model_module": "@jupyter-widgets/base",
            "_model_module_version": "1.2.0",
            "_model_name": "LayoutModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "LayoutView",
            "align_content": null,
            "align_items": null,
            "align_self": null,
            "border": null,
            "bottom": null,
            "display": null,
            "flex": null,
            "flex_flow": null,
            "grid_area": null,
            "grid_auto_columns": null,
            "grid_auto_flow": null,
            "grid_auto_rows": null,
            "grid_column": null,
            "grid_gap": null,
            "grid_row": null,
            "grid_template_areas": null,
            "grid_template_columns": null,
            "grid_template_rows": null,
            "height": null,
            "justify_content": null,
            "justify_items": null,
            "left": null,
            "margin": null,
            "max_height": null,
            "max_width": null,
            "min_height": null,
            "min_width": null,
            "object_fit": null,
            "object_position": null,
            "order": null,
            "overflow": null,
            "overflow_x": null,
            "overflow_y": null,
            "padding": null,
            "right": null,
            "top": null,
            "visibility": null,
            "width": null
          }
        },
        "d87c3e20cffb45499b60578d0c61472b": {
          "model_module": "@jupyter-widgets/controls",
          "model_name": "ProgressStyleModel",
          "model_module_version": "1.5.0",
          "state": {
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "ProgressStyleModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "StyleView",
            "bar_color": null,
            "description_width": ""
          }
        },
        "e28753ae72d24f228eaed71d5b6cc914": {
          "model_module": "@jupyter-widgets/base",
          "model_name": "LayoutModel",
          "model_module_version": "1.2.0",
          "state": {
            "_model_module": "@jupyter-widgets/base",
            "_model_module_version": "1.2.0",
            "_model_name": "LayoutModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "LayoutView",
            "align_content": null,
            "align_items": null,
            "align_self": null,
            "border": null,
            "bottom": null,
            "display": null,
            "flex": null,
            "flex_flow": null,
            "grid_area": null,
            "grid_auto_columns": null,
            "grid_auto_flow": null,
            "grid_auto_rows": null,
            "grid_column": null,
            "grid_gap": null,
            "grid_row": null,
            "grid_template_areas": null,
            "grid_template_columns": null,
            "grid_template_rows": null,
            "height": null,
            "justify_content": null,
            "justify_items": null,
            "left": null,
            "margin": null,
            "max_height": null,
            "max_width": null,
            "min_height": null,
            "min_width": null,
            "object_fit": null,
            "object_position": null,
            "order": null,
            "overflow": null,
            "overflow_x": null,
            "overflow_y": null,
            "padding": null,
            "right": null,
            "top": null,
            "visibility": null,
            "width": null
          }
        },
        "f3d4797e9fbf42669609893ba72f8559": {
          "model_module": "@jupyter-widgets/controls",
          "model_name": "DescriptionStyleModel",
          "model_module_version": "1.5.0",
          "state": {
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "DescriptionStyleModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "StyleView",
            "description_width": ""
          }
        },
        "21a31e5b9ea5408f88dfb2c7c3f48669": {
          "model_module": "@jupyter-widgets/controls",
          "model_name": "HBoxModel",
          "model_module_version": "1.5.0",
          "state": {
            "_dom_classes": [],
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "HBoxModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/controls",
            "_view_module_version": "1.5.0",
            "_view_name": "HBoxView",
            "box_style": "",
            "children": [
              "IPY_MODEL_a1f03502f4e34eb8852b1332037e487f",
              "IPY_MODEL_7bbc448f2c3847e1bbada58144f5e14b",
              "IPY_MODEL_23a1a90003a84e39977a4401cd01f9ee"
            ],
            "layout": "IPY_MODEL_8cfeddd313f041ff92ddadc5c53114c9"
          }
        },
        "a1f03502f4e34eb8852b1332037e487f": {
          "model_module": "@jupyter-widgets/controls",
          "model_name": "HTMLModel",
          "model_module_version": "1.5.0",
          "state": {
            "_dom_classes": [],
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "HTMLModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/controls",
            "_view_module_version": "1.5.0",
            "_view_name": "HTMLView",
            "description": "",
            "description_tooltip": null,
            "layout": "IPY_MODEL_8b5c927c5288473e93419093f9ba120d",
            "placeholder": "​",
            "style": "IPY_MODEL_bd112bcf89cb427ca7dd5d3acd1bb62a",
            "value": "100%"
          }
        },
        "7bbc448f2c3847e1bbada58144f5e14b": {
          "model_module": "@jupyter-widgets/controls",
          "model_name": "FloatProgressModel",
          "model_module_version": "1.5.0",
          "state": {
            "_dom_classes": [],
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "FloatProgressModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/controls",
            "_view_module_version": "1.5.0",
            "_view_name": "ProgressView",
            "bar_style": "success",
            "description": "",
            "description_tooltip": null,
            "layout": "IPY_MODEL_0276c153bc524bc0ae25027610cf9b86",
            "max": 28881,
            "min": 0,
            "orientation": "horizontal",
            "style": "IPY_MODEL_a19525b4396b4e5f9dbf1f4f7798fa43",
            "value": 28881
          }
        },
        "23a1a90003a84e39977a4401cd01f9ee": {
          "model_module": "@jupyter-widgets/controls",
          "model_name": "HTMLModel",
          "model_module_version": "1.5.0",
          "state": {
            "_dom_classes": [],
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "HTMLModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/controls",
            "_view_module_version": "1.5.0",
            "_view_name": "HTMLView",
            "description": "",
            "description_tooltip": null,
            "layout": "IPY_MODEL_4c89c4f392d44c4ba3baaca4a1c19e66",
            "placeholder": "​",
            "style": "IPY_MODEL_7eb7cc40b022495a8788956f228d5948",
            "value": " 28881/28881 [00:00&lt;00:00, 1278004.89it/s]"
          }
        },
        "8cfeddd313f041ff92ddadc5c53114c9": {
          "model_module": "@jupyter-widgets/base",
          "model_name": "LayoutModel",
          "model_module_version": "1.2.0",
          "state": {
            "_model_module": "@jupyter-widgets/base",
            "_model_module_version": "1.2.0",
            "_model_name": "LayoutModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "LayoutView",
            "align_content": null,
            "align_items": null,
            "align_self": null,
            "border": null,
            "bottom": null,
            "display": null,
            "flex": null,
            "flex_flow": null,
            "grid_area": null,
            "grid_auto_columns": null,
            "grid_auto_flow": null,
            "grid_auto_rows": null,
            "grid_column": null,
            "grid_gap": null,
            "grid_row": null,
            "grid_template_areas": null,
            "grid_template_columns": null,
            "grid_template_rows": null,
            "height": null,
            "justify_content": null,
            "justify_items": null,
            "left": null,
            "margin": null,
            "max_height": null,
            "max_width": null,
            "min_height": null,
            "min_width": null,
            "object_fit": null,
            "object_position": null,
            "order": null,
            "overflow": null,
            "overflow_x": null,
            "overflow_y": null,
            "padding": null,
            "right": null,
            "top": null,
            "visibility": null,
            "width": null
          }
        },
        "8b5c927c5288473e93419093f9ba120d": {
          "model_module": "@jupyter-widgets/base",
          "model_name": "LayoutModel",
          "model_module_version": "1.2.0",
          "state": {
            "_model_module": "@jupyter-widgets/base",
            "_model_module_version": "1.2.0",
            "_model_name": "LayoutModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "LayoutView",
            "align_content": null,
            "align_items": null,
            "align_self": null,
            "border": null,
            "bottom": null,
            "display": null,
            "flex": null,
            "flex_flow": null,
            "grid_area": null,
            "grid_auto_columns": null,
            "grid_auto_flow": null,
            "grid_auto_rows": null,
            "grid_column": null,
            "grid_gap": null,
            "grid_row": null,
            "grid_template_areas": null,
            "grid_template_columns": null,
            "grid_template_rows": null,
            "height": null,
            "justify_content": null,
            "justify_items": null,
            "left": null,
            "margin": null,
            "max_height": null,
            "max_width": null,
            "min_height": null,
            "min_width": null,
            "object_fit": null,
            "object_position": null,
            "order": null,
            "overflow": null,
            "overflow_x": null,
            "overflow_y": null,
            "padding": null,
            "right": null,
            "top": null,
            "visibility": null,
            "width": null
          }
        },
        "bd112bcf89cb427ca7dd5d3acd1bb62a": {
          "model_module": "@jupyter-widgets/controls",
          "model_name": "DescriptionStyleModel",
          "model_module_version": "1.5.0",
          "state": {
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "DescriptionStyleModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "StyleView",
            "description_width": ""
          }
        },
        "0276c153bc524bc0ae25027610cf9b86": {
          "model_module": "@jupyter-widgets/base",
          "model_name": "LayoutModel",
          "model_module_version": "1.2.0",
          "state": {
            "_model_module": "@jupyter-widgets/base",
            "_model_module_version": "1.2.0",
            "_model_name": "LayoutModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "LayoutView",
            "align_content": null,
            "align_items": null,
            "align_self": null,
            "border": null,
            "bottom": null,
            "display": null,
            "flex": null,
            "flex_flow": null,
            "grid_area": null,
            "grid_auto_columns": null,
            "grid_auto_flow": null,
            "grid_auto_rows": null,
            "grid_column": null,
            "grid_gap": null,
            "grid_row": null,
            "grid_template_areas": null,
            "grid_template_columns": null,
            "grid_template_rows": null,
            "height": null,
            "justify_content": null,
            "justify_items": null,
            "left": null,
            "margin": null,
            "max_height": null,
            "max_width": null,
            "min_height": null,
            "min_width": null,
            "object_fit": null,
            "object_position": null,
            "order": null,
            "overflow": null,
            "overflow_x": null,
            "overflow_y": null,
            "padding": null,
            "right": null,
            "top": null,
            "visibility": null,
            "width": null
          }
        },
        "a19525b4396b4e5f9dbf1f4f7798fa43": {
          "model_module": "@jupyter-widgets/controls",
          "model_name": "ProgressStyleModel",
          "model_module_version": "1.5.0",
          "state": {
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "ProgressStyleModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "StyleView",
            "bar_color": null,
            "description_width": ""
          }
        },
        "4c89c4f392d44c4ba3baaca4a1c19e66": {
          "model_module": "@jupyter-widgets/base",
          "model_name": "LayoutModel",
          "model_module_version": "1.2.0",
          "state": {
            "_model_module": "@jupyter-widgets/base",
            "_model_module_version": "1.2.0",
            "_model_name": "LayoutModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "LayoutView",
            "align_content": null,
            "align_items": null,
            "align_self": null,
            "border": null,
            "bottom": null,
            "display": null,
            "flex": null,
            "flex_flow": null,
            "grid_area": null,
            "grid_auto_columns": null,
            "grid_auto_flow": null,
            "grid_auto_rows": null,
            "grid_column": null,
            "grid_gap": null,
            "grid_row": null,
            "grid_template_areas": null,
            "grid_template_columns": null,
            "grid_template_rows": null,
            "height": null,
            "justify_content": null,
            "justify_items": null,
            "left": null,
            "margin": null,
            "max_height": null,
            "max_width": null,
            "min_height": null,
            "min_width": null,
            "object_fit": null,
            "object_position": null,
            "order": null,
            "overflow": null,
            "overflow_x": null,
            "overflow_y": null,
            "padding": null,
            "right": null,
            "top": null,
            "visibility": null,
            "width": null
          }
        },
        "7eb7cc40b022495a8788956f228d5948": {
          "model_module": "@jupyter-widgets/controls",
          "model_name": "DescriptionStyleModel",
          "model_module_version": "1.5.0",
          "state": {
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "DescriptionStyleModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "StyleView",
            "description_width": ""
          }
        },
        "ac9ed86d3d524b78bfbeb00c6aca9537": {
          "model_module": "@jupyter-widgets/controls",
          "model_name": "HBoxModel",
          "model_module_version": "1.5.0",
          "state": {
            "_dom_classes": [],
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "HBoxModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/controls",
            "_view_module_version": "1.5.0",
            "_view_name": "HBoxView",
            "box_style": "",
            "children": [
              "IPY_MODEL_0d7b21754f7d4cf89489f9c4be22a9d8",
              "IPY_MODEL_adf998d250ff47dfa581f6a82cefea92",
              "IPY_MODEL_20cefde820e84fb1bf60dc74d5bc6800"
            ],
            "layout": "IPY_MODEL_5bbd95963f2d4096bc1dde86ab62ad4a"
          }
        },
        "0d7b21754f7d4cf89489f9c4be22a9d8": {
          "model_module": "@jupyter-widgets/controls",
          "model_name": "HTMLModel",
          "model_module_version": "1.5.0",
          "state": {
            "_dom_classes": [],
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "HTMLModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/controls",
            "_view_module_version": "1.5.0",
            "_view_name": "HTMLView",
            "description": "",
            "description_tooltip": null,
            "layout": "IPY_MODEL_3cf8e724375b4eb28a32006a77a525e8",
            "placeholder": "​",
            "style": "IPY_MODEL_aa529581902c4b239200c4ab29d694b3",
            "value": "100%"
          }
        },
        "adf998d250ff47dfa581f6a82cefea92": {
          "model_module": "@jupyter-widgets/controls",
          "model_name": "FloatProgressModel",
          "model_module_version": "1.5.0",
          "state": {
            "_dom_classes": [],
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "FloatProgressModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/controls",
            "_view_module_version": "1.5.0",
            "_view_name": "ProgressView",
            "bar_style": "success",
            "description": "",
            "description_tooltip": null,
            "layout": "IPY_MODEL_7779a750a21147f0b2f1469f84c7ee10",
            "max": 1648877,
            "min": 0,
            "orientation": "horizontal",
            "style": "IPY_MODEL_857bbe9630664a7ea71603fde979a0ef",
            "value": 1648877
          }
        },
        "20cefde820e84fb1bf60dc74d5bc6800": {
          "model_module": "@jupyter-widgets/controls",
          "model_name": "HTMLModel",
          "model_module_version": "1.5.0",
          "state": {
            "_dom_classes": [],
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "HTMLModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/controls",
            "_view_module_version": "1.5.0",
            "_view_name": "HTMLView",
            "description": "",
            "description_tooltip": null,
            "layout": "IPY_MODEL_99e19f71d71a4b30be41fab4fff940f1",
            "placeholder": "​",
            "style": "IPY_MODEL_d2fff2577d4c4c0fa1cd984b7b1f9fc5",
            "value": " 1648877/1648877 [00:00&lt;00:00, 20821904.28it/s]"
          }
        },
        "5bbd95963f2d4096bc1dde86ab62ad4a": {
          "model_module": "@jupyter-widgets/base",
          "model_name": "LayoutModel",
          "model_module_version": "1.2.0",
          "state": {
            "_model_module": "@jupyter-widgets/base",
            "_model_module_version": "1.2.0",
            "_model_name": "LayoutModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "LayoutView",
            "align_content": null,
            "align_items": null,
            "align_self": null,
            "border": null,
            "bottom": null,
            "display": null,
            "flex": null,
            "flex_flow": null,
            "grid_area": null,
            "grid_auto_columns": null,
            "grid_auto_flow": null,
            "grid_auto_rows": null,
            "grid_column": null,
            "grid_gap": null,
            "grid_row": null,
            "grid_template_areas": null,
            "grid_template_columns": null,
            "grid_template_rows": null,
            "height": null,
            "justify_content": null,
            "justify_items": null,
            "left": null,
            "margin": null,
            "max_height": null,
            "max_width": null,
            "min_height": null,
            "min_width": null,
            "object_fit": null,
            "object_position": null,
            "order": null,
            "overflow": null,
            "overflow_x": null,
            "overflow_y": null,
            "padding": null,
            "right": null,
            "top": null,
            "visibility": null,
            "width": null
          }
        },
        "3cf8e724375b4eb28a32006a77a525e8": {
          "model_module": "@jupyter-widgets/base",
          "model_name": "LayoutModel",
          "model_module_version": "1.2.0",
          "state": {
            "_model_module": "@jupyter-widgets/base",
            "_model_module_version": "1.2.0",
            "_model_name": "LayoutModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "LayoutView",
            "align_content": null,
            "align_items": null,
            "align_self": null,
            "border": null,
            "bottom": null,
            "display": null,
            "flex": null,
            "flex_flow": null,
            "grid_area": null,
            "grid_auto_columns": null,
            "grid_auto_flow": null,
            "grid_auto_rows": null,
            "grid_column": null,
            "grid_gap": null,
            "grid_row": null,
            "grid_template_areas": null,
            "grid_template_columns": null,
            "grid_template_rows": null,
            "height": null,
            "justify_content": null,
            "justify_items": null,
            "left": null,
            "margin": null,
            "max_height": null,
            "max_width": null,
            "min_height": null,
            "min_width": null,
            "object_fit": null,
            "object_position": null,
            "order": null,
            "overflow": null,
            "overflow_x": null,
            "overflow_y": null,
            "padding": null,
            "right": null,
            "top": null,
            "visibility": null,
            "width": null
          }
        },
        "aa529581902c4b239200c4ab29d694b3": {
          "model_module": "@jupyter-widgets/controls",
          "model_name": "DescriptionStyleModel",
          "model_module_version": "1.5.0",
          "state": {
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "DescriptionStyleModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "StyleView",
            "description_width": ""
          }
        },
        "7779a750a21147f0b2f1469f84c7ee10": {
          "model_module": "@jupyter-widgets/base",
          "model_name": "LayoutModel",
          "model_module_version": "1.2.0",
          "state": {
            "_model_module": "@jupyter-widgets/base",
            "_model_module_version": "1.2.0",
            "_model_name": "LayoutModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "LayoutView",
            "align_content": null,
            "align_items": null,
            "align_self": null,
            "border": null,
            "bottom": null,
            "display": null,
            "flex": null,
            "flex_flow": null,
            "grid_area": null,
            "grid_auto_columns": null,
            "grid_auto_flow": null,
            "grid_auto_rows": null,
            "grid_column": null,
            "grid_gap": null,
            "grid_row": null,
            "grid_template_areas": null,
            "grid_template_columns": null,
            "grid_template_rows": null,
            "height": null,
            "justify_content": null,
            "justify_items": null,
            "left": null,
            "margin": null,
            "max_height": null,
            "max_width": null,
            "min_height": null,
            "min_width": null,
            "object_fit": null,
            "object_position": null,
            "order": null,
            "overflow": null,
            "overflow_x": null,
            "overflow_y": null,
            "padding": null,
            "right": null,
            "top": null,
            "visibility": null,
            "width": null
          }
        },
        "857bbe9630664a7ea71603fde979a0ef": {
          "model_module": "@jupyter-widgets/controls",
          "model_name": "ProgressStyleModel",
          "model_module_version": "1.5.0",
          "state": {
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "ProgressStyleModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "StyleView",
            "bar_color": null,
            "description_width": ""
          }
        },
        "99e19f71d71a4b30be41fab4fff940f1": {
          "model_module": "@jupyter-widgets/base",
          "model_name": "LayoutModel",
          "model_module_version": "1.2.0",
          "state": {
            "_model_module": "@jupyter-widgets/base",
            "_model_module_version": "1.2.0",
            "_model_name": "LayoutModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "LayoutView",
            "align_content": null,
            "align_items": null,
            "align_self": null,
            "border": null,
            "bottom": null,
            "display": null,
            "flex": null,
            "flex_flow": null,
            "grid_area": null,
            "grid_auto_columns": null,
            "grid_auto_flow": null,
            "grid_auto_rows": null,
            "grid_column": null,
            "grid_gap": null,
            "grid_row": null,
            "grid_template_areas": null,
            "grid_template_columns": null,
            "grid_template_rows": null,
            "height": null,
            "justify_content": null,
            "justify_items": null,
            "left": null,
            "margin": null,
            "max_height": null,
            "max_width": null,
            "min_height": null,
            "min_width": null,
            "object_fit": null,
            "object_position": null,
            "order": null,
            "overflow": null,
            "overflow_x": null,
            "overflow_y": null,
            "padding": null,
            "right": null,
            "top": null,
            "visibility": null,
            "width": null
          }
        },
        "d2fff2577d4c4c0fa1cd984b7b1f9fc5": {
          "model_module": "@jupyter-widgets/controls",
          "model_name": "DescriptionStyleModel",
          "model_module_version": "1.5.0",
          "state": {
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "DescriptionStyleModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "StyleView",
            "description_width": ""
          }
        },
        "df6122ae6e9b4416a111b306e13fc2db": {
          "model_module": "@jupyter-widgets/controls",
          "model_name": "HBoxModel",
          "model_module_version": "1.5.0",
          "state": {
            "_dom_classes": [],
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "HBoxModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/controls",
            "_view_module_version": "1.5.0",
            "_view_name": "HBoxView",
            "box_style": "",
            "children": [
              "IPY_MODEL_4b5d8d8fef684333950ce3a83886737f",
              "IPY_MODEL_5cf9b8ecd2524695b538184cabe9df16",
              "IPY_MODEL_8a90fa4a020c4f19b2a4e54ffa3ecc2d"
            ],
            "layout": "IPY_MODEL_6df204f32846409c8c66e9fb046670ff"
          }
        },
        "4b5d8d8fef684333950ce3a83886737f": {
          "model_module": "@jupyter-widgets/controls",
          "model_name": "HTMLModel",
          "model_module_version": "1.5.0",
          "state": {
            "_dom_classes": [],
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "HTMLModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/controls",
            "_view_module_version": "1.5.0",
            "_view_name": "HTMLView",
            "description": "",
            "description_tooltip": null,
            "layout": "IPY_MODEL_2c1750dccfb64a5a99320b655c620ee6",
            "placeholder": "​",
            "style": "IPY_MODEL_fd255bd71f6448f59ed5635f2bafd923",
            "value": "100%"
          }
        },
        "5cf9b8ecd2524695b538184cabe9df16": {
          "model_module": "@jupyter-widgets/controls",
          "model_name": "FloatProgressModel",
          "model_module_version": "1.5.0",
          "state": {
            "_dom_classes": [],
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "FloatProgressModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/controls",
            "_view_module_version": "1.5.0",
            "_view_name": "ProgressView",
            "bar_style": "success",
            "description": "",
            "description_tooltip": null,
            "layout": "IPY_MODEL_4bdd03c2d6cc486082e65d62654e77e6",
            "max": 4542,
            "min": 0,
            "orientation": "horizontal",
            "style": "IPY_MODEL_c8ab123f08864784b9dc2a065590fa08",
            "value": 4542
          }
        },
        "8a90fa4a020c4f19b2a4e54ffa3ecc2d": {
          "model_module": "@jupyter-widgets/controls",
          "model_name": "HTMLModel",
          "model_module_version": "1.5.0",
          "state": {
            "_dom_classes": [],
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "HTMLModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/controls",
            "_view_module_version": "1.5.0",
            "_view_name": "HTMLView",
            "description": "",
            "description_tooltip": null,
            "layout": "IPY_MODEL_42ea377247934693bdc86b60e7b31a04",
            "placeholder": "​",
            "style": "IPY_MODEL_4dc6d88a399e487c91484d27029c3e2e",
            "value": " 4542/4542 [00:00&lt;00:00, 202215.59it/s]"
          }
        },
        "6df204f32846409c8c66e9fb046670ff": {
          "model_module": "@jupyter-widgets/base",
          "model_name": "LayoutModel",
          "model_module_version": "1.2.0",
          "state": {
            "_model_module": "@jupyter-widgets/base",
            "_model_module_version": "1.2.0",
            "_model_name": "LayoutModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "LayoutView",
            "align_content": null,
            "align_items": null,
            "align_self": null,
            "border": null,
            "bottom": null,
            "display": null,
            "flex": null,
            "flex_flow": null,
            "grid_area": null,
            "grid_auto_columns": null,
            "grid_auto_flow": null,
            "grid_auto_rows": null,
            "grid_column": null,
            "grid_gap": null,
            "grid_row": null,
            "grid_template_areas": null,
            "grid_template_columns": null,
            "grid_template_rows": null,
            "height": null,
            "justify_content": null,
            "justify_items": null,
            "left": null,
            "margin": null,
            "max_height": null,
            "max_width": null,
            "min_height": null,
            "min_width": null,
            "object_fit": null,
            "object_position": null,
            "order": null,
            "overflow": null,
            "overflow_x": null,
            "overflow_y": null,
            "padding": null,
            "right": null,
            "top": null,
            "visibility": null,
            "width": null
          }
        },
        "2c1750dccfb64a5a99320b655c620ee6": {
          "model_module": "@jupyter-widgets/base",
          "model_name": "LayoutModel",
          "model_module_version": "1.2.0",
          "state": {
            "_model_module": "@jupyter-widgets/base",
            "_model_module_version": "1.2.0",
            "_model_name": "LayoutModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "LayoutView",
            "align_content": null,
            "align_items": null,
            "align_self": null,
            "border": null,
            "bottom": null,
            "display": null,
            "flex": null,
            "flex_flow": null,
            "grid_area": null,
            "grid_auto_columns": null,
            "grid_auto_flow": null,
            "grid_auto_rows": null,
            "grid_column": null,
            "grid_gap": null,
            "grid_row": null,
            "grid_template_areas": null,
            "grid_template_columns": null,
            "grid_template_rows": null,
            "height": null,
            "justify_content": null,
            "justify_items": null,
            "left": null,
            "margin": null,
            "max_height": null,
            "max_width": null,
            "min_height": null,
            "min_width": null,
            "object_fit": null,
            "object_position": null,
            "order": null,
            "overflow": null,
            "overflow_x": null,
            "overflow_y": null,
            "padding": null,
            "right": null,
            "top": null,
            "visibility": null,
            "width": null
          }
        },
        "fd255bd71f6448f59ed5635f2bafd923": {
          "model_module": "@jupyter-widgets/controls",
          "model_name": "DescriptionStyleModel",
          "model_module_version": "1.5.0",
          "state": {
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "DescriptionStyleModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "StyleView",
            "description_width": ""
          }
        },
        "4bdd03c2d6cc486082e65d62654e77e6": {
          "model_module": "@jupyter-widgets/base",
          "model_name": "LayoutModel",
          "model_module_version": "1.2.0",
          "state": {
            "_model_module": "@jupyter-widgets/base",
            "_model_module_version": "1.2.0",
            "_model_name": "LayoutModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "LayoutView",
            "align_content": null,
            "align_items": null,
            "align_self": null,
            "border": null,
            "bottom": null,
            "display": null,
            "flex": null,
            "flex_flow": null,
            "grid_area": null,
            "grid_auto_columns": null,
            "grid_auto_flow": null,
            "grid_auto_rows": null,
            "grid_column": null,
            "grid_gap": null,
            "grid_row": null,
            "grid_template_areas": null,
            "grid_template_columns": null,
            "grid_template_rows": null,
            "height": null,
            "justify_content": null,
            "justify_items": null,
            "left": null,
            "margin": null,
            "max_height": null,
            "max_width": null,
            "min_height": null,
            "min_width": null,
            "object_fit": null,
            "object_position": null,
            "order": null,
            "overflow": null,
            "overflow_x": null,
            "overflow_y": null,
            "padding": null,
            "right": null,
            "top": null,
            "visibility": null,
            "width": null
          }
        },
        "c8ab123f08864784b9dc2a065590fa08": {
          "model_module": "@jupyter-widgets/controls",
          "model_name": "ProgressStyleModel",
          "model_module_version": "1.5.0",
          "state": {
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "ProgressStyleModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "StyleView",
            "bar_color": null,
            "description_width": ""
          }
        },
        "42ea377247934693bdc86b60e7b31a04": {
          "model_module": "@jupyter-widgets/base",
          "model_name": "LayoutModel",
          "model_module_version": "1.2.0",
          "state": {
            "_model_module": "@jupyter-widgets/base",
            "_model_module_version": "1.2.0",
            "_model_name": "LayoutModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "LayoutView",
            "align_content": null,
            "align_items": null,
            "align_self": null,
            "border": null,
            "bottom": null,
            "display": null,
            "flex": null,
            "flex_flow": null,
            "grid_area": null,
            "grid_auto_columns": null,
            "grid_auto_flow": null,
            "grid_auto_rows": null,
            "grid_column": null,
            "grid_gap": null,
            "grid_row": null,
            "grid_template_areas": null,
            "grid_template_columns": null,
            "grid_template_rows": null,
            "height": null,
            "justify_content": null,
            "justify_items": null,
            "left": null,
            "margin": null,
            "max_height": null,
            "max_width": null,
            "min_height": null,
            "min_width": null,
            "object_fit": null,
            "object_position": null,
            "order": null,
            "overflow": null,
            "overflow_x": null,
            "overflow_y": null,
            "padding": null,
            "right": null,
            "top": null,
            "visibility": null,
            "width": null
          }
        },
        "4dc6d88a399e487c91484d27029c3e2e": {
          "model_module": "@jupyter-widgets/controls",
          "model_name": "DescriptionStyleModel",
          "model_module_version": "1.5.0",
          "state": {
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "DescriptionStyleModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "StyleView",
            "description_width": ""
          }
        },
        "0682caa3f84a438b9a3dfb495be6b696": {
          "model_module": "@jupyter-widgets/controls",
          "model_name": "HBoxModel",
          "model_module_version": "1.5.0",
          "state": {
            "_dom_classes": [],
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "HBoxModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/controls",
            "_view_module_version": "1.5.0",
            "_view_name": "HBoxView",
            "box_style": "",
            "children": [
              "IPY_MODEL_e6fe40ad6ad545c685aa6fdb23b828a0",
              "IPY_MODEL_a80b1b2beee64d459a93af857e25af95",
              "IPY_MODEL_ad8218da526a4837b4ebe389bf30fa37"
            ],
            "layout": "IPY_MODEL_17bcdf2629074184b950479d3a5070ca"
          }
        },
        "e6fe40ad6ad545c685aa6fdb23b828a0": {
          "model_module": "@jupyter-widgets/controls",
          "model_name": "HTMLModel",
          "model_module_version": "1.5.0",
          "state": {
            "_dom_classes": [],
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "HTMLModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/controls",
            "_view_module_version": "1.5.0",
            "_view_name": "HTMLView",
            "description": "",
            "description_tooltip": null,
            "layout": "IPY_MODEL_f3ca3b9ee7f04164bee010121ebee005",
            "placeholder": "​",
            "style": "IPY_MODEL_edd56c491a6d40b787a7a757f5e328d0",
            "value": "100%"
          }
        },
        "a80b1b2beee64d459a93af857e25af95": {
          "model_module": "@jupyter-widgets/controls",
          "model_name": "FloatProgressModel",
          "model_module_version": "1.5.0",
          "state": {
            "_dom_classes": [],
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "FloatProgressModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/controls",
            "_view_module_version": "1.5.0",
            "_view_name": "ProgressView",
            "bar_style": "success",
            "description": "",
            "description_tooltip": null,
            "layout": "IPY_MODEL_1d64edc1df8a4402ad2a84952a2c2c21",
            "max": 9912422,
            "min": 0,
            "orientation": "horizontal",
            "style": "IPY_MODEL_f11b420f7ece46eeb5ede36ba9e09210",
            "value": 9912422
          }
        },
        "ad8218da526a4837b4ebe389bf30fa37": {
          "model_module": "@jupyter-widgets/controls",
          "model_name": "HTMLModel",
          "model_module_version": "1.5.0",
          "state": {
            "_dom_classes": [],
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "HTMLModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/controls",
            "_view_module_version": "1.5.0",
            "_view_name": "HTMLView",
            "description": "",
            "description_tooltip": null,
            "layout": "IPY_MODEL_32ab8103f798447f8d24636057e1c4fe",
            "placeholder": "​",
            "style": "IPY_MODEL_8cf97cdf227648ca91d8e17c6f3861fc",
            "value": " 9912422/9912422 [00:00&lt;00:00, 25177119.67it/s]"
          }
        },
        "17bcdf2629074184b950479d3a5070ca": {
          "model_module": "@jupyter-widgets/base",
          "model_name": "LayoutModel",
          "model_module_version": "1.2.0",
          "state": {
            "_model_module": "@jupyter-widgets/base",
            "_model_module_version": "1.2.0",
            "_model_name": "LayoutModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "LayoutView",
            "align_content": null,
            "align_items": null,
            "align_self": null,
            "border": null,
            "bottom": null,
            "display": null,
            "flex": null,
            "flex_flow": null,
            "grid_area": null,
            "grid_auto_columns": null,
            "grid_auto_flow": null,
            "grid_auto_rows": null,
            "grid_column": null,
            "grid_gap": null,
            "grid_row": null,
            "grid_template_areas": null,
            "grid_template_columns": null,
            "grid_template_rows": null,
            "height": null,
            "justify_content": null,
            "justify_items": null,
            "left": null,
            "margin": null,
            "max_height": null,
            "max_width": null,
            "min_height": null,
            "min_width": null,
            "object_fit": null,
            "object_position": null,
            "order": null,
            "overflow": null,
            "overflow_x": null,
            "overflow_y": null,
            "padding": null,
            "right": null,
            "top": null,
            "visibility": null,
            "width": null
          }
        },
        "f3ca3b9ee7f04164bee010121ebee005": {
          "model_module": "@jupyter-widgets/base",
          "model_name": "LayoutModel",
          "model_module_version": "1.2.0",
          "state": {
            "_model_module": "@jupyter-widgets/base",
            "_model_module_version": "1.2.0",
            "_model_name": "LayoutModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "LayoutView",
            "align_content": null,
            "align_items": null,
            "align_self": null,
            "border": null,
            "bottom": null,
            "display": null,
            "flex": null,
            "flex_flow": null,
            "grid_area": null,
            "grid_auto_columns": null,
            "grid_auto_flow": null,
            "grid_auto_rows": null,
            "grid_column": null,
            "grid_gap": null,
            "grid_row": null,
            "grid_template_areas": null,
            "grid_template_columns": null,
            "grid_template_rows": null,
            "height": null,
            "justify_content": null,
            "justify_items": null,
            "left": null,
            "margin": null,
            "max_height": null,
            "max_width": null,
            "min_height": null,
            "min_width": null,
            "object_fit": null,
            "object_position": null,
            "order": null,
            "overflow": null,
            "overflow_x": null,
            "overflow_y": null,
            "padding": null,
            "right": null,
            "top": null,
            "visibility": null,
            "width": null
          }
        },
        "edd56c491a6d40b787a7a757f5e328d0": {
          "model_module": "@jupyter-widgets/controls",
          "model_name": "DescriptionStyleModel",
          "model_module_version": "1.5.0",
          "state": {
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "DescriptionStyleModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "StyleView",
            "description_width": ""
          }
        },
        "1d64edc1df8a4402ad2a84952a2c2c21": {
          "model_module": "@jupyter-widgets/base",
          "model_name": "LayoutModel",
          "model_module_version": "1.2.0",
          "state": {
            "_model_module": "@jupyter-widgets/base",
            "_model_module_version": "1.2.0",
            "_model_name": "LayoutModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "LayoutView",
            "align_content": null,
            "align_items": null,
            "align_self": null,
            "border": null,
            "bottom": null,
            "display": null,
            "flex": null,
            "flex_flow": null,
            "grid_area": null,
            "grid_auto_columns": null,
            "grid_auto_flow": null,
            "grid_auto_rows": null,
            "grid_column": null,
            "grid_gap": null,
            "grid_row": null,
            "grid_template_areas": null,
            "grid_template_columns": null,
            "grid_template_rows": null,
            "height": null,
            "justify_content": null,
            "justify_items": null,
            "left": null,
            "margin": null,
            "max_height": null,
            "max_width": null,
            "min_height": null,
            "min_width": null,
            "object_fit": null,
            "object_position": null,
            "order": null,
            "overflow": null,
            "overflow_x": null,
            "overflow_y": null,
            "padding": null,
            "right": null,
            "top": null,
            "visibility": null,
            "width": null
          }
        },
        "f11b420f7ece46eeb5ede36ba9e09210": {
          "model_module": "@jupyter-widgets/controls",
          "model_name": "ProgressStyleModel",
          "model_module_version": "1.5.0",
          "state": {
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "ProgressStyleModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "StyleView",
            "bar_color": null,
            "description_width": ""
          }
        },
        "32ab8103f798447f8d24636057e1c4fe": {
          "model_module": "@jupyter-widgets/base",
          "model_name": "LayoutModel",
          "model_module_version": "1.2.0",
          "state": {
            "_model_module": "@jupyter-widgets/base",
            "_model_module_version": "1.2.0",
            "_model_name": "LayoutModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "LayoutView",
            "align_content": null,
            "align_items": null,
            "align_self": null,
            "border": null,
            "bottom": null,
            "display": null,
            "flex": null,
            "flex_flow": null,
            "grid_area": null,
            "grid_auto_columns": null,
            "grid_auto_flow": null,
            "grid_auto_rows": null,
            "grid_column": null,
            "grid_gap": null,
            "grid_row": null,
            "grid_template_areas": null,
            "grid_template_columns": null,
            "grid_template_rows": null,
            "height": null,
            "justify_content": null,
            "justify_items": null,
            "left": null,
            "margin": null,
            "max_height": null,
            "max_width": null,
            "min_height": null,
            "min_width": null,
            "object_fit": null,
            "object_position": null,
            "order": null,
            "overflow": null,
            "overflow_x": null,
            "overflow_y": null,
            "padding": null,
            "right": null,
            "top": null,
            "visibility": null,
            "width": null
          }
        },
        "8cf97cdf227648ca91d8e17c6f3861fc": {
          "model_module": "@jupyter-widgets/controls",
          "model_name": "DescriptionStyleModel",
          "model_module_version": "1.5.0",
          "state": {
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "DescriptionStyleModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "StyleView",
            "description_width": ""
          }
        },
        "8f21f583c2b945e49be68d7f6c5f677e": {
          "model_module": "@jupyter-widgets/controls",
          "model_name": "HBoxModel",
          "model_module_version": "1.5.0",
          "state": {
            "_dom_classes": [],
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "HBoxModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/controls",
            "_view_module_version": "1.5.0",
            "_view_name": "HBoxView",
            "box_style": "",
            "children": [
              "IPY_MODEL_d16eb7c10c8145f7854a2a9f9dc4d8cf",
              "IPY_MODEL_d6899de509134307b7ac12b45e5b72d6",
              "IPY_MODEL_c284fca7832240368f13f4effb2a71e9"
            ],
            "layout": "IPY_MODEL_35876458314a4510a0a9e134f9d1e95c"
          }
        },
        "d16eb7c10c8145f7854a2a9f9dc4d8cf": {
          "model_module": "@jupyter-widgets/controls",
          "model_name": "HTMLModel",
          "model_module_version": "1.5.0",
          "state": {
            "_dom_classes": [],
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "HTMLModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/controls",
            "_view_module_version": "1.5.0",
            "_view_name": "HTMLView",
            "description": "",
            "description_tooltip": null,
            "layout": "IPY_MODEL_d666260fd53742918de352714ef19ff4",
            "placeholder": "​",
            "style": "IPY_MODEL_7fd7006eae5544919ec02cce074d265e",
            "value": "100%"
          }
        },
        "d6899de509134307b7ac12b45e5b72d6": {
          "model_module": "@jupyter-widgets/controls",
          "model_name": "FloatProgressModel",
          "model_module_version": "1.5.0",
          "state": {
            "_dom_classes": [],
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "FloatProgressModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/controls",
            "_view_module_version": "1.5.0",
            "_view_name": "ProgressView",
            "bar_style": "success",
            "description": "",
            "description_tooltip": null,
            "layout": "IPY_MODEL_0abb37de8fb44e7ea4eff476ddb7540f",
            "max": 28881,
            "min": 0,
            "orientation": "horizontal",
            "style": "IPY_MODEL_c76055834cc743ecbc41121aed1c8885",
            "value": 28881
          }
        },
        "c284fca7832240368f13f4effb2a71e9": {
          "model_module": "@jupyter-widgets/controls",
          "model_name": "HTMLModel",
          "model_module_version": "1.5.0",
          "state": {
            "_dom_classes": [],
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "HTMLModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/controls",
            "_view_module_version": "1.5.0",
            "_view_name": "HTMLView",
            "description": "",
            "description_tooltip": null,
            "layout": "IPY_MODEL_0508ffaa73b64caea80b9e53b7f61454",
            "placeholder": "​",
            "style": "IPY_MODEL_e5ff2ddbd95844ae9aa97063659184ed",
            "value": " 28881/28881 [00:00&lt;00:00, 1605466.97it/s]"
          }
        },
        "35876458314a4510a0a9e134f9d1e95c": {
          "model_module": "@jupyter-widgets/base",
          "model_name": "LayoutModel",
          "model_module_version": "1.2.0",
          "state": {
            "_model_module": "@jupyter-widgets/base",
            "_model_module_version": "1.2.0",
            "_model_name": "LayoutModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "LayoutView",
            "align_content": null,
            "align_items": null,
            "align_self": null,
            "border": null,
            "bottom": null,
            "display": null,
            "flex": null,
            "flex_flow": null,
            "grid_area": null,
            "grid_auto_columns": null,
            "grid_auto_flow": null,
            "grid_auto_rows": null,
            "grid_column": null,
            "grid_gap": null,
            "grid_row": null,
            "grid_template_areas": null,
            "grid_template_columns": null,
            "grid_template_rows": null,
            "height": null,
            "justify_content": null,
            "justify_items": null,
            "left": null,
            "margin": null,
            "max_height": null,
            "max_width": null,
            "min_height": null,
            "min_width": null,
            "object_fit": null,
            "object_position": null,
            "order": null,
            "overflow": null,
            "overflow_x": null,
            "overflow_y": null,
            "padding": null,
            "right": null,
            "top": null,
            "visibility": null,
            "width": null
          }
        },
        "d666260fd53742918de352714ef19ff4": {
          "model_module": "@jupyter-widgets/base",
          "model_name": "LayoutModel",
          "model_module_version": "1.2.0",
          "state": {
            "_model_module": "@jupyter-widgets/base",
            "_model_module_version": "1.2.0",
            "_model_name": "LayoutModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "LayoutView",
            "align_content": null,
            "align_items": null,
            "align_self": null,
            "border": null,
            "bottom": null,
            "display": null,
            "flex": null,
            "flex_flow": null,
            "grid_area": null,
            "grid_auto_columns": null,
            "grid_auto_flow": null,
            "grid_auto_rows": null,
            "grid_column": null,
            "grid_gap": null,
            "grid_row": null,
            "grid_template_areas": null,
            "grid_template_columns": null,
            "grid_template_rows": null,
            "height": null,
            "justify_content": null,
            "justify_items": null,
            "left": null,
            "margin": null,
            "max_height": null,
            "max_width": null,
            "min_height": null,
            "min_width": null,
            "object_fit": null,
            "object_position": null,
            "order": null,
            "overflow": null,
            "overflow_x": null,
            "overflow_y": null,
            "padding": null,
            "right": null,
            "top": null,
            "visibility": null,
            "width": null
          }
        },
        "7fd7006eae5544919ec02cce074d265e": {
          "model_module": "@jupyter-widgets/controls",
          "model_name": "DescriptionStyleModel",
          "model_module_version": "1.5.0",
          "state": {
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "DescriptionStyleModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "StyleView",
            "description_width": ""
          }
        },
        "0abb37de8fb44e7ea4eff476ddb7540f": {
          "model_module": "@jupyter-widgets/base",
          "model_name": "LayoutModel",
          "model_module_version": "1.2.0",
          "state": {
            "_model_module": "@jupyter-widgets/base",
            "_model_module_version": "1.2.0",
            "_model_name": "LayoutModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "LayoutView",
            "align_content": null,
            "align_items": null,
            "align_self": null,
            "border": null,
            "bottom": null,
            "display": null,
            "flex": null,
            "flex_flow": null,
            "grid_area": null,
            "grid_auto_columns": null,
            "grid_auto_flow": null,
            "grid_auto_rows": null,
            "grid_column": null,
            "grid_gap": null,
            "grid_row": null,
            "grid_template_areas": null,
            "grid_template_columns": null,
            "grid_template_rows": null,
            "height": null,
            "justify_content": null,
            "justify_items": null,
            "left": null,
            "margin": null,
            "max_height": null,
            "max_width": null,
            "min_height": null,
            "min_width": null,
            "object_fit": null,
            "object_position": null,
            "order": null,
            "overflow": null,
            "overflow_x": null,
            "overflow_y": null,
            "padding": null,
            "right": null,
            "top": null,
            "visibility": null,
            "width": null
          }
        },
        "c76055834cc743ecbc41121aed1c8885": {
          "model_module": "@jupyter-widgets/controls",
          "model_name": "ProgressStyleModel",
          "model_module_version": "1.5.0",
          "state": {
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "ProgressStyleModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "StyleView",
            "bar_color": null,
            "description_width": ""
          }
        },
        "0508ffaa73b64caea80b9e53b7f61454": {
          "model_module": "@jupyter-widgets/base",
          "model_name": "LayoutModel",
          "model_module_version": "1.2.0",
          "state": {
            "_model_module": "@jupyter-widgets/base",
            "_model_module_version": "1.2.0",
            "_model_name": "LayoutModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "LayoutView",
            "align_content": null,
            "align_items": null,
            "align_self": null,
            "border": null,
            "bottom": null,
            "display": null,
            "flex": null,
            "flex_flow": null,
            "grid_area": null,
            "grid_auto_columns": null,
            "grid_auto_flow": null,
            "grid_auto_rows": null,
            "grid_column": null,
            "grid_gap": null,
            "grid_row": null,
            "grid_template_areas": null,
            "grid_template_columns": null,
            "grid_template_rows": null,
            "height": null,
            "justify_content": null,
            "justify_items": null,
            "left": null,
            "margin": null,
            "max_height": null,
            "max_width": null,
            "min_height": null,
            "min_width": null,
            "object_fit": null,
            "object_position": null,
            "order": null,
            "overflow": null,
            "overflow_x": null,
            "overflow_y": null,
            "padding": null,
            "right": null,
            "top": null,
            "visibility": null,
            "width": null
          }
        },
        "e5ff2ddbd95844ae9aa97063659184ed": {
          "model_module": "@jupyter-widgets/controls",
          "model_name": "DescriptionStyleModel",
          "model_module_version": "1.5.0",
          "state": {
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "DescriptionStyleModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "StyleView",
            "description_width": ""
          }
        },
        "3bcd756036a54100bf14fd785667c092": {
          "model_module": "@jupyter-widgets/controls",
          "model_name": "HBoxModel",
          "model_module_version": "1.5.0",
          "state": {
            "_dom_classes": [],
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "HBoxModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/controls",
            "_view_module_version": "1.5.0",
            "_view_name": "HBoxView",
            "box_style": "",
            "children": [
              "IPY_MODEL_47b5d1a112f64bf49de801fe48dcde2c",
              "IPY_MODEL_3b745cd3f2b041c4a3df8284ddb44ec2",
              "IPY_MODEL_a41f3fa24c474a8db5173182002b8b3e"
            ],
            "layout": "IPY_MODEL_0679b226b013462083c4a137f05b8a3f"
          }
        },
        "47b5d1a112f64bf49de801fe48dcde2c": {
          "model_module": "@jupyter-widgets/controls",
          "model_name": "HTMLModel",
          "model_module_version": "1.5.0",
          "state": {
            "_dom_classes": [],
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "HTMLModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/controls",
            "_view_module_version": "1.5.0",
            "_view_name": "HTMLView",
            "description": "",
            "description_tooltip": null,
            "layout": "IPY_MODEL_221be29a10384bec91cfc7786c44e0a9",
            "placeholder": "​",
            "style": "IPY_MODEL_67b3f4d9637240df882850326a34baa4",
            "value": "100%"
          }
        },
        "3b745cd3f2b041c4a3df8284ddb44ec2": {
          "model_module": "@jupyter-widgets/controls",
          "model_name": "FloatProgressModel",
          "model_module_version": "1.5.0",
          "state": {
            "_dom_classes": [],
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "FloatProgressModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/controls",
            "_view_module_version": "1.5.0",
            "_view_name": "ProgressView",
            "bar_style": "success",
            "description": "",
            "description_tooltip": null,
            "layout": "IPY_MODEL_961f36cc634144a3b921640552cb5323",
            "max": 1648877,
            "min": 0,
            "orientation": "horizontal",
            "style": "IPY_MODEL_00a97b29029a4c2795fd9d673dbe7215",
            "value": 1648877
          }
        },
        "a41f3fa24c474a8db5173182002b8b3e": {
          "model_module": "@jupyter-widgets/controls",
          "model_name": "HTMLModel",
          "model_module_version": "1.5.0",
          "state": {
            "_dom_classes": [],
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "HTMLModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/controls",
            "_view_module_version": "1.5.0",
            "_view_name": "HTMLView",
            "description": "",
            "description_tooltip": null,
            "layout": "IPY_MODEL_b004c8409add445c9042c581644d61c0",
            "placeholder": "​",
            "style": "IPY_MODEL_d25a9981b9724054ac196455e563bea0",
            "value": " 1648877/1648877 [00:00&lt;00:00, 19768274.78it/s]"
          }
        },
        "0679b226b013462083c4a137f05b8a3f": {
          "model_module": "@jupyter-widgets/base",
          "model_name": "LayoutModel",
          "model_module_version": "1.2.0",
          "state": {
            "_model_module": "@jupyter-widgets/base",
            "_model_module_version": "1.2.0",
            "_model_name": "LayoutModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "LayoutView",
            "align_content": null,
            "align_items": null,
            "align_self": null,
            "border": null,
            "bottom": null,
            "display": null,
            "flex": null,
            "flex_flow": null,
            "grid_area": null,
            "grid_auto_columns": null,
            "grid_auto_flow": null,
            "grid_auto_rows": null,
            "grid_column": null,
            "grid_gap": null,
            "grid_row": null,
            "grid_template_areas": null,
            "grid_template_columns": null,
            "grid_template_rows": null,
            "height": null,
            "justify_content": null,
            "justify_items": null,
            "left": null,
            "margin": null,
            "max_height": null,
            "max_width": null,
            "min_height": null,
            "min_width": null,
            "object_fit": null,
            "object_position": null,
            "order": null,
            "overflow": null,
            "overflow_x": null,
            "overflow_y": null,
            "padding": null,
            "right": null,
            "top": null,
            "visibility": null,
            "width": null
          }
        },
        "221be29a10384bec91cfc7786c44e0a9": {
          "model_module": "@jupyter-widgets/base",
          "model_name": "LayoutModel",
          "model_module_version": "1.2.0",
          "state": {
            "_model_module": "@jupyter-widgets/base",
            "_model_module_version": "1.2.0",
            "_model_name": "LayoutModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "LayoutView",
            "align_content": null,
            "align_items": null,
            "align_self": null,
            "border": null,
            "bottom": null,
            "display": null,
            "flex": null,
            "flex_flow": null,
            "grid_area": null,
            "grid_auto_columns": null,
            "grid_auto_flow": null,
            "grid_auto_rows": null,
            "grid_column": null,
            "grid_gap": null,
            "grid_row": null,
            "grid_template_areas": null,
            "grid_template_columns": null,
            "grid_template_rows": null,
            "height": null,
            "justify_content": null,
            "justify_items": null,
            "left": null,
            "margin": null,
            "max_height": null,
            "max_width": null,
            "min_height": null,
            "min_width": null,
            "object_fit": null,
            "object_position": null,
            "order": null,
            "overflow": null,
            "overflow_x": null,
            "overflow_y": null,
            "padding": null,
            "right": null,
            "top": null,
            "visibility": null,
            "width": null
          }
        },
        "67b3f4d9637240df882850326a34baa4": {
          "model_module": "@jupyter-widgets/controls",
          "model_name": "DescriptionStyleModel",
          "model_module_version": "1.5.0",
          "state": {
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "DescriptionStyleModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "StyleView",
            "description_width": ""
          }
        },
        "961f36cc634144a3b921640552cb5323": {
          "model_module": "@jupyter-widgets/base",
          "model_name": "LayoutModel",
          "model_module_version": "1.2.0",
          "state": {
            "_model_module": "@jupyter-widgets/base",
            "_model_module_version": "1.2.0",
            "_model_name": "LayoutModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "LayoutView",
            "align_content": null,
            "align_items": null,
            "align_self": null,
            "border": null,
            "bottom": null,
            "display": null,
            "flex": null,
            "flex_flow": null,
            "grid_area": null,
            "grid_auto_columns": null,
            "grid_auto_flow": null,
            "grid_auto_rows": null,
            "grid_column": null,
            "grid_gap": null,
            "grid_row": null,
            "grid_template_areas": null,
            "grid_template_columns": null,
            "grid_template_rows": null,
            "height": null,
            "justify_content": null,
            "justify_items": null,
            "left": null,
            "margin": null,
            "max_height": null,
            "max_width": null,
            "min_height": null,
            "min_width": null,
            "object_fit": null,
            "object_position": null,
            "order": null,
            "overflow": null,
            "overflow_x": null,
            "overflow_y": null,
            "padding": null,
            "right": null,
            "top": null,
            "visibility": null,
            "width": null
          }
        },
        "00a97b29029a4c2795fd9d673dbe7215": {
          "model_module": "@jupyter-widgets/controls",
          "model_name": "ProgressStyleModel",
          "model_module_version": "1.5.0",
          "state": {
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "ProgressStyleModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "StyleView",
            "bar_color": null,
            "description_width": ""
          }
        },
        "b004c8409add445c9042c581644d61c0": {
          "model_module": "@jupyter-widgets/base",
          "model_name": "LayoutModel",
          "model_module_version": "1.2.0",
          "state": {
            "_model_module": "@jupyter-widgets/base",
            "_model_module_version": "1.2.0",
            "_model_name": "LayoutModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "LayoutView",
            "align_content": null,
            "align_items": null,
            "align_self": null,
            "border": null,
            "bottom": null,
            "display": null,
            "flex": null,
            "flex_flow": null,
            "grid_area": null,
            "grid_auto_columns": null,
            "grid_auto_flow": null,
            "grid_auto_rows": null,
            "grid_column": null,
            "grid_gap": null,
            "grid_row": null,
            "grid_template_areas": null,
            "grid_template_columns": null,
            "grid_template_rows": null,
            "height": null,
            "justify_content": null,
            "justify_items": null,
            "left": null,
            "margin": null,
            "max_height": null,
            "max_width": null,
            "min_height": null,
            "min_width": null,
            "object_fit": null,
            "object_position": null,
            "order": null,
            "overflow": null,
            "overflow_x": null,
            "overflow_y": null,
            "padding": null,
            "right": null,
            "top": null,
            "visibility": null,
            "width": null
          }
        },
        "d25a9981b9724054ac196455e563bea0": {
          "model_module": "@jupyter-widgets/controls",
          "model_name": "DescriptionStyleModel",
          "model_module_version": "1.5.0",
          "state": {
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "DescriptionStyleModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "StyleView",
            "description_width": ""
          }
        },
        "b0666d0566af4c8ea716b835e3f10bd1": {
          "model_module": "@jupyter-widgets/controls",
          "model_name": "HBoxModel",
          "model_module_version": "1.5.0",
          "state": {
            "_dom_classes": [],
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "HBoxModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/controls",
            "_view_module_version": "1.5.0",
            "_view_name": "HBoxView",
            "box_style": "",
            "children": [
              "IPY_MODEL_6399e318eaad4903a4b798489aeecbb1",
              "IPY_MODEL_9d5710c9d09b42eaada8450872ec5dab",
              "IPY_MODEL_638dca8a30db4b61ae8bb190ef433acc"
            ],
            "layout": "IPY_MODEL_841d7f6acb604fb39489ecabefd2a198"
          }
        },
        "6399e318eaad4903a4b798489aeecbb1": {
          "model_module": "@jupyter-widgets/controls",
          "model_name": "HTMLModel",
          "model_module_version": "1.5.0",
          "state": {
            "_dom_classes": [],
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "HTMLModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/controls",
            "_view_module_version": "1.5.0",
            "_view_name": "HTMLView",
            "description": "",
            "description_tooltip": null,
            "layout": "IPY_MODEL_ac333eb723034faa93b92d165a5c6db1",
            "placeholder": "​",
            "style": "IPY_MODEL_7bee58598d9640df9415deebf1f5dc55",
            "value": "100%"
          }
        },
        "9d5710c9d09b42eaada8450872ec5dab": {
          "model_module": "@jupyter-widgets/controls",
          "model_name": "FloatProgressModel",
          "model_module_version": "1.5.0",
          "state": {
            "_dom_classes": [],
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "FloatProgressModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/controls",
            "_view_module_version": "1.5.0",
            "_view_name": "ProgressView",
            "bar_style": "success",
            "description": "",
            "description_tooltip": null,
            "layout": "IPY_MODEL_c1ed3eeee590463098cf4da80a5a2cdd",
            "max": 4542,
            "min": 0,
            "orientation": "horizontal",
            "style": "IPY_MODEL_a8c3e0744e3246d78d047afacde5a9d5",
            "value": 4542
          }
        },
        "638dca8a30db4b61ae8bb190ef433acc": {
          "model_module": "@jupyter-widgets/controls",
          "model_name": "HTMLModel",
          "model_module_version": "1.5.0",
          "state": {
            "_dom_classes": [],
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "HTMLModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/controls",
            "_view_module_version": "1.5.0",
            "_view_name": "HTMLView",
            "description": "",
            "description_tooltip": null,
            "layout": "IPY_MODEL_54467dd23b2c48c5b323b1fb920c010e",
            "placeholder": "​",
            "style": "IPY_MODEL_7b15de41f3104996bc7e83682c141758",
            "value": " 4542/4542 [00:00&lt;00:00, 274586.38it/s]"
          }
        },
        "841d7f6acb604fb39489ecabefd2a198": {
          "model_module": "@jupyter-widgets/base",
          "model_name": "LayoutModel",
          "model_module_version": "1.2.0",
          "state": {
            "_model_module": "@jupyter-widgets/base",
            "_model_module_version": "1.2.0",
            "_model_name": "LayoutModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "LayoutView",
            "align_content": null,
            "align_items": null,
            "align_self": null,
            "border": null,
            "bottom": null,
            "display": null,
            "flex": null,
            "flex_flow": null,
            "grid_area": null,
            "grid_auto_columns": null,
            "grid_auto_flow": null,
            "grid_auto_rows": null,
            "grid_column": null,
            "grid_gap": null,
            "grid_row": null,
            "grid_template_areas": null,
            "grid_template_columns": null,
            "grid_template_rows": null,
            "height": null,
            "justify_content": null,
            "justify_items": null,
            "left": null,
            "margin": null,
            "max_height": null,
            "max_width": null,
            "min_height": null,
            "min_width": null,
            "object_fit": null,
            "object_position": null,
            "order": null,
            "overflow": null,
            "overflow_x": null,
            "overflow_y": null,
            "padding": null,
            "right": null,
            "top": null,
            "visibility": null,
            "width": null
          }
        },
        "ac333eb723034faa93b92d165a5c6db1": {
          "model_module": "@jupyter-widgets/base",
          "model_name": "LayoutModel",
          "model_module_version": "1.2.0",
          "state": {
            "_model_module": "@jupyter-widgets/base",
            "_model_module_version": "1.2.0",
            "_model_name": "LayoutModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "LayoutView",
            "align_content": null,
            "align_items": null,
            "align_self": null,
            "border": null,
            "bottom": null,
            "display": null,
            "flex": null,
            "flex_flow": null,
            "grid_area": null,
            "grid_auto_columns": null,
            "grid_auto_flow": null,
            "grid_auto_rows": null,
            "grid_column": null,
            "grid_gap": null,
            "grid_row": null,
            "grid_template_areas": null,
            "grid_template_columns": null,
            "grid_template_rows": null,
            "height": null,
            "justify_content": null,
            "justify_items": null,
            "left": null,
            "margin": null,
            "max_height": null,
            "max_width": null,
            "min_height": null,
            "min_width": null,
            "object_fit": null,
            "object_position": null,
            "order": null,
            "overflow": null,
            "overflow_x": null,
            "overflow_y": null,
            "padding": null,
            "right": null,
            "top": null,
            "visibility": null,
            "width": null
          }
        },
        "7bee58598d9640df9415deebf1f5dc55": {
          "model_module": "@jupyter-widgets/controls",
          "model_name": "DescriptionStyleModel",
          "model_module_version": "1.5.0",
          "state": {
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "DescriptionStyleModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "StyleView",
            "description_width": ""
          }
        },
        "c1ed3eeee590463098cf4da80a5a2cdd": {
          "model_module": "@jupyter-widgets/base",
          "model_name": "LayoutModel",
          "model_module_version": "1.2.0",
          "state": {
            "_model_module": "@jupyter-widgets/base",
            "_model_module_version": "1.2.0",
            "_model_name": "LayoutModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "LayoutView",
            "align_content": null,
            "align_items": null,
            "align_self": null,
            "border": null,
            "bottom": null,
            "display": null,
            "flex": null,
            "flex_flow": null,
            "grid_area": null,
            "grid_auto_columns": null,
            "grid_auto_flow": null,
            "grid_auto_rows": null,
            "grid_column": null,
            "grid_gap": null,
            "grid_row": null,
            "grid_template_areas": null,
            "grid_template_columns": null,
            "grid_template_rows": null,
            "height": null,
            "justify_content": null,
            "justify_items": null,
            "left": null,
            "margin": null,
            "max_height": null,
            "max_width": null,
            "min_height": null,
            "min_width": null,
            "object_fit": null,
            "object_position": null,
            "order": null,
            "overflow": null,
            "overflow_x": null,
            "overflow_y": null,
            "padding": null,
            "right": null,
            "top": null,
            "visibility": null,
            "width": null
          }
        },
        "a8c3e0744e3246d78d047afacde5a9d5": {
          "model_module": "@jupyter-widgets/controls",
          "model_name": "ProgressStyleModel",
          "model_module_version": "1.5.0",
          "state": {
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "ProgressStyleModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "StyleView",
            "bar_color": null,
            "description_width": ""
          }
        },
        "54467dd23b2c48c5b323b1fb920c010e": {
          "model_module": "@jupyter-widgets/base",
          "model_name": "LayoutModel",
          "model_module_version": "1.2.0",
          "state": {
            "_model_module": "@jupyter-widgets/base",
            "_model_module_version": "1.2.0",
            "_model_name": "LayoutModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "LayoutView",
            "align_content": null,
            "align_items": null,
            "align_self": null,
            "border": null,
            "bottom": null,
            "display": null,
            "flex": null,
            "flex_flow": null,
            "grid_area": null,
            "grid_auto_columns": null,
            "grid_auto_flow": null,
            "grid_auto_rows": null,
            "grid_column": null,
            "grid_gap": null,
            "grid_row": null,
            "grid_template_areas": null,
            "grid_template_columns": null,
            "grid_template_rows": null,
            "height": null,
            "justify_content": null,
            "justify_items": null,
            "left": null,
            "margin": null,
            "max_height": null,
            "max_width": null,
            "min_height": null,
            "min_width": null,
            "object_fit": null,
            "object_position": null,
            "order": null,
            "overflow": null,
            "overflow_x": null,
            "overflow_y": null,
            "padding": null,
            "right": null,
            "top": null,
            "visibility": null,
            "width": null
          }
        },
        "7b15de41f3104996bc7e83682c141758": {
          "model_module": "@jupyter-widgets/controls",
          "model_name": "DescriptionStyleModel",
          "model_module_version": "1.5.0",
          "state": {
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "DescriptionStyleModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "StyleView",
            "description_width": ""
          }
        },
        "9cd42a082e2c486c9103936a5feeb9dd": {
          "model_module": "@jupyter-widgets/controls",
          "model_name": "HBoxModel",
          "model_module_version": "1.5.0",
          "state": {
            "_dom_classes": [],
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "HBoxModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/controls",
            "_view_module_version": "1.5.0",
            "_view_name": "HBoxView",
            "box_style": "",
            "children": [
              "IPY_MODEL_89121e8de88149819740e86d77dacb65",
              "IPY_MODEL_79e5d20fc6184a8a89321fcfc9b9bb0a",
              "IPY_MODEL_7a9e7b8ff5f043a48e351ade72dee629"
            ],
            "layout": "IPY_MODEL_96cbc923b2e94a069e5428e492c3c87d"
          }
        },
        "89121e8de88149819740e86d77dacb65": {
          "model_module": "@jupyter-widgets/controls",
          "model_name": "HTMLModel",
          "model_module_version": "1.5.0",
          "state": {
            "_dom_classes": [],
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "HTMLModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/controls",
            "_view_module_version": "1.5.0",
            "_view_name": "HTMLView",
            "description": "",
            "description_tooltip": null,
            "layout": "IPY_MODEL_1259ed5bc6b54c49b24409eb45d3b4c7",
            "placeholder": "​",
            "style": "IPY_MODEL_d429bce01e794576adaf1f7cbe2f117c",
            "value": "100%"
          }
        },
        "79e5d20fc6184a8a89321fcfc9b9bb0a": {
          "model_module": "@jupyter-widgets/controls",
          "model_name": "FloatProgressModel",
          "model_module_version": "1.5.0",
          "state": {
            "_dom_classes": [],
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "FloatProgressModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/controls",
            "_view_module_version": "1.5.0",
            "_view_name": "ProgressView",
            "bar_style": "success",
            "description": "",
            "description_tooltip": null,
            "layout": "IPY_MODEL_49e17d4e1a9a48b4aac9c897987e817f",
            "max": 170498071,
            "min": 0,
            "orientation": "horizontal",
            "style": "IPY_MODEL_0722ef4bbe454f8fa63bb6a8a26c10f8",
            "value": 170498071
          }
        },
        "7a9e7b8ff5f043a48e351ade72dee629": {
          "model_module": "@jupyter-widgets/controls",
          "model_name": "HTMLModel",
          "model_module_version": "1.5.0",
          "state": {
            "_dom_classes": [],
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "HTMLModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/controls",
            "_view_module_version": "1.5.0",
            "_view_name": "HTMLView",
            "description": "",
            "description_tooltip": null,
            "layout": "IPY_MODEL_5a6e379606814bec945f65da30bb573d",
            "placeholder": "​",
            "style": "IPY_MODEL_c3714160355d475b8c5f24835518100c",
            "value": " 170498071/170498071 [00:01&lt;00:00, 112748851.41it/s]"
          }
        },
        "96cbc923b2e94a069e5428e492c3c87d": {
          "model_module": "@jupyter-widgets/base",
          "model_name": "LayoutModel",
          "model_module_version": "1.2.0",
          "state": {
            "_model_module": "@jupyter-widgets/base",
            "_model_module_version": "1.2.0",
            "_model_name": "LayoutModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "LayoutView",
            "align_content": null,
            "align_items": null,
            "align_self": null,
            "border": null,
            "bottom": null,
            "display": null,
            "flex": null,
            "flex_flow": null,
            "grid_area": null,
            "grid_auto_columns": null,
            "grid_auto_flow": null,
            "grid_auto_rows": null,
            "grid_column": null,
            "grid_gap": null,
            "grid_row": null,
            "grid_template_areas": null,
            "grid_template_columns": null,
            "grid_template_rows": null,
            "height": null,
            "justify_content": null,
            "justify_items": null,
            "left": null,
            "margin": null,
            "max_height": null,
            "max_width": null,
            "min_height": null,
            "min_width": null,
            "object_fit": null,
            "object_position": null,
            "order": null,
            "overflow": null,
            "overflow_x": null,
            "overflow_y": null,
            "padding": null,
            "right": null,
            "top": null,
            "visibility": null,
            "width": null
          }
        },
        "1259ed5bc6b54c49b24409eb45d3b4c7": {
          "model_module": "@jupyter-widgets/base",
          "model_name": "LayoutModel",
          "model_module_version": "1.2.0",
          "state": {
            "_model_module": "@jupyter-widgets/base",
            "_model_module_version": "1.2.0",
            "_model_name": "LayoutModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "LayoutView",
            "align_content": null,
            "align_items": null,
            "align_self": null,
            "border": null,
            "bottom": null,
            "display": null,
            "flex": null,
            "flex_flow": null,
            "grid_area": null,
            "grid_auto_columns": null,
            "grid_auto_flow": null,
            "grid_auto_rows": null,
            "grid_column": null,
            "grid_gap": null,
            "grid_row": null,
            "grid_template_areas": null,
            "grid_template_columns": null,
            "grid_template_rows": null,
            "height": null,
            "justify_content": null,
            "justify_items": null,
            "left": null,
            "margin": null,
            "max_height": null,
            "max_width": null,
            "min_height": null,
            "min_width": null,
            "object_fit": null,
            "object_position": null,
            "order": null,
            "overflow": null,
            "overflow_x": null,
            "overflow_y": null,
            "padding": null,
            "right": null,
            "top": null,
            "visibility": null,
            "width": null
          }
        },
        "d429bce01e794576adaf1f7cbe2f117c": {
          "model_module": "@jupyter-widgets/controls",
          "model_name": "DescriptionStyleModel",
          "model_module_version": "1.5.0",
          "state": {
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "DescriptionStyleModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "StyleView",
            "description_width": ""
          }
        },
        "49e17d4e1a9a48b4aac9c897987e817f": {
          "model_module": "@jupyter-widgets/base",
          "model_name": "LayoutModel",
          "model_module_version": "1.2.0",
          "state": {
            "_model_module": "@jupyter-widgets/base",
            "_model_module_version": "1.2.0",
            "_model_name": "LayoutModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "LayoutView",
            "align_content": null,
            "align_items": null,
            "align_self": null,
            "border": null,
            "bottom": null,
            "display": null,
            "flex": null,
            "flex_flow": null,
            "grid_area": null,
            "grid_auto_columns": null,
            "grid_auto_flow": null,
            "grid_auto_rows": null,
            "grid_column": null,
            "grid_gap": null,
            "grid_row": null,
            "grid_template_areas": null,
            "grid_template_columns": null,
            "grid_template_rows": null,
            "height": null,
            "justify_content": null,
            "justify_items": null,
            "left": null,
            "margin": null,
            "max_height": null,
            "max_width": null,
            "min_height": null,
            "min_width": null,
            "object_fit": null,
            "object_position": null,
            "order": null,
            "overflow": null,
            "overflow_x": null,
            "overflow_y": null,
            "padding": null,
            "right": null,
            "top": null,
            "visibility": null,
            "width": null
          }
        },
        "0722ef4bbe454f8fa63bb6a8a26c10f8": {
          "model_module": "@jupyter-widgets/controls",
          "model_name": "ProgressStyleModel",
          "model_module_version": "1.5.0",
          "state": {
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "ProgressStyleModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "StyleView",
            "bar_color": null,
            "description_width": ""
          }
        },
        "5a6e379606814bec945f65da30bb573d": {
          "model_module": "@jupyter-widgets/base",
          "model_name": "LayoutModel",
          "model_module_version": "1.2.0",
          "state": {
            "_model_module": "@jupyter-widgets/base",
            "_model_module_version": "1.2.0",
            "_model_name": "LayoutModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "LayoutView",
            "align_content": null,
            "align_items": null,
            "align_self": null,
            "border": null,
            "bottom": null,
            "display": null,
            "flex": null,
            "flex_flow": null,
            "grid_area": null,
            "grid_auto_columns": null,
            "grid_auto_flow": null,
            "grid_auto_rows": null,
            "grid_column": null,
            "grid_gap": null,
            "grid_row": null,
            "grid_template_areas": null,
            "grid_template_columns": null,
            "grid_template_rows": null,
            "height": null,
            "justify_content": null,
            "justify_items": null,
            "left": null,
            "margin": null,
            "max_height": null,
            "max_width": null,
            "min_height": null,
            "min_width": null,
            "object_fit": null,
            "object_position": null,
            "order": null,
            "overflow": null,
            "overflow_x": null,
            "overflow_y": null,
            "padding": null,
            "right": null,
            "top": null,
            "visibility": null,
            "width": null
          }
        },
        "c3714160355d475b8c5f24835518100c": {
          "model_module": "@jupyter-widgets/controls",
          "model_name": "DescriptionStyleModel",
          "model_module_version": "1.5.0",
          "state": {
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "DescriptionStyleModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "StyleView",
            "description_width": ""
          }
        }
      }
    }
  },
  "nbformat": 4,
  "nbformat_minor": 0
}