{
 "cells": [
  {
   "cell_type": "markdown",
   "id": "56889fcd-f08a-4475-9928-af817748f5c8",
   "metadata": {
    "nbgrader": {
     "grade": false,
     "grade_id": "cell-49d9973023c85a7f",
     "locked": true,
     "schema_version": 3,
     "solution": false,
     "task": false
    }
   },
   "source": [
    "## Numerical Analysis - Fall semester 2025\n",
    "# Serie 01 - Scientific computing with Python"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "cd29979b-0db2-4e5a-bd16-13711576ddae",
   "metadata": {
    "nbgrader": {
     "grade": false,
     "grade_id": "cell-6b13ce753305bc77",
     "locked": true,
     "schema_version": 3,
     "solution": false,
     "task": false
    }
   },
   "source": [
    "This notebook serves as a brief introduction to the programming language [Python](https://www.python.org/). If you are already very familiar with Python, [Jupyter notebooks](https://jupyter.org/), and the [NumPy](https://numpy.org/) and [Matplotlib](https://matplotlib.org/) packages, you may skip the explanations and directly attempt the exercises below the green boxes &#x1F7E9;."
   ]
  },
  {
   "cell_type": "markdown",
   "id": "9d03af2f-b95e-4748-802a-35d7c38f8f3b",
   "metadata": {
    "nbgrader": {
     "grade": false,
     "grade_id": "cell-b106366fbd018302",
     "locked": true,
     "schema_version": 3,
     "solution": false,
     "task": false
    }
   },
   "source": [
    "<hr style=\"clear:both\">\n",
    "\n",
    "### Jupyter notebooks for mixing text with code"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "7eaf2601-6ab8-487c-b970-bd6a5d442c47",
   "metadata": {
    "nbgrader": {
     "grade": false,
     "grade_id": "cell-748feed3c6e5328f",
     "locked": true,
     "schema_version": 3,
     "solution": false,
     "task": false
    }
   },
   "source": [
    "A Jupyter notebook is a useful tool for mixing text with code. Each notebook is made up of multiple cells stacked on top of each other. Cells can be added, deleted, and rearranged. Each cell contains either some text which is formatted with Markdown (what you are reading now) or some code which is interpreted with the Python language."
   ]
  },
  {
   "cell_type": "markdown",
   "id": "09ca17c4-7546-4985-a9c8-398b97b888be",
   "metadata": {
    "nbgrader": {
     "grade": false,
     "grade_id": "cell-42666ea831d96794",
     "locked": true,
     "schema_version": 3,
     "solution": false,
     "task": false
    }
   },
   "source": [
    "<div class=\"alert alert-info\">\n",
    "\n",
    "**Note:** You can run the code in Python cells by clicking on them and pressing `Ctrl+Enter` or `Shift+Enter`.\n",
    "</div>"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "ab299358-cd4f-4bf5-b3de-8be0e929f1dc",
   "metadata": {
    "nbgrader": {
     "grade": false,
     "grade_id": "cell-7ef5e78bc412157b",
     "locked": true,
     "schema_version": 3,
     "solution": false,
     "task": false
    }
   },
   "source": [
    "The `print` function can be used to display text or numbers below a Python cell."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "d693539d-12a1-4343-b9ba-eab02f30da4c",
   "metadata": {
    "nbgrader": {
     "grade": false,
     "grade_id": "cell-d43dad501d6d9fb6",
     "locked": true,
     "schema_version": 3,
     "solution": false,
     "task": false
    }
   },
   "outputs": [],
   "source": [
    "print(\"Hello World!\")  # prints \"Hello World!\" below this Python cell\n",
    "print(42)  # prints '42' below this Python cell"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "fab66938-b71d-4938-880a-79136e0ee07d",
   "metadata": {
    "nbgrader": {
     "grade": false,
     "grade_id": "cell-7420fe8b889db4e1",
     "locked": true,
     "schema_version": 3,
     "solution": false,
     "task": false
    }
   },
   "source": [
    "<div class=\"alert alert-info\">\n",
    "\n",
    "**Note:** You can add cells to this notebook by clicking on any cell and pressing `B` or the `+` icon in the navigation bar. This allows you to try out your own code and will be particularly useful for solving the exercises.\n",
    "</div>"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "2247886f-203f-4ea5-b07f-800ae848196b",
   "metadata": {
    "nbgrader": {
     "grade": false,
     "grade_id": "cell-cbecd875d0e05349",
     "locked": true,
     "schema_version": 3,
     "solution": false,
     "task": false
    }
   },
   "source": [
    "<hr style=\"clear:both\">\n",
    "\n",
    "### Basic Python functionalities "
   ]
  },
  {
   "cell_type": "markdown",
   "id": "89238544-d1e7-4cd2-b931-5471e076c765",
   "metadata": {
    "nbgrader": {
     "grade": false,
     "grade_id": "cell-a9b4b1c3d6673dbf",
     "locked": true,
     "schema_version": 3,
     "solution": false,
     "task": false
    }
   },
   "source": [
    "Most of you have already written code in the Python language before. Therefore, we will just quickly review the features of Python which you will need throughout the course. If you have never worked with Python before, you can refer to [The Python Tutorial](https://docs.python.org/3/tutorial/index.html) to fill in the details."
   ]
  },
  {
   "cell_type": "markdown",
   "id": "9aeb8a83-386e-4b0a-a182-c8a9c2134bd3",
   "metadata": {
    "nbgrader": {
     "grade": false,
     "grade_id": "cell-05212f7b515cf18d",
     "locked": true,
     "schema_version": 3,
     "solution": false,
     "task": false
    }
   },
   "source": [
    "**Variables** can memorize a number, a list, or other objects for later use. They can be a combination of letters (`a-z`, `A-Z`), numbers (`0-9`, but not as the first character), and underscores (`_`). You can assign to a variable using the `=` operator."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "d7ba94b7-d2d5-46b9-bb43-4139f3fa90fe",
   "metadata": {
    "nbgrader": {
     "grade": false,
     "grade_id": "cell-d4f5e52683be68fc",
     "locked": true,
     "schema_version": 3,
     "solution": false,
     "task": false
    }
   },
   "outputs": [],
   "source": [
    "x = 5  # define a variable\n",
    "y = x - 3  # define a variable based on another variable\n",
    "data = [5, 2, 7, 1]  # define a list"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "f2a1b4be-9771-4c26-9628-c66edde1f033",
   "metadata": {
    "nbgrader": {
     "grade": false,
     "grade_id": "cell-2958acbd964d69da",
     "locked": true,
     "schema_version": 3,
     "solution": false,
     "task": false
    }
   },
   "source": [
    "<div class=\"alert alert-info\">\n",
    "\n",
    "**Note:** A variable defined in one cell can be accessed from any other cells, after the cell has been run. However, a variable should not be used in cells located above the cell where it is defined, since this variable will not be known when the cells in your notebook are ran from top to bottom, which will be the case when other people try to reproduce your results.\n",
    "</div>"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "7cc8dab6-b754-463b-85a2-decd88264ce6",
   "metadata": {
    "nbgrader": {
     "grade": false,
     "grade_id": "cell-ca13ef59e5f29ca8",
     "locked": true,
     "schema_version": 3,
     "solution": false,
     "task": false
    }
   },
   "source": [
    "You can access the elements or slices of a list using square brackets `[]`."
   ]
  },
  {
   "cell_type": "markdown",
   "id": "bfe005a3-f553-4be2-8f07-8a5f16189482",
   "metadata": {
    "nbgrader": {
     "grade": false,
     "grade_id": "cell-80a3fb7162518351",
     "locked": true,
     "schema_version": 3,
     "solution": false,
     "task": false
    }
   },
   "source": [
    "<div class=\"alert alert-warning\">\n",
    "    \n",
    "**Warning:** In Python, unlike in some other scientific programming languages (MATLAB, Fortran, Julia, and R), the index of the first element in a list is `0`.\n",
    "</div>"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "3369f0a3-dfc3-4062-9edd-921fb535e3eb",
   "metadata": {
    "nbgrader": {
     "grade": false,
     "grade_id": "cell-c65b2358f0d9d261",
     "locked": true,
     "schema_version": 3,
     "solution": false,
     "task": false
    }
   },
   "outputs": [],
   "source": [
    "print(data[1])  # second element\n",
    "print(data[1:])  # all elements starting from the second\n",
    "print(data[1:3])  # all elements between the second and fourth\n",
    "print(data[-1])  # the last element\n",
    "print(len(data))  # the number of elements"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "ddfb6ed8-7c42-4291-a224-185424955dec",
   "metadata": {
    "nbgrader": {
     "grade": false,
     "grade_id": "cell-b88073df2125d4e5",
     "locked": true,
     "schema_version": 3,
     "solution": false,
     "task": false
    }
   },
   "source": [
    "**Mathematical operations** allow you to manipulate numbers and/or variables."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "304b657e-a241-447b-a320-0debf661ea85",
   "metadata": {
    "nbgrader": {
     "grade": false,
     "grade_id": "cell-4d9c788c97425db9",
     "locked": true,
     "schema_version": 3,
     "solution": false,
     "task": false
    }
   },
   "outputs": [],
   "source": [
    "print(x + y)  # addition\n",
    "print(x - y)  # subtraction\n",
    "print(x * y)  # multiplication\n",
    "print(x / y)  # division\n",
    "print(x % y)  # Modulus (remainder)\n",
    "print(x ** y)  # Exponentiation"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "5aa2da4a-e05d-4732-905e-9af55736c221",
   "metadata": {
    "nbgrader": {
     "grade": false,
     "grade_id": "cell-4c64faad68e6b299",
     "locked": true,
     "schema_version": 3,
     "solution": false,
     "task": false
    }
   },
   "source": [
    "**Comparison operators** help determine relations between two variables."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "3a1a744d-bad1-4608-a9e5-f250f3aacf22",
   "metadata": {
    "nbgrader": {
     "grade": false,
     "grade_id": "cell-321cfe290e949f80",
     "locked": true,
     "schema_version": 3,
     "solution": false,
     "task": false
    }
   },
   "outputs": [],
   "source": [
    "print(x > y)  # greater than\n",
    "print(x >= y)  # greater than or equal\n",
    "print(x == y)  # equal\n",
    "print(x <= y)  # smaller than or equal\n",
    "print(x < y)  # smaller than"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "3ac33513-8df6-46be-9ca6-5aeea8b28c47",
   "metadata": {
    "nbgrader": {
     "grade": false,
     "grade_id": "cell-1e36c27136f9c6a9",
     "locked": true,
     "schema_version": 3,
     "solution": false,
     "task": false
    }
   },
   "source": [
    "**If/elif/else**-clauses and **for**-loops help control the flow of a Python program. Notice the use of colons `:` after each statement, and that code blocks in Python must be indented."
   ]
  },
  {
   "cell_type": "markdown",
   "id": "88026024-c10c-401b-bbef-5021b3dfc190",
   "metadata": {
    "nbgrader": {
     "grade": false,
     "grade_id": "cell-1cf55cda0a3a1c5b",
     "locked": true,
     "schema_version": 3,
     "solution": false,
     "task": false
    }
   },
   "source": [
    "To demonstrate these statements, we write a small code-block which loops over all numbers from $0$ to $4$ and prints whether these numbers are smaller, equal, or greater than $2$."
   ]
  },
  {
   "cell_type": "markdown",
   "id": "8a29c690-60fe-4952-a962-5a8465803424",
   "metadata": {
    "nbgrader": {
     "grade": false,
     "grade_id": "cell-cad69411442a14fe",
     "locked": true,
     "schema_version": 3,
     "solution": false,
     "task": false
    }
   },
   "source": [
    "<div class=\"alert alert-info\">\n",
    "\n",
    "**Note:** Oftentimes, we will need to iterate through a range of numbers. In Python, this can be done with the `range` function. For two integers `a` and `b`, `range(a, b)` iterates through `[a, a+1, ..., b-2, b-1]`. If no starting point is specified, the range automatically starts at `0`, such that `range(5) -> [0, 1, 2, 3, 4]`.\n",
    "</div>"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "4d75b04a-d971-4c08-9526-9142f754e0bd",
   "metadata": {
    "nbgrader": {
     "grade": false,
     "grade_id": "cell-86441ab6ac0a2148",
     "locked": true,
     "schema_version": 3,
     "solution": false,
     "task": false
    }
   },
   "outputs": [],
   "source": [
    "for i in range(5):\n",
    "    if i < 2:\n",
    "        print(f\"{i} is smaller than 2\")\n",
    "    elif i == 2:\n",
    "        print(f\"{i} is equal to 2\")\n",
    "    else:  # i > 2\n",
    "        print(f\"{i} is greater than 2\")"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "c81937e9-a828-4d1c-979c-b6ce0501a990",
   "metadata": {
    "nbgrader": {
     "grade": false,
     "grade_id": "cell-1e619772691553e5",
     "locked": true,
     "schema_version": 3,
     "solution": false,
     "task": false
    }
   },
   "source": [
    "**Functions** can be thought of as code blocks which can be run on demand. They are a nice way of keeping your code readable and organized, and help avoiding code repetitions. They are defined with the `def` keyword. Again, indentation is mandatory. You can define an output (or multiple outputs separated by commas) of a function using the `return` keyword. During execution, once a `return` is encountered, the function is exited immediately and the remaining code in the function is not run."
   ]
  },
  {
   "cell_type": "markdown",
   "id": "2617b84f-bbbc-4192-aae2-311cc0efe6f8",
   "metadata": {
    "nbgrader": {
     "grade": false,
     "grade_id": "cell-9328d20775e0465c",
     "locked": true,
     "schema_version": 3,
     "solution": false,
     "task": false
    }
   },
   "source": [
    "As an example, we define a function which returns all *real* solutions $x$ to the equality\n",
    "$$\n",
    "    a x^2 + b x + c = 0\n",
    "$$\n",
    "for $a \\neq 0$."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "b049543b-1e13-4454-aac5-7371489da3c3",
   "metadata": {
    "nbgrader": {
     "grade": false,
     "grade_id": "cell-9155f6a75336fcfa",
     "locked": true,
     "schema_version": 3,
     "solution": false,
     "task": false
    }
   },
   "outputs": [],
   "source": [
    "def solve_quadratic_equation(a, b, c):\n",
    "    \"\"\" finds all real x such that a * x ** 2  + b * x + c = 0 when a =/= 0 \"\"\"\n",
    "    discriminant = b ** 2 - 4 * a * c\n",
    "    if discriminant > 0:\n",
    "        x_1 = (- b - (discriminant) ** (1 / 2)) / (2 * a)\n",
    "        x_2 = (- b + (discriminant) ** (1 / 2)) / (2 * a)\n",
    "        return x_1, x_2\n",
    "    elif discriminant == 0:\n",
    "        x = - b / (2 * a)\n",
    "        return x\n",
    "    else:  # discriminant < 0 (only complex solutions)\n",
    "        return None"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "07fdd99e-499d-4bc0-b13e-5d7fb79b1672",
   "metadata": {
    "nbgrader": {
     "grade": false,
     "grade_id": "cell-dfc47fb6b0799f69",
     "locked": true,
     "schema_version": 3,
     "solution": false,
     "task": false
    }
   },
   "source": [
    "We may then call the function we have just defined for different function arguments."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "e45a38d8-8225-4dd4-873f-0788d95e3ac4",
   "metadata": {
    "nbgrader": {
     "grade": false,
     "grade_id": "cell-d822d961212f7234",
     "locked": true,
     "schema_version": 3,
     "solution": false,
     "task": false
    }
   },
   "outputs": [],
   "source": [
    "print(f\"the real solution/s to 4 * x ** 2 - 1 = 0 is/are x = {solve_quadratic_equation(4, 0, -1)}\")\n",
    "print(f\"the real solution/s to x ** 2 - 2 * x + 1 = 0 is/are x = {solve_quadratic_equation(1, -2, 1)}\")"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "ce9130ab-d9df-48dc-80b9-f661837dc707",
   "metadata": {
    "nbgrader": {
     "grade": false,
     "grade_id": "cell-008bd4a56495e335",
     "locked": true,
     "schema_version": 3,
     "solution": false,
     "task": false
    }
   },
   "source": [
    "To quickly check if a function works as expected, we will use the `assert` keyword. An assertion does nothing if a condition is verified, but prints an \"error\" message if the condition is violated."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "d0ffb579-1e77-4919-b063-2f8f233fcc1e",
   "metadata": {
    "nbgrader": {
     "grade": false,
     "grade_id": "cell-80fa41faec03b988",
     "locked": true,
     "schema_version": 3,
     "solution": false,
     "task": false
    }
   },
   "outputs": [],
   "source": [
    "assert (x := solve_quadratic_equation(1, 0, 0)) == 0, f\"the solution to x ** 2 = 0 should be 0, but got {x} instead\"\n",
    "assert (x := solve_quadratic_equation(1, 0, 1)) == None, f\"there exists no solution to x ** 2 + 1 = 0, but got {x} instead\""
   ]
  },
  {
   "cell_type": "markdown",
   "id": "979a367f-5e45-417b-bec6-c03fb5eb89b9",
   "metadata": {
    "nbgrader": {
     "grade": false,
     "grade_id": "cell-be8dabbf96f6a8e4",
     "locked": true,
     "schema_version": 3,
     "solution": false,
     "task": false
    }
   },
   "source": [
    "<div class=\"alert alert-success\">\n",
    "    \n",
    "**Exercise 1:** In the below cell, complete the `fibonacci` function which takes as input a natural number $n$ and returns the $n$-th Fibonacci number $F_n$, which is defined as\n",
    "$$\n",
    "F_n = \\begin{cases} 0, & \\text{if $n = 0$} \\\\ 1, & \\text{if $n = 1$} \\\\ F_{n-1} + F_{n-2}, & \\text{if $n \\geq 2$} \\end{cases}\n",
    "$$\n",
    "*Hint:* There are many ways in which you can solve this problem. For example by calling the `fibonacci` function recursively, or by generating the Fibonacci numbers in ascending order, starting from the two smallest $F_0$ and $F_1$.\n",
    "</div>\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "7bfdfed4-01be-490c-8b46-f0d4e6cb523d",
   "metadata": {
    "nbgrader": {
     "grade": false,
     "grade_id": "cell-8c4f56d698407194",
     "locked": false,
     "schema_version": 3,
     "solution": true,
     "task": false
    }
   },
   "outputs": [],
   "source": [
    "def fibonacci(n):\n",
    "    \"\"\" returns the n-th Fibonacci number \"\"\"\n",
    "\n",
    "    if n < 2:  # Return first two Fibonacci numbers F_0 = 0 and F_1 = 1 directly\n",
    "        return n\n",
    "\n",
    "    ### BEGIN SOLUTION\n",
    "    F_last_last = 0  # The penultimate Fibonacci number\n",
    "    F_last = 1  # The previous Fibonacci number\n",
    "    F_now = ...  # The current Fibonacci number\n",
    "\n",
    "    for i in range(1, n):\n",
    "       F_now = F_last + F_last_last\n",
    "       F_last_last = F_last\n",
    "       F_last = F_now\n",
    "    return F_now\n",
    "    ### END SOLUTION"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "178111d2-dfe9-4252-85f3-980a2b00bd17",
   "metadata": {
    "nbgrader": {
     "grade": false,
     "grade_id": "cell-e25526eda8646619",
     "locked": true,
     "schema_version": 3,
     "solution": false,
     "task": false
    }
   },
   "source": [
    "Let's run your function implementation through some basic tests, to see if it works as expected."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "083aeff5-d037-49f2-9e15-a720349659ba",
   "metadata": {
    "nbgrader": {
     "grade": false,
     "grade_id": "cell-0a189771f1b50ea6",
     "locked": true,
     "schema_version": 3,
     "solution": false,
     "task": false
    }
   },
   "outputs": [],
   "source": [
    "assert not (fibonacci(1) is None), f\"'fibonacci(1)' returned nothing, make sure to 'return' your result\"; assert isinstance(fibonacci(1), int), f\"expected 'fibonacci' to return an integer, but got {type(fibonacci(1))}\"; assert (F := fibonacci(0)) == 0, f\"'fibonacci(0)' should return 0, but got {F}\"; assert (F := fibonacci(1)) == 1, f\"'fibonacci(1)' should return 1, but got {F}\"; assert (F := fibonacci(2)) == 1, f\"'fibonacci(2)' should return 1, but got {F}\"; assert (F := fibonacci(3)) == 2, f\"'fibonacci(3)' should return 2, but got {F}\"; assert (F := fibonacci(5)) == 5, f\"'fibonacci(5)' should return 5, but got {F}\"; assert (F := fibonacci(9)) == 34, f\"'fibonacci(9)' should return 34, but got {F}\"; print(\"Great job! It seems like your function works correctly.\")"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "5e2cdcdd-872d-48d2-88c7-e22ff5a3c33b",
   "metadata": {
    "nbgrader": {
     "grade": false,
     "grade_id": "cell-777e5047e37c73cf",
     "locked": true,
     "schema_version": 3,
     "solution": false,
     "task": false
    }
   },
   "source": [
    "<hr style=\"clear:both\">\n",
    "\n",
    "### The `numpy` package for scientific computing"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "c7316486-1fed-4a31-9d15-093ac95ebc44",
   "metadata": {
    "nbgrader": {
     "grade": false,
     "grade_id": "cell-19bf6a03ac910bca",
     "locked": true,
     "schema_version": 3,
     "solution": false,
     "task": false
    }
   },
   "source": [
    "Particularly when manipulating large amounts of data, the basic Python functionalities can be very slow and restrictive. To illustrate this, let us consider a simple example: the cumulative sum of a list, i.e. the list which contains the sum of all previous entries of an input list.\n",
    "$$\n",
    "\\mathrm{cumsum}([1, 2, 3, 4, 5]) = [1, 3, 6, 10, 15]\n",
    "$$"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "28bd6640-c187-4aef-8f65-dbc0adb05fc2",
   "metadata": {
    "nbgrader": {
     "grade": false,
     "grade_id": "cell-1009b4bfdfb947c3",
     "locked": true,
     "schema_version": 3,
     "solution": false,
     "task": false
    }
   },
   "outputs": [],
   "source": [
    "def cumsum(input_list):\n",
    "    \"\"\" computes the cumulative sum of the input list \"\"\"\n",
    "    \n",
    "    cumsum_now = 0  # stores the current cumulative sum\n",
    "    cumsum_list = []  # stores all cumulative sums\n",
    "    for entry in input_list:  # iterate through all the entries of the input list\n",
    "        cumsum_now = cumsum_now + entry  # add the entry to the current cumulative sum\n",
    "        cumsum_list.append(cumsum_now)  # append the current cumulative sum\n",
    "    return cumsum_list"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "c8540422-43ab-4c73-a223-feda3dc04e29",
   "metadata": {
    "nbgrader": {
     "grade": false,
     "grade_id": "cell-dedf4830caaad3d6",
     "locked": true,
     "schema_version": 3,
     "solution": false,
     "task": false
    }
   },
   "source": [
    "Let's time it using the `%timeit` Jupyter notebook magic command. This will run the function multiple times, and tell us the average runtime."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "1ef1436e-3bbd-46c5-a1d0-b68154a0492b",
   "metadata": {
    "nbgrader": {
     "grade": false,
     "grade_id": "cell-87f7b282ad8d7b1e",
     "locked": true,
     "schema_version": 3,
     "solution": false,
     "task": false
    }
   },
   "outputs": [],
   "source": [
    "%timeit -n 10 cumsum(range(1_000_000))"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "f96e030a-9d0d-4e4a-a907-b601b320f71c",
   "metadata": {
    "nbgrader": {
     "grade": false,
     "grade_id": "cell-581822081a399c3b",
     "locked": true,
     "schema_version": 3,
     "solution": false,
     "task": false
    }
   },
   "source": [
    "That took quite some time... We will now use the NumPy package to speed up this operation. To use a package, we will need to use the `import` keyword. Often, the `as` keyword is used to save the package to a more convenient variable, as to avoid having to write `numpy` whenever we want to use a function from this package. On Noto, many packages are already installed and can simply be imported. If you are working locally, you'll need to [install packages manually](https://packaging.python.org/en/latest/tutorials/installing-packages/)."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "19159add-708c-42e8-aeaf-73c5416ac422",
   "metadata": {
    "nbgrader": {
     "grade": false,
     "grade_id": "cell-2461664b34c4be80",
     "locked": true,
     "schema_version": 3,
     "solution": false,
     "task": false
    }
   },
   "outputs": [],
   "source": [
    "import numpy as np"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "a6b484e4-90b5-4162-a248-154ce6788d70",
   "metadata": {
    "nbgrader": {
     "grade": false,
     "grade_id": "cell-7ecdc77d4b0bde2f",
     "locked": true,
     "schema_version": 3,
     "solution": false,
     "task": false
    }
   },
   "source": [
    "<div class=\"alert alert-info\">\n",
    "\n",
    "**Note:** After restarting your notebook, you will always have to re-run the cell where you import the package(s) you want to use.\n",
    "</div>"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "255353c9-74b2-4cec-ab8e-c7179aae52d9",
   "metadata": {
    "nbgrader": {
     "grade": false,
     "grade_id": "cell-0a9fcd401854f102",
     "locked": true,
     "schema_version": 3,
     "solution": false,
     "task": false
    }
   },
   "source": [
    "Conveniently, the `cumsum` function is already implemented in the NumPy package. There is also a \"faster\" alternative to the `range` function in NumPy: `np.arange`."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "ebc2f982-5c9e-4c5d-8766-938f193b3dfc",
   "metadata": {
    "nbgrader": {
     "grade": false,
     "grade_id": "cell-4add07bc66f175cb",
     "locked": true,
     "schema_version": 3,
     "solution": false,
     "task": false
    }
   },
   "outputs": [],
   "source": [
    "%timeit -n 10 np.cumsum(np.arange(1_000_000))"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "38ea22de-3f1f-408c-85fa-e9f7bccbb209",
   "metadata": {
    "nbgrader": {
     "grade": false,
     "grade_id": "cell-d6d010c9304b49dc",
     "locked": true,
     "schema_version": 3,
     "solution": false,
     "task": false
    }
   },
   "source": [
    "Wow! With NumPy it is so simple and *a lot* faster. This is why we will make sure to work with NumPy wherever we can. Furthermore, NumPy has a lot of specialized functions designed for scientific computing. In fact, there are thousands, and it is almost impossible to keep track of all of them. You can find them in [NumPy documentation](https://numpy.org/doc/stable/reference/routines.html). Below we summarize some of the functions which you are most likely going to use or encounter at some point during this course."
   ]
  },
  {
   "cell_type": "markdown",
   "id": "13482fcc-96f3-48e5-93b0-9929c163b24a",
   "metadata": {
    "nbgrader": {
     "grade": false,
     "grade_id": "cell-cd937b2b6bd6c40f",
     "locked": true,
     "schema_version": 3,
     "solution": false,
     "task": false
    }
   },
   "source": [
    "**NumPy arrays** are the underlying array object in NumPy. They can be used to store vectors, matrices, and much more."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "446d9bd5-568e-4b06-98eb-fd3427af269c",
   "metadata": {
    "nbgrader": {
     "grade": false,
     "grade_id": "cell-690384bc6f9bbb3c",
     "locked": true,
     "schema_version": 3,
     "solution": false,
     "task": false
    }
   },
   "outputs": [],
   "source": [
    "print(np.array([1, 2, 5]))  # vector of size 3 with entries 1, 2, 5\n",
    "print(np.arange(2, 6))  # vector with all integers starting from 2 and ending before 6\n",
    "print(np.linspace(0, 1, 5))  # vector with 5 evenly spaced values between 0 and 1\n",
    "print(np.zeros(4))  # zero-vector of size 4\n",
    "print(np.ones(2))  # vector of ones of size 2"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "7ecde989-cc01-42c1-b197-40711b10f239",
   "metadata": {
    "nbgrader": {
     "grade": false,
     "grade_id": "cell-8cf55322ade8de6b",
     "locked": true,
     "schema_version": 3,
     "solution": false,
     "task": false
    }
   },
   "source": [
    "All the above commands also work for defining matrices of various shapes."
   ]
  },
  {
   "cell_type": "markdown",
   "id": "347e799f-b351-48db-88f2-053c7770b546",
   "metadata": {
    "nbgrader": {
     "grade": false,
     "grade_id": "cell-125148f9089bfeb3",
     "locked": true,
     "schema_version": 3,
     "solution": false,
     "task": false
    }
   },
   "source": [
    "<div class=\"alert alert-warning\">\n",
    "    \n",
    "**Warning:** To define a matrix of zeros or ones, you will have to input its shape as a list `np.zeros([2, 2])` and not as separate arguments `np.zeros(2, 2)`. \n",
    "</div>"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "37b0c574-3b6e-4395-8596-f7f7b2480565",
   "metadata": {
    "nbgrader": {
     "grade": false,
     "grade_id": "cell-c0301d51c80ca66b",
     "locked": true,
     "schema_version": 3,
     "solution": false,
     "task": false
    }
   },
   "outputs": [],
   "source": [
    "print(np.array([[1, -1], [2, 2]]))  # 2x2 matrix with rows [1, -1] and [2, 2]\n",
    "print(np.zeros([2, 4]))  # 2x4 zero-matrix\n",
    "print(np.ones([3, 3]))  # 3x3 matrix of ones\n",
    "print(np.eye(3))  # 3x3 identity matrix"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "4c8e5c32-06a4-4c3b-945f-2f731d000389",
   "metadata": {
    "nbgrader": {
     "grade": false,
     "grade_id": "cell-4bf2ceeff7fbaea4",
     "locked": true,
     "schema_version": 3,
     "solution": false,
     "task": false
    }
   },
   "source": [
    "There are also commands which are particularly useful for designing mathematical matrices."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "d0678158-b14a-4ce3-90ad-a963aef2a943",
   "metadata": {
    "nbgrader": {
     "grade": false,
     "grade_id": "cell-ad5f8e553ff521a2",
     "locked": true,
     "schema_version": 3,
     "solution": false,
     "task": false
    }
   },
   "outputs": [],
   "source": [
    "print(np.eye(3))  # 3x3 identity matrix \n",
    "print(np.diag([1, 2, 3]))  # diagonal matrix with diagonal elements [1, 2, 3]\n",
    "print(np.diag([1, 2, 3], k=1))  # matrix with elements [1, 2, 3] on first off-diagonal "
   ]
  },
  {
   "cell_type": "markdown",
   "id": "5fa3b7f3-ed16-4387-bd58-ac288fdfe5f2",
   "metadata": {
    "nbgrader": {
     "grade": false,
     "grade_id": "cell-da8bf4d223f12ce5",
     "locked": true,
     "schema_version": 3,
     "solution": false,
     "task": false
    }
   },
   "source": [
    "NumPy arrays can be stored as variables. Accessing elements or slices of NumPy arrays works just like it did with lists. But with NumPy arrays, we can also access elements by using arrays of indices."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "1f0fb96e-7719-431d-a921-88ca0fec2277",
   "metadata": {
    "nbgrader": {
     "grade": false,
     "grade_id": "cell-b20818ae62cd4208",
     "locked": true,
     "schema_version": 3,
     "solution": false,
     "task": false
    }
   },
   "outputs": [],
   "source": [
    "A = np.array([[1, 2, 3, 4], [5, 6, 7, 8]])  # some 2x4 array\n",
    "print(A)\n",
    "print(A[0, 1])  # element in first row, second column\n",
    "print(A[:, :2])  # first two columns\n",
    "print(A[np.array([0, 1]), np.array([2, 3])])  # entries at first row, third column; and second row, fourth column"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "0f2abd48-2960-4558-b8f3-01f73b96f188",
   "metadata": {
    "nbgrader": {
     "grade": false,
     "grade_id": "cell-208336552079a04d",
     "locked": true,
     "schema_version": 3,
     "solution": false,
     "task": false
    }
   },
   "source": [
    "This also allows us to easily redefine certain entries or slices of NumPy arrays."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "e314b0dc-05b0-4563-bec7-ff3d1f222b3c",
   "metadata": {
    "nbgrader": {
     "grade": false,
     "grade_id": "cell-107db7e4471878ed",
     "locked": true,
     "schema_version": 3,
     "solution": false,
     "task": false
    }
   },
   "outputs": [],
   "source": [
    "A = np.array([[1, 2, 3, 4], [5, 6, 7, 8]])  # some 2x4 array\n",
    "print(A)\n",
    "A[:, 0] = 0  # set first column of array to zero\n",
    "print(A)\n",
    "A[0, 1:4] = np.array([5, 5, 5])  # set second to fourth entry to [5, 4, 3]\n",
    "print(A)\n",
    "A[np.array([0, 1]), np.array([2, 3])] = np.array([3, 3])  # set (0, 2) to 3 and (1, 3) element to 3\n",
    "print(A)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "68eed3ed-ecd8-40f1-a01a-3a7981daf019",
   "metadata": {
    "nbgrader": {
     "grade": false,
     "grade_id": "cell-1d540c860601ff59",
     "locked": true,
     "schema_version": 3,
     "solution": false,
     "task": false
    }
   },
   "source": [
    "NumPy array can easily be manipulated and analyzed using various methods, which are accessed with the dot (`.`) operator."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "4de840cb-9f2c-4185-9f94-ceaaf27138d2",
   "metadata": {
    "nbgrader": {
     "grade": false,
     "grade_id": "cell-095d5437b3f89f55",
     "locked": true,
     "schema_version": 3,
     "solution": false,
     "task": false
    }
   },
   "outputs": [],
   "source": [
    "print(A.T)  # transpose\n",
    "print(A.flatten())  # put rows of matrix into a vector\n",
    "print(A.max())  # maximum element\n",
    "print(A.sum())  # sum of all elements"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "c09f4182-f9c9-406a-906a-57ced45b2c56",
   "metadata": {
    "nbgrader": {
     "grade": false,
     "grade_id": "cell-fce93a23b2d90677",
     "locked": true,
     "schema_version": 3,
     "solution": false,
     "task": false
    }
   },
   "source": [
    "<div class=\"alert alert-warning\">\n",
    "    \n",
    "**Warning:** The array methods, accessed with a dot (`.`), usually *won't* change the array they are used on itself, but will instead return the resulting array as a copy, which we can assign to another variable. \n",
    "</div>"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "018e7477-d216-49f5-8db8-8cd2da60c5fe",
   "metadata": {
    "nbgrader": {
     "grade": false,
     "grade_id": "cell-85836c88ac5aa8cd",
     "locked": true,
     "schema_version": 3,
     "solution": false,
     "task": false
    }
   },
   "outputs": [],
   "source": [
    "A_clipped = A.clip(3)  # returns an array where all values smaller than 3 are replaced by 3\n",
    "print(A)  # original array\n",
    "print(A_clipped)  # clipped array"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "ad6e7372-26c1-4b03-b543-2e4616caebc8",
   "metadata": {
    "nbgrader": {
     "grade": false,
     "grade_id": "cell-b59534b323712db1",
     "locked": true,
     "schema_version": 3,
     "solution": false,
     "task": false
    }
   },
   "source": [
    "We can also retrieve and manipulate the shape of an array."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "f7da5077-c85c-4c0f-8a56-b3400ea44b1e",
   "metadata": {
    "nbgrader": {
     "grade": false,
     "grade_id": "cell-1db622af0995b8d5",
     "locked": true,
     "schema_version": 3,
     "solution": false,
     "task": false
    }
   },
   "outputs": [],
   "source": [
    "print(A.shape)  # shape of array\n",
    "print(A.reshape([4, 2]))  # change 2x4 array to 4x2 array\n",
    "print(A.reshape([1, -1]))  # change 2x4 array to 1x(everything else) array (similar to '.flatten()')"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "1c4c533f-133d-40ec-a4e6-408a2c8ed5fd",
   "metadata": {
    "nbgrader": {
     "grade": false,
     "grade_id": "cell-bedc64e70bc999d6",
     "locked": true,
     "schema_version": 3,
     "solution": false,
     "task": false
    }
   },
   "source": [
    "We can stack and combine NumPy arrays to generate a new array from two or more existing arrays."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "ce4834ab-22e0-4840-8f6a-9810a691a148",
   "metadata": {
    "nbgrader": {
     "grade": false,
     "grade_id": "cell-cb5364770f190f67",
     "locked": true,
     "schema_version": 3,
     "solution": false,
     "task": false
    }
   },
   "outputs": [],
   "source": [
    "x = np.array([1, 2, 3])  # some length 3 vector\n",
    "print(x)  # original vector\n",
    "print(np.append(x, x))  # append a vector to itself\n",
    "print(np.concatenate([A, A, A], axis=0))  # vertically stack a matrix three times\n",
    "print(np.concatenate([A, A, A], axis=1))  # horizontally stack a matrix three times"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "3ef731d6-8050-4b1d-bcd5-a10a676c524a",
   "metadata": {
    "nbgrader": {
     "grade": false,
     "grade_id": "cell-5c0be05703ba06da",
     "locked": true,
     "schema_version": 3,
     "solution": false,
     "task": false
    }
   },
   "source": [
    "<div class=\"alert alert-info\">\n",
    "\n",
    "**Note:** The `axis` argument in the `np.concatenate` function specifies along which dimension the arrays are stacked: `axis=0` stacks them vertically (column-wise), while `axis=1` stacks them horizontally (row-wise).\n",
    "</div>"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "44ab2f6b-0d2a-4bdf-b701-7c95f43b9f55",
   "metadata": {
    "nbgrader": {
     "grade": false,
     "grade_id": "cell-ac4def8ae95f5012",
     "locked": true,
     "schema_version": 3,
     "solution": false,
     "task": false
    }
   },
   "source": [
    "<div class=\"alert alert-success\">\n",
    "    \n",
    "**Exercise 2:** Given a length $n \\in \\mathbb{N}$ NumPy array $c$, write a function which returns its *companion matrix*, i.e. the $n \\times n$ NumPy array\n",
    "$$C = \\begin{pmatrix} 0 & 0 &  \\dots & 0 & -c[0] \\\\ 1 & 0 &  \\dots & 0 &-c[1] \\\\ 0 & 1 & \\dots & 0 & -c[2] \\\\ \\vdots & \\vdots & \\ddots & & \\vdots \\\\ 0 & 0 & \\dots & 1 & -c[n-1] \\end{pmatrix}.$$\n",
    "You are not allowed to use Python `for`-loops, as they will make your function significantly slower for large $n$.\n",
    "\n",
    "*Hint:* Concatenating NumPy arrays is often quite tricky. As an alternative, you can use NumPy arrays to access/define a set of indices of a NumPy array simultaneously. For example, `C[np.arange(n), np.arange(n)] = np.ones(n)` will set all diagonal elements of the $n \\times n$ array `C` to $1$.\n",
    "</div>"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "0f64b5e3-b1a5-47f6-a734-2a2c50af55ba",
   "metadata": {
    "nbgrader": {
     "grade": false,
     "grade_id": "cell-086e592394f20e75",
     "locked": false,
     "schema_version": 3,
     "solution": true,
     "task": false
    }
   },
   "outputs": [],
   "source": [
    "def companion_matrix(c):\n",
    "    \"\"\" assembles the matrix C \"\"\"\n",
    "\n",
    "    n = len(c)    # Length of input array\n",
    "\n",
    "    ### BEGIN SOLUTION\n",
    "    C = np.zeros((n, n))  # Companion matrix\n",
    "    C[:, -1] = -c\n",
    "    C[np.arange(1, n), np.arange(n-1)] = 1\n",
    "    return C\n",
    "    ### END SOLUTION"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "db6d4062-272d-4ad7-a0bd-46ef706df7bd",
   "metadata": {
    "nbgrader": {
     "grade": false,
     "grade_id": "cell-699055c7f6c802c3",
     "locked": true,
     "schema_version": 3,
     "solution": false,
     "task": false
    }
   },
   "source": [
    "Let's run your function through some basic tests to see if it works as expected."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "f04bdfb2-d1ec-4dcb-9ea6-5385833ec076",
   "metadata": {
    "nbgrader": {
     "grade": false,
     "grade_id": "cell-f262b3bc21a02be1",
     "locked": true,
     "schema_version": 3,
     "solution": false,
     "task": false
    }
   },
   "outputs": [],
   "source": [
    "assert not (companion_matrix(np.ones(2)) is None), f\"your function 'companion_matrix' returned nothing, make sure to 'return' your result\"; assert isinstance(C := companion_matrix(np.ones(2)), np.ndarray), f\"expected 'companion_matrix' to return a numpy array, but got {type(C)}\"; assert (shape := companion_matrix(np.ones(2)).shape) == (2, 2), f\"expected 'companion_matrix' to return a matrix of shape (2, 2), but got shape {shape}\"; assert (C := companion_matrix(np.array([1]))) == (C_exp := np.array([[-1]])), f\"'companion_matrix(np.array([1]))' should return \\n {C_exp} \\n but got \\n {C}\"; assert np.allclose(C := companion_matrix(np.array([1, 2, 3])), C_exp := np.array([[0, 0, -1], [1, 0, -2], [0, 1, -3]])), f\"'companion_matrix(np.array([1, 2, 3]))' should return \\n {C_exp} \\n but got \\n {C}\"; print(\"Nice! Your function passes our tests.\")"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "7d6cd164-0586-45e6-9022-cc31ed4f81fd",
   "metadata": {
    "nbgrader": {
     "grade": false,
     "grade_id": "cell-cdb465f0051ae2dc",
     "locked": true,
     "schema_version": 3,
     "solution": false,
     "task": false
    }
   },
   "source": [
    "**Mathematical operations** with NumPy arrays are performed entrywise."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "55eae15b-abb1-4e43-86b6-17250c0c488b",
   "metadata": {
    "nbgrader": {
     "grade": false,
     "grade_id": "cell-051142c7818f4a59",
     "locked": true,
     "schema_version": 3,
     "solution": false,
     "task": false
    }
   },
   "outputs": [],
   "source": [
    "X = np.diag([2, 2, 2])\n",
    "Y = np.arange(1, 10).reshape(3, 3)\n",
    "print(X)  # original array\n",
    "print(Y)  # original array\n",
    "print(X + Y)  # (entrywise) addition\n",
    "print(X - Y)  # (entrywise) subtraction\n",
    "print(X * Y)  # (entrywise) multiplication\n",
    "print(X / Y)  # (entrywise) division\n",
    "print(X % Y)  # (entrywise) modulus (remainder)\n",
    "print(X ** Y)  # (entrywise) exponentiation"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "707e6c0a-245a-4fbe-b8d7-608508ec6b9c",
   "metadata": {
    "nbgrader": {
     "grade": false,
     "grade_id": "cell-e294d52f0dc58f91",
     "locked": true,
     "schema_version": 3,
     "solution": false,
     "task": false
    }
   },
   "source": [
    "<div class=\"alert alert-info\">\n",
    "\n",
    "**Note:** `X`and `Y` do not need to have the same shape for these operations to work. The NumPy [broadcasting system](https://numpy.org/doc/stable/user/basics.broadcasting.html) will often appropriately adjust the shapes of both arrays for the operation to make sense.\n",
    "</div>"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "64da950c-de91-44cf-ac78-6f1b0ab8bb0c",
   "metadata": {
    "nbgrader": {
     "grade": false,
     "grade_id": "cell-a8848b8ee75081fc",
     "locked": true,
     "schema_version": 3,
     "solution": false,
     "task": false
    }
   },
   "outputs": [],
   "source": [
    "print(Y * 5)  # (entrywise) scalar multiplication\n",
    "print(Y - np.array([1, 2, 3]))  # (row-wise) subtraction of vector from array"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "fe3735d8-c731-4a0a-91ba-303e386a61f1",
   "metadata": {
    "nbgrader": {
     "grade": false,
     "grade_id": "cell-ca20adeeb0ae6f50",
     "locked": true,
     "schema_version": 3,
     "solution": false,
     "task": false
    }
   },
   "source": [
    "The same is valid for certain mathematical functions."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "1768b593-f0a1-4972-9eb4-4e594f2f2e55",
   "metadata": {
    "nbgrader": {
     "grade": false,
     "grade_id": "cell-c27be2b6ea169ed5",
     "locked": true,
     "schema_version": 3,
     "solution": false,
     "task": false
    }
   },
   "outputs": [],
   "source": [
    "print(np.sin(X))  # (entrywise) sine function\n",
    "print(np.cos(X))  # (entrywise) cosine function\n",
    "print(np.exp(X))  # (entrywise) exponential function\n",
    "print(np.abs(X))  # (entrywise) absolute value function"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "d7f5d7bd-159f-400e-9221-d03256d794da",
   "metadata": {
    "nbgrader": {
     "grade": false,
     "grade_id": "cell-aa89fb3f71016df9",
     "locked": true,
     "schema_version": 3,
     "solution": false,
     "task": false
    }
   },
   "source": [
    "A large collection of basic **linear algebra operations** are also available in NumPy."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "4e7a918e-565c-4bd8-bf19-49b7ba709fd0",
   "metadata": {
    "nbgrader": {
     "grade": false,
     "grade_id": "cell-23e27fd205c2cc90",
     "locked": true,
     "schema_version": 3,
     "solution": false,
     "task": false
    }
   },
   "outputs": [],
   "source": [
    "print(X @ Y)  # matrix-matrix multiplication\n",
    "print(np.trace(X))  # trace\n",
    "print(np.linalg.det(X))  # determinant\n",
    "print(np.linalg.inv(X))  # inverse"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "0e8f07c5-7fbc-48ed-8f87-a43c7e9c9fd3",
   "metadata": {
    "nbgrader": {
     "grade": false,
     "grade_id": "cell-d5f58e5ab72936fd",
     "locked": true,
     "schema_version": 3,
     "solution": false,
     "task": false
    }
   },
   "source": [
    "<div class=\"alert alert-success\">\n",
    "    \n",
    "**Exercise 3:** Given the vectors $x \\in \\mathbb{R}^n$ and $\\mu \\in \\mathbb{R}^n$, and the matrix $\\Sigma \\in \\mathbb{R}^{n \\times n}$, compute the value of the *multivariate Gaussian PDF*\n",
    "$$\n",
    "f(x;\\mu, \\Sigma) = (2 \\pi)^{-n/2} \\det(\\Sigma)^{-1/2} \\exp\\left(-\\frac{1}{2} (x - \\mu)^{\\top} \\Sigma^{-1} (x - \\mu)\\right).\n",
    "$$\n",
    "\n",
    "*Hint:* The NumPy variable `np.pi` gives a good approximation to $\\pi$.\n",
    "</div>"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "8156eec9-6103-478e-a2c4-5c0dbb8d9d44",
   "metadata": {
    "nbgrader": {
     "grade": false,
     "grade_id": "cell-e3e96e320478743b",
     "locked": false,
     "schema_version": 3,
     "solution": true,
     "task": false
    }
   },
   "outputs": [],
   "source": [
    "def gaussian_pdf(x, mu, sigma):\n",
    "    \"\"\" evaluates the multivariate Gaussian PDF with mean vector 'mu' and covariance matrix 'sigma' \"\"\"\n",
    "\n",
    "    n = len(x)\n",
    "\n",
    "    ### BEGIN SOLUTION\n",
    "    pdf = (2 * np.pi) ** (-n/2) * np.linalg.det(sigma) ** (-1/2) * np.exp( -1/2 * (x - mu).T @ np.linalg.inv(sigma) @ (x - mu) )\n",
    "    return pdf\n",
    "    ### END SOLUTION"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "a7770fdf-d2e8-4510-8186-2acdaf55eaed",
   "metadata": {
    "nbgrader": {
     "grade": false,
     "grade_id": "cell-f7c202da14c1f6cf",
     "locked": true,
     "schema_version": 3,
     "solution": false,
     "task": false
    }
   },
   "source": [
    "Let's again do some tests to see if your implementation works correctly."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "9f12fbf6-a969-4034-938e-5f9cf4fe189b",
   "metadata": {
    "nbgrader": {
     "grade": false,
     "grade_id": "cell-20d33dfe252f3bb9",
     "locked": true,
     "schema_version": 3,
     "solution": false,
     "task": false
    }
   },
   "outputs": [],
   "source": [
    "assert not (gaussian_pdf(np.zeros(2), np.zeros(2), np.eye(2)) is None), f\"your function 'gaussian_pdf' returned nothing, make sure to 'return' your result\"; assert isinstance(f := gaussian_pdf(np.zeros(2), np.zeros(2), np.eye(2)), float), f\"expected 'gaussian_pdf' to return a floating point number, but got {type(f)}\"; assert np.allclose(f := gaussian_pdf(np.ones(2), np.zeros(2), np.eye(2)), f_exp := 1 / (2 * np.pi) * np.exp(-1)), f\"'gaussian_pdf(np.ones(2), np.zeros(2), np.eye(2))' should return \\n {f_exp} \\n but got \\n {f}\"; assert np.allclose(f := gaussian_pdf(np.zeros(2), np.ones(2), np.array([[0, -1], [1, 0]])), f_exp := 1 / (2 * np.pi)), f\"'gaussian_pdf(np.zeros(2), np.ones(2), np.array([[0, -1], [-1, 0]]))' should return \\n {f_exp} \\n but got \\n {f}\"; print(\"Good one! Your function returns the correct result on some basic tests.\")"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "8a0710ab-4496-484b-a2b1-b50242fb957c",
   "metadata": {
    "nbgrader": {
     "grade": false,
     "grade_id": "cell-b73f8a475715e605",
     "locked": true,
     "schema_version": 3,
     "solution": false,
     "task": false
    }
   },
   "source": [
    "<hr style=\"clear:both\">\n",
    "\n",
    "### The `matplotlib` package for data visualization"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "895ea86c-19cc-428f-925f-43e6688f45da",
   "metadata": {
    "nbgrader": {
     "grade": false,
     "grade_id": "cell-5dea88fe246e0041",
     "locked": true,
     "schema_version": 3,
     "solution": false,
     "task": false
    }
   },
   "source": [
    "For plotting, we will import the `pyplot` module from the `matplotlib` package."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "2a8601be-627b-4e9e-b67c-65e73394e7ab",
   "metadata": {
    "nbgrader": {
     "grade": false,
     "grade_id": "cell-c5684ea338d4c216",
     "locked": true,
     "schema_version": 3,
     "solution": false,
     "task": false
    }
   },
   "outputs": [],
   "source": [
    "import matplotlib.pyplot as plt"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "f9d2cb2f-38d4-4853-bfb1-43259455f0fa",
   "metadata": {
    "nbgrader": {
     "grade": false,
     "grade_id": "cell-d61f245dbeec8a14",
     "locked": true,
     "schema_version": 3,
     "solution": false,
     "task": false
    }
   },
   "source": [
    "Plotting with matplotlib is quite easy. Have a look at the following example of the function\n",
    "\n",
    "$$ y(x) = \\sin(5 x) \\cos(x) + \\sin(10 x) \\cos(2 x) $$\n",
    "\n",
    "evaluated at $100$ linearly spaced values in the interval $[-1, 1]$. "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "bf4d8cd5-6f1a-4992-9877-0f06c07e4a25",
   "metadata": {
    "nbgrader": {
     "grade": false,
     "grade_id": "cell-7cd1444b78044697",
     "locked": true,
     "schema_version": 3,
     "solution": false,
     "task": false
    }
   },
   "outputs": [],
   "source": [
    "x = np.linspace(-1, 1, 100)\n",
    "y = np.sin(5 * x) * np.cos(x) + np.sin(10 * x) * np.cos(2 * x)\n",
    "plt.plot(x, y)\n",
    "plt.show()"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "d4de10c5-68a8-477f-a794-68cbbca7899c",
   "metadata": {
    "nbgrader": {
     "grade": false,
     "grade_id": "cell-5ca9f1ebfe7460c9",
     "locked": true,
     "schema_version": 3,
     "solution": false,
     "task": false
    }
   },
   "source": [
    "To make your plots easier to understand for others, particularly when plotting multiple curves simultaneously, we can add **labels**."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "b9da8023-99c0-4b14-8c58-4a3bb55d4dc1",
   "metadata": {
    "nbgrader": {
     "grade": false,
     "grade_id": "cell-e07108057bc3b499",
     "locked": true,
     "schema_version": 3,
     "solution": false,
     "task": false
    }
   },
   "outputs": [],
   "source": [
    "x = np.linspace(-1, 1, 100)\n",
    "y_1 = np.sin(5 * x) * np.cos(1 * x) + np.sin(10 * x) * np.cos(8 * x)\n",
    "y_2 = np.sin(2 * x) * np.cos(3 * x) + np.sin(4 * x) * np.cos(2 * x)\n",
    "y_3 = np.sin(1 * x) * np.cos(5 * x) + np.sin(2 * x) * np.cos(1 * x)\n",
    "plt.plot(x, y_1, label=\"first curve\")\n",
    "plt.plot(x, y_2, label=\"second curve\")\n",
    "plt.plot(x, y_3, label=\"third curve\")\n",
    "plt.title(\"plot title\")\n",
    "plt.xlabel(\"x-axis label\")\n",
    "plt.ylabel(\"y-axis label\")\n",
    "plt.legend()\n",
    "plt.show()"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "2bd35682-31b3-4ab1-9a30-9654c94c5979",
   "metadata": {
    "nbgrader": {
     "grade": false,
     "grade_id": "cell-89c6b968a1a02b46",
     "locked": true,
     "schema_version": 3,
     "solution": false,
     "task": false
    }
   },
   "source": [
    "<div class=\"alert alert-success\">\n",
    "    \n",
    "**Exercise 4:** Generate and plot *Lissajous curves* which are defined by the data\n",
    "$$\n",
    "\\begin{cases} \n",
    "x_k = \\cos(a t_k) \\\\\n",
    "y_k = \\sin(b t_k)\n",
    "\\end{cases}\n",
    "$$\n",
    "with $t_k = \\frac{2 \\pi k}{500}~\\text{for}~k = 0, 1, \\dots, 500$ and for $(a, b) = (7, 5)$ and $(5, 1)$. Make sure to label the two curves in a legend, name the $x$- and $y$-axes, and give your plot an appropriate title.\n",
    "</div>"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "a6c389c6-2ea9-4679-945c-bdb3cd81f19d",
   "metadata": {
    "nbgrader": {
     "grade": false,
     "grade_id": "cell-d0754a742d696590",
     "locked": false,
     "schema_version": 3,
     "solution": true,
     "task": false
    }
   },
   "outputs": [],
   "source": [
    "t = np.linspace(0, 2 * np.pi, 501)\n",
    "\n",
    "### BEGIN SOLUTION\n",
    "x_75 = np.cos(7 * t)\n",
    "y_75 = np.sin(5 * t)\n",
    "plt.plot(x_75, y_75, label=\"(7, 5)-Lissajous curve\")\n",
    "x_51 = np.cos(5 * t)\n",
    "y_51 = np.sin(1 * t)\n",
    "plt.plot(x_51, y_51, label=\"(5, 1)-Lissajous curve\")\n",
    "plt.title(\"Lissajous curves\")\n",
    "plt.xlabel(\"$x$\")\n",
    "plt.ylabel(\"$y$\")\n",
    "plt.legend()\n",
    "plt.show()\n",
    "### END SOLUTION"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "44fc56c1-869b-4f9c-8a26-72d68c145076",
   "metadata": {
    "nbgrader": {
     "grade": false,
     "grade_id": "cell-9d2ddbc2af58ade1",
     "locked": true,
     "schema_version": 3,
     "solution": false,
     "task": false
    }
   },
   "source": [
    "<hr style=\"clear:both\">\n",
    "\n",
    "## The end\n",
    "\n",
    "Congratulations! You have made it to the end of the first exercise notebook. "
   ]
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 3 (ipykernel)",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.12.3"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 5
}
