{
 "cells": [
  {
   "cell_type": "markdown",
   "id": "0a0b6512",
   "metadata": {},
   "source": [
    "File 01-contingency_table.py\n",
    "\n",
    "\n",
    "Michel Bierlaire\n",
    "\n",
    "Sat Aug 02 2025, 16:31:08\n",
    "\n",
    "\n",
    "\n"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "770f6858",
   "metadata": {},
   "source": [
    "We first import various elements needed for the script."
   ]
  },
  {
   "cell_type": "markdown",
   "id": "dfecd22c",
   "metadata": {},
   "source": [
    "The Pandas package, in charge of the management of the data."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "7182cf4f",
   "metadata": {},
   "outputs": [],
   "source": [
    "import pandas as pd\n"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "af5f283c",
   "metadata": {},
   "source": [
    "A function to display the results in the notebook"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "007c1ddd",
   "metadata": {},
   "outputs": [],
   "source": [
    "from IPython.core.display_functions import display\n"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "cf3cfc45",
   "metadata": {},
   "source": [
    "Biogeme itself."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "c3b6d89e",
   "metadata": {},
   "outputs": [],
   "source": [
    "from biogeme.biogeme import BIOGEME\n"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "4b14eb7c",
   "metadata": {},
   "source": [
    "The Biogeme interface with the database."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "2a4321c7",
   "metadata": {},
   "outputs": [],
   "source": [
    "from biogeme.database import Database\n"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "c6cfaf95",
   "metadata": {},
   "source": [
    "Finally, we import some expressions needed to write the model specification."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "c0c75599",
   "metadata": {},
   "outputs": [],
   "source": [
    "from biogeme.expressions import Beta, Variable, log\n",
    "from biogeme.results_processing import get_pandas_estimated_parameters\n"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "c1aa5205",
   "metadata": {},
   "source": [
    "The objective of this exercise is to estimate the parameters of the\n",
    "simple model introduced in the lecture, using Biogeme."
   ]
  },
  {
   "cell_type": "markdown",
   "id": "6a18aeec",
   "metadata": {},
   "source": [
    "We want to build a model that predicts the market penetration of\n",
    "electric vehicles (EV) as a function of the income level. We have a\n",
    "sample of 1000 individuals. The data is defined using the Pandas\n",
    "package."
   ]
  },
  {
   "cell_type": "markdown",
   "id": "a797235e",
   "metadata": {},
   "source": [
    "Each row of the database corresponds to a cell of the contingency table."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "96c58a22",
   "metadata": {},
   "outputs": [],
   "source": [
    "data = pd.DataFrame(\n",
    "    {\n",
    "        'Age': [1, 1, 2, 2, 3, 3],\n",
    "        'Electric': [1, 0, 1, 0, 1, 0],\n",
    "        'Total': [65, 835, 55, 1045, 5, 495],\n",
    "    }\n",
    ")\n",
    "display(data)\n"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "7d1d77f2",
   "metadata": {},
   "source": [
    "We first import the data in the Biogeme database object."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "500c43d8",
   "metadata": {},
   "outputs": [],
   "source": [
    "database = Database('contingency', data)\n"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "bdee1847",
   "metadata": {},
   "source": [
    "Definition of the variables. We use the same names as the Pandas columns."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "c7b1015e",
   "metadata": {},
   "outputs": [],
   "source": [
    "Age = Variable('Age')\n",
    "Electric = Variable('Electric')\n",
    "Total = Variable('Total')\n"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "86e00d3b",
   "metadata": {},
   "source": [
    "Definition of the parameters to be estimated."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "13317390",
   "metadata": {},
   "outputs": [],
   "source": [
    "pi1 = Beta('pi1', 0.5, 0, 1, 0)\n",
    "pi2 = Beta('pi2', 0.5, 0, 1, 0)\n",
    "pi3 = Beta('pi3', 0.5, 0, 1, 0)\n",
    "\n"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "93d5bb3e",
   "metadata": {},
   "source": [
    "We associate with each observation the relevant parameter, depending\n",
    "on the value of the variable Age. Note that the expressions like\n",
    "`Age == 1` returns 1 if True and 0 if False. Therefore, for each row\n",
    "in the database, `pi` will be either `pi1`, `pi2` or `pi3`, depending\n",
    "on the value of `Age`."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "fb37c9a1",
   "metadata": {},
   "outputs": [],
   "source": [
    "\n",
    "pi = (Age == 1) * pi1 + (Age == 2) * pi2 + (Age == 3) * pi3\n"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "7fcf9516",
   "metadata": {},
   "source": [
    "The contribution of each observation to the log likelihood function\n",
    "depends on the value of the variable Electric, and must be applied\n",
    "as many times as reported by Total, that is the value of the\n",
    "corresponding cell of the contingency table."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "70ea5f74",
   "metadata": {},
   "outputs": [],
   "source": [
    "\n",
    "loglike = Total * log(pi) * (Electric == 1) + Total * log(1 - pi) * (Electric == 0)\n"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "e4da8fb1",
   "metadata": {},
   "source": [
    "We create an instance of Biogeme, combining the model and the data"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "40c5cd25",
   "metadata": {},
   "outputs": [],
   "source": [
    "biogeme = BIOGEME(database, loglike)\n",
    "biogeme.model_name = 'contingency'\n"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "b315d16d",
   "metadata": {},
   "source": [
    "We estimate the parameters"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "2b47ce8e",
   "metadata": {},
   "outputs": [],
   "source": [
    "results = biogeme.estimate()\n"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "44967ee2",
   "metadata": {},
   "source": [
    "We obtain the results in a pandas table"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "d04a86c6",
   "metadata": {},
   "outputs": [],
   "source": [
    "pandas_results = get_pandas_estimated_parameters(estimation_results=results)\n",
    "display(pandas_results)\n"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "f0535ed8",
   "metadata": {},
   "source": [
    "We can check that it is equal to the share in eah category."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "36bfcff0",
   "metadata": {},
   "outputs": [],
   "source": [
    "for i in (1, 2, 3):\n",
    "    total_for_age = data[data['Age'] == i]['Total'].sum()\n",
    "    electric_for_age = data[(data['Age'] == i) & (data['Electric'] == 1)]['Total'].sum()\n",
    "    print(f'pi_{i} = {electric_for_age / total_for_age:.3g}')\n",
    "\n"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "61e6d564",
   "metadata": {},
   "source": [
    "We can now predict future market shares for a scenario with a new\n",
    "distribution of age."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "df1fb9c9",
   "metadata": {},
   "outputs": [],
   "source": [
    "pi1_estimate = pandas_results.loc[pandas_results['Name'] == 'pi1', 'Value'].values[0]\n",
    "pi2_estimate = pandas_results.loc[pandas_results['Name'] == 'pi2', 'Value'].values[0]\n",
    "pi3_estimate = pandas_results.loc[pandas_results['Name'] == 'pi3', 'Value'].values[0]\n",
    "\n",
    "market_share = pi1_estimate * 0.25 + pi2_estimate * 0.50 + pi3_estimate * 0.25\n",
    "print(f'Predicted market share: {100*market_share:.2g}%')\n"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "834a5fac",
   "metadata": {},
   "source": [
    "Consider now a similar data set where we have collected data per income category, coded as follows:\n",
    "1: low, 2: medium, 3: high."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "b73b4509",
   "metadata": {},
   "outputs": [],
   "source": [
    "income_data = pd.DataFrame(\n",
    "    {\n",
    "        'Income': [1, 1, 2, 2, 3, 3],\n",
    "        'Electric': [1, 0, 1, 0, 1, 0],\n",
    "        'Total': [15, 200, 50, 450, 135, 150],\n",
    "    }\n",
    ")\n",
    "display(income_data)\n"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "819b59da",
   "metadata": {},
   "source": [
    "1. Estimate the parameters of the  model predicting the choice of\n",
    "electrical vehicle as a function of income.\n",
    "\n",
    "2. Consider a scenario where  the\n",
    "income distribution is as follows: 7.5% of the population\n",
    "with low income, 40% of the population with medium income\n",
    "and 52.5% of the population with high income. Use the\n",
    "estimated model to forecast the market share of electric\n",
    "vehicles under this scenario."
   ]
  }
 ],
 "metadata": {},
 "nbformat": 4,
 "nbformat_minor": 5
}