{
 "cells": [
  {
   "cell_type": "markdown",
   "id": "bcd1906a",
   "metadata": {},
   "source": [
    "File 01-binary_netherlands.py\n",
    "\n",
    "\n",
    "Michel Bierlaire\n",
    "\n",
    "Sat Aug 02 2025, 16:43:05\n",
    "\n",
    "\n",
    "\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "4a5f072a",
   "metadata": {},
   "outputs": [],
   "source": [
    "\n",
    "import pandas as pd\n",
    "from IPython.core.display_functions import display\n",
    "from biogeme.biogeme import BIOGEME\n",
    "from biogeme.database import Database\n",
    "from biogeme.expressions import Beta, Variable, exp, log\n",
    "from biogeme.results_processing import get_pandas_estimated_parameters\n"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "a75258c2",
   "metadata": {},
   "source": [
    "The goal of this computer session is to\n",
    "\n",
    "1. become familiar with the Python syntax in Biogeme and\n",
    "2. estimate and interpret a binary logit model.\n",
    "\n",
    "We are using an old dataset for a binary transportation mode choice,\n",
    "collected in the Netherlands. The data set is available as\n",
    "http://transp-or.epfl.ch/data/netherlands.dat, and its description is\n",
    "available as\n",
    "http://transp-or.epfl.ch/documents/technicalReports/CS_NetherlandsDescription.pdf.\n",
    "\n",
    "We recommend to download the dataset in your local directory."
   ]
  },
  {
   "cell_type": "markdown",
   "id": "d0ca1585",
   "metadata": {},
   "source": [
    "# Data preparation"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "2fdf23ef",
   "metadata": {},
   "source": [
    "We first import the data into Pandas, using any interface that\n",
    "Pandas allows. Here, we simply read the data from a text file, where\n",
    "the data are separated by tabs."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "daa82ee1",
   "metadata": {},
   "outputs": [],
   "source": [
    "df = pd.read_csv('netherlands.dat', sep='\\t')\n",
    "display(df)\n"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "2ac491b1",
   "metadata": {},
   "source": [
    "We then import this database into Biogeme."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "cea63484",
   "metadata": {},
   "outputs": [],
   "source": [
    "database = Database('netherlands', df)\n"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "7bcb4d72",
   "metadata": {},
   "source": [
    "We identify the columns that will be used as variable in our model."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "36b401a9",
   "metadata": {},
   "outputs": [],
   "source": [
    "sp = Variable('sp')\n",
    "rail_ivtt = Variable('rail_ivtt')\n",
    "rail_acc_time = Variable('rail_acc_time')\n",
    "rail_egr_time = Variable('rail_egr_time')\n",
    "car_ivtt = Variable('car_ivtt')\n",
    "car_walk_time = Variable('car_walk_time')\n",
    "car_cost = Variable('car_cost')\n",
    "rail_cost = Variable('rail_cost')\n",
    "choice = Variable('choice')\n"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "afbf5c4e",
   "metadata": {},
   "source": [
    "The data set contains both stated preferences (SP) and revealed\n",
    "preferences (RP) data. We are using only RP data. Therefore, we\n",
    "exclude the SP data."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "a2dc1d88",
   "metadata": {},
   "outputs": [],
   "source": [
    "exclude = sp != 0\n",
    "database.remove(exclude)\n"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "9b72b428",
   "metadata": {},
   "source": [
    "We can see that the data set has reduced from 1739 rows down to 228 rows."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "e5f28e74",
   "metadata": {},
   "outputs": [],
   "source": [
    "print(f'Shape of the data: {database.dataframe.shape}')\n"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "664295c7",
   "metadata": {},
   "source": [
    "Here is the reduced dat set."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "7b6ccb96",
   "metadata": {},
   "outputs": [],
   "source": [
    "display(database.dataframe)\n"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "83da0282",
   "metadata": {},
   "source": [
    "We can aso define new variables from existing one."
   ]
  },
  {
   "cell_type": "markdown",
   "id": "e9553586",
   "metadata": {},
   "source": [
    "The total travel time by rail is the sum of the in-vehicle travel\n",
    "time, the access time (time from the origin of the trip to the first\n",
    "train station) and the egress time (time from the last train station\n",
    "to the final destination)."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "b80f0706",
   "metadata": {},
   "outputs": [],
   "source": [
    "rail_time = rail_ivtt + rail_acc_time + rail_egr_time\n"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "6550b505",
   "metadata": {},
   "source": [
    "The total travel time by car is the sum of the in-vehicle travel\n",
    "time and the walking time, to and from the parking."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "a02e0906",
   "metadata": {},
   "outputs": [],
   "source": [
    "car_time = car_ivtt + car_walk_time\n"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "f396e0cf",
   "metadata": {},
   "source": [
    "The data set has been collected before the existence of Euro, and\n",
    "the costs are coded in Dutch Guilders. In order to simplify the\n",
    "interpretation of the results, we use the conversion of Guilders\n",
    "into Euros."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "8c0162c0",
   "metadata": {},
   "outputs": [],
   "source": [
    "DUTCH_GUILDERS_TO_EUROS = 0.44378022\n",
    "car_cost_euro = car_cost * DUTCH_GUILDERS_TO_EUROS\n",
    "rail_cost_euro = rail_cost * DUTCH_GUILDERS_TO_EUROS\n"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "6b797e51",
   "metadata": {},
   "source": [
    "# Model specification"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "e1040ae3",
   "metadata": {},
   "source": [
    "We are now ready to specify the choice model. We start with a simple\n",
    "model, that contains only one alternative specific constant:\n",
    "\\begin{align*}\n",
    "V_\\text{car} &= \\text{ASC}_\\text{car}, \\\\ V_\\text{rail} &= 0.\n",
    "\\end{align*}"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "2fb0a62d",
   "metadata": {},
   "source": [
    "We define the unknown parameter using the Biogeme expression `Beta`,\n",
    "that takes 5 arguments:\n",
    "\n",
    "- the name of the parameter (it is advised to use the exact same\n",
    "name for the corresponding Python variable),\n",
    "- the starting value for the estimation (usually, 0),\n",
    "- a lower bound on the value of the coefficient, or `None` for no\n",
    "bound,\n",
    "- an upper bound, or `None`for no bound,\n",
    "- a parameter that is 1 if the value of the parameter must be fixed\n",
    "to its starting value, and 0 if it has to be estimated."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "f0b1415a",
   "metadata": {},
   "outputs": [],
   "source": [
    "asc_car = Beta(name='asc_car', value=0, lowerbound=None, upperbound=None, status=0)\n"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "49676ac1",
   "metadata": {},
   "source": [
    "We now write the utility functions:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "cd5e5041",
   "metadata": {},
   "outputs": [],
   "source": [
    "v_car = asc_car\n",
    "v_rail = 0\n"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "2b163e72",
   "metadata": {},
   "source": [
    "And we write the choice model:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "2333d4e9",
   "metadata": {},
   "outputs": [],
   "source": [
    "prob_car = 1 / (1 + exp(v_rail - v_car))\n",
    "prob_rail = 1 - prob_car\n"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "c8926567",
   "metadata": {},
   "source": [
    "Biogeme needs the formula of the contribution of each observation to\n",
    "the log likelihood function, which depends on the observed choice:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "b27da374",
   "metadata": {},
   "outputs": [],
   "source": [
    "prob_observation = prob_car * (choice == 0) + prob_rail * (choice == 1)\n",
    "logprob = log(prob_observation)\n"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "8f370a0d",
   "metadata": {},
   "source": [
    "We initialize Biogeme with this expression, and the database."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "d707cfc9",
   "metadata": {},
   "outputs": [],
   "source": [
    "biogeme = BIOGEME(database, logprob)\n",
    "biogeme.model_name = 'binary_netherlands'\n"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "76fc094c",
   "metadata": {},
   "source": [
    "# Estimation of the parameter(s)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "ef01bf4f",
   "metadata": {},
   "source": [
    "We are now ready to estimate the parameter. Biogeme tries to read a\n",
    "file `__binary_netherlands.iter` containing intermediary results\n",
    "from a previous estimation run. If it does not find it, it triggers\n",
    "a warning that can be ignored."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "cf25f7e5",
   "metadata": {},
   "outputs": [],
   "source": [
    "results = biogeme.estimate()\n"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "e74d7235",
   "metadata": {},
   "source": [
    "The results are stored in a object that allows to access various\n",
    "information about the estimation. Use the help function to have a\n",
    "detailed description."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "04beec53",
   "metadata": {},
   "outputs": [],
   "source": [
    "help(results)\n"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "18ad2178",
   "metadata": {},
   "source": [
    "We first display some summary information:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "a360c0e9",
   "metadata": {},
   "outputs": [],
   "source": [
    "print(results.short_summary())\n"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "47a5c2a1",
   "metadata": {},
   "source": [
    "Then we display the estimation results"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "d27d90a1",
   "metadata": {},
   "outputs": [],
   "source": [
    "display(get_pandas_estimated_parameters(estimation_results=results))\n"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "6870b405",
   "metadata": {},
   "source": [
    "The results are also available in an HTML file than can be opened in\n",
    "your preferred browser."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "52cc1cdf",
   "metadata": {},
   "outputs": [],
   "source": [
    "print(f'HTML file: {biogeme.html_filename}')\n"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "53d343a0",
   "metadata": {},
   "source": [
    "You now need to improve the model by including attributes: travel cost and travel time.\n",
    "1. Try a specification where the coefficients of these attributes are generic.\n",
    "2. Try a specification where the coefficients of these attributes are alternative specific.\n",
    "3. Try a specification where the time coefficient is generic and the cost coefficient is alternative specific.\n",
    "4. Try a specification where the cost coefficient is generic and the time coefficient is alternative specific.\n",
    "5. Comment the results. Identify your preferred model, and explain why."
   ]
  }
 ],
 "metadata": {},
 "nbformat": 4,
 "nbformat_minor": 5
}