{
 "cells": [
  {
   "cell_type": "markdown",
   "id": "99afa2c8",
   "metadata": {},
   "source": [
    "# ENG-209, Partie 2, Série 9"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "aa6df74e-2c2e-4f49-8649-ed3cc0f11a42",
   "metadata": {},
   "source": [
    "## Classification par Régression Logistique\n",
    "\n",
    "Dans cet exercice, vous étudierez les données relatives aux ressources humaines afin de découvrir les facteurs qui conduisent à l'attrition des employés. L'objectif est de créer un modèle de régression logistique pour prédire le départ des employés.\n",
    "\n",
    "#### À propos des données\n",
    "\n",
    "Les données de cet exercice sont issues du projet Kaggle [_IBM HR Analytics Employee Attrition & Performance_](https://www.kaggle.com/datasets/pavansubhasht/ibm-hr-analytics-attrition-dataset).\n",
    "\n",
    "Pour vous faciliter la tâche, nous avons déjà préparé les données pour les utiliser avec des modèles de régression logistique.\n",
    "\n",
    "Les données sont disponible dans le fichier `../data/hr/hr-churn-1.csv`\n",
    "\n",
    "#### Énoncé d'exercice\n",
    "\n",
    "Pour cet exercice nous vous demandons:\n",
    "1. D'explorer et comprendre les données mises à votre disposition.\n",
    "2. D'utiliser certaines de ces données pour créer un modèle de [régression logistique](https://scikit-learn.org/stable/modules/generated/sklearn.linear_model.LogisticRegression.html) qui vous permette d'estimer la probabilité de départ des employés.\n",
    "3. De vérifier la qualité de votre modèle et de vous assurer qu'il est généralisable.\n",
    "4. Ajuster le threshold pour faire varier les false positives vs false négatives (voir [predict_proba](https://scikit-learn.org/stable/modules/generated/sklearn.linear_model.LogisticRegression.html#sklearn.linear_model.LogisticRegression.predict_proba)).\n",
    "\n",
    "💡 **Astuces**\n",
    "\n",
    "* Utilisez les méthodes pandas pour lire et comprendre les données.\n",
    "* Utilisez les bibliothèques scikit-learn, pour entrainer votre model:\n",
    "    - sklearn.linear_model.LogisticRegression\n",
    "    - sklearn.model_selection.train_test_split\n",
    "    - sklearn.preprocessing.StandardScaler\n",
    "    - sklearn.pipeline.Pipeline\n",
    "* Assurez-vous que vos données de test sont représentatives de vos données d'apprentissage. En particulier, veillez à ce que le taux d'attrition soit respecté dans les données d'entrainment et de test (voir les paramètres de la documentation `train_test_split`).\n",
    "* N'hesitez pas à jouer avec les paramètres de la LogisticRegression pour améliorer vos résultats (mais veuillez bien lire la [documentation](https://scikit-learn.org/stable/modules/linear_model.html#logistic-regression))\n",
    "* Utilisez les bibliothèques scikit-learn pour vérifier votre modèle:\n",
    "    - sklearn.metrics.confusion_matrix\n",
    "    - sklearn.metrics.classification_report\n",
    "---"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "91f890ec-b832-42e5-8396-1598ab19d207",
   "metadata": {
    "tags": []
   },
   "outputs": [],
   "source": [
    "import pandas as pd\n",
    "import sklearn as sk\n",
    "import numpy as np\n",
    "import matplotlib.pyplot as plt"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "45d3aba0-dd01-4af8-ae31-88719b9bd8d7",
   "metadata": {
    "tags": []
   },
   "outputs": [],
   "source": [
    "pd.set_option('display.max_columns', None)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "e734cf25-a7ee-47a0-87be-ed3d2f8cc985",
   "metadata": {
    "tags": []
   },
   "outputs": [],
   "source": [
    "hr_churn_df=pd.read_csv('../data/hr/hr-churn-1.csv')"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "609241de-c0e0-432f-8118-38689156da39",
   "metadata": {
    "tags": []
   },
   "outputs": [],
   "source": [
    "# A vous de jouer!"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "91f8758e-cf17-412a-a4f8-9a3d5c6b3e96",
   "metadata": {},
   "source": [
    "### Exploration des données"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "157f883f-81c3-4db9-b06a-7c3d9b255f14",
   "metadata": {
    "tags": []
   },
   "outputs": [],
   "source": [
    "# Statistiques etc.\n",
    "hr_churn_df.???"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "fd1b5219",
   "metadata": {},
   "outputs": [],
   "source": []
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "04e0f5de",
   "metadata": {},
   "outputs": [],
   "source": []
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "f7da55c6",
   "metadata": {},
   "outputs": [],
   "source": []
  },
  {
   "cell_type": "markdown",
   "id": "2a5dff35",
   "metadata": {},
   "source": [
    "Pour vous aider, nous vous proposons une ébauche de fonction (à compléter) permettant de comparer des histogrammes de différentes variables entre les employés qui restent et ceux qui partent. N’hésitez pas à l’améliorer, par exemple en ajoutant une courbe représentant le pourcentage de départs pour chaque « bin » de l’histogramme."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "23a82328",
   "metadata": {},
   "outputs": [],
   "source": [
    "def compare_histogram(df: pd.DataFrame, variable: str):\n",
    "    fig, ax = plt.subplots()\n",
    "\n",
    "    ax.hist(x=hr_churn_df[??? < 1][variable], color='red', alpha=0.5, label='Restent')\n",
    "    ax.hist(x=hr_churn_df[??? > 0][variable], color='blue', alpha=0.5, label=\"Partent\")\n",
    "\n",
    "    ax.set_title(f\"Taux de départs en fonction de {variable}\")\n",
    "    ax.set_xlabel(variable)\n",
    "    ax.set_ylabel(\"Fréquence\")\n",
    "\n",
    "    ax.legend(loc='best')\n",
    "\n",
    "    plt.show()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "0fe9697a",
   "metadata": {},
   "outputs": [],
   "source": [
    "# Autant de cells que nécessaire"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "e8d5b146",
   "metadata": {},
   "outputs": [],
   "source": []
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "3dfccf41",
   "metadata": {},
   "outputs": [],
   "source": []
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "eef38cc4",
   "metadata": {},
   "outputs": [],
   "source": []
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "d512ffb1",
   "metadata": {},
   "outputs": [],
   "source": []
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "25123e51",
   "metadata": {},
   "outputs": [],
   "source": []
  },
  {
   "cell_type": "markdown",
   "id": "391a7507",
   "metadata": {},
   "source": [
    "#### Commentaires\n",
    "\n",
    "- Vos commentaires ici"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "78baca46",
   "metadata": {},
   "source": [
    "### Modélisation"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "a9c551f9-4ace-4070-b17a-1f4587a70b5b",
   "metadata": {
    "tags": []
   },
   "outputs": [],
   "source": []
  },
  {
   "cell_type": "markdown",
   "id": "1927d51b-0cea-4da5-8e92-c89fa985415a",
   "metadata": {},
   "source": [
    "#### Split Train/Test\n",
    "\n",
    "Utilisons 60% des données pour l'apprentissage du model de régression logistique et le reste pour tester le model (test_size=0.4)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "3c3ff27c-af1b-4fd3-a3bc-d6300a5bac39",
   "metadata": {
    "tags": []
   },
   "outputs": [],
   "source": [
    "X=hr_churn_df.loc[:,~hr_churn_df.columns.isin(['Attrition'])] # All except attrition\n",
    "y=???\n",
    "x_train, x_test, y_train, y_test = ??? # Attention, il est peut être utile d'utiliser le paramètre stratify"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "7803ec68",
   "metadata": {},
   "source": [
    "#### Apprentissage"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "bf9af373-ff2c-4589-8624-1d4f3745b813",
   "metadata": {},
   "source": [
    "Pipelines d'apprentissage, n'hésitez pas à jouer avec les paramètres."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "83abb0ab-a1f8-411b-871b-8db11cde16ff",
   "metadata": {
    "tags": []
   },
   "outputs": [],
   "source": [
    "model=Pipeline(\n",
    "    [\n",
    "    ???\n",
    "    ])"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "fd139a62-6cef-49ee-a409-042a0d75204c",
   "metadata": {
    "tags": []
   },
   "outputs": [],
   "source": [
    "# Fit"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "5a9a4f2e-5f88-4165-9f76-d477f017b424",
   "metadata": {},
   "source": [
    "Variables indépendentes utilisées ?"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "f337edf0-70a8-4c84-a31c-209e8ff6a928",
   "metadata": {},
   "source": [
    "### Validation du modèle de régression"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "aad9eec7-13b8-4e4e-bb7f-0dde996fb52e",
   "metadata": {
    "tags": []
   },
   "outputs": [],
   "source": [
    "y_pred=???"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "0f17022b-7a24-45c5-9157-fe288d5a6f4c",
   "metadata": {
    "tags": []
   },
   "outputs": [],
   "source": []
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "61186b46-7be9-4915-a8f9-45bfba282a41",
   "metadata": {
    "tags": []
   },
   "outputs": [],
   "source": []
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "090485f6-267e-44ed-b431-186af6f00da4",
   "metadata": {
    "tags": []
   },
   "outputs": [],
   "source": [
    "# Calculer la matrice de confusion - voir le notebook de cours de régression logistique."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "72a3c55b-af32-4b88-a798-d7739f52f32c",
   "metadata": {
    "tags": []
   },
   "outputs": [],
   "source": [
    "from sklearn.metrics import confusion_matrix, ConfusionMatrixDisplay\n",
    "\n",
    "cm = confusion_matrix(y_test, y_pred)\n",
    "\n",
    "# Tracer la matrice de confusion - voir notebook de cours de régression logistique."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "2f49dc6f-2414-4bbc-b41e-99a81badfe81",
   "metadata": {
    "tags": []
   },
   "outputs": [],
   "source": [
    "print(f'{classification_report(y_test,y_pred)}')"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "e91e93c0-1495-4bd3-a7a9-d36a365a8052",
   "metadata": {},
   "source": [
    "Faisons maintenant varier le threshold de detection"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "d7d8e4f1-ed54-4c80-a569-6da85d899df6",
   "metadata": {
    "tags": []
   },
   "outputs": [],
   "source": [
    "threshold = 0.3 # Threshold 0.5 est le défaut, que ce passe t-il lorsqu'on augmente ou diminiue ce threshold\n",
    "probs = model.predict_proba(X_test) # Predict proba retourne la probabilité de chaque class, la somme est 1.\n",
    "y_pred_t = (probs[:, 1] >= threshold).astype(int)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "7cab2c75",
   "metadata": {},
   "source": [
    "Répéter les méthodes de validation ci-dessus pour cette nouvelle prédiction"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "61ce6f08-d9aa-4e91-b0ef-d8928253c8f6",
   "metadata": {
    "tags": []
   },
   "outputs": [],
   "source": []
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "510827fc-5193-4ea7-8371-c086b9275026",
   "metadata": {
    "tags": []
   },
   "outputs": [],
   "source": []
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "2748d877-3b93-4401-ab37-3a6aab33870c",
   "metadata": {},
   "outputs": [],
   "source": []
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "venv",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.12.11"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 5
}