{ "cells": [ { "cell_type": "markdown", "id": "9e1e5739", "metadata": {}, "source": [ "# Exercise 5: Image classification - Part 2" ] }, { "cell_type": "markdown", "id": "fa78a92a", "metadata": {}, "source": [ "## 0. Activate your virtual environment\n", "1. Since the Noto update, it is not possible anymore to run pip commands without having first activated your personal virtual environment. Follow the guidelines provided in `🗀 / Documentation / 10_Envs_and_kernels.ipynb` to create your venv\n", "2. Once your environment has been created, you can select it for the current notebook on the top right of the window: replace `Python` by the `Py3 Pint` environment that your have created." ] }, { "cell_type": "markdown", "id": "f1936ed2", "metadata": {}, "source": [ "## 1. Install dependencies, if needed\n", "\n", "Open a Terminal by going on `File -> New -> Terminal`, then activate the environment that you have created with `my_venvs_activate pint_env`. Finally, run the command below in the terminal to install the dependencies:\n", "```bash\n", "pip install -q pillow matplotlib scikit-image scikit-learn gdown\n", "```\n", "At this point, you can come back to this Jupyter Notebook." ] }, { "cell_type": "markdown", "id": "49444b14", "metadata": {}, "source": [ "## 2. Extract regions and features\n", "\n", "- Download the images by running the next cell" ] }, { "cell_type": "code", "execution_count": 1, "id": "844f320d", "metadata": {}, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "['regions', 'gt', 'images', 'features']\n" ] } ], "source": [ "import zipfile\n", "import gdown\n", "import os\n", "\n", "if not os.path.exists(\"images_week5\"):\n", " id = \"1_ZeK5eEZQqenYK-mIdIAcLS66mwfKGBg\"\n", " gdown.download(id=id, output=\"images_week5.zip\", quiet=True)\n", " with zipfile.ZipFile(\"images_week5.zip\", 'r') as zip_ref:\n", " zip_ref.extractall()\n", " \n", "print(os.listdir(\"images_week5\"))" ] }, { "cell_type": "markdown", "id": "586e418e", "metadata": {}, "source": [ "- Alternatively, download the images from [here](https://drive.google.com/file/d/1_ZeK5eEZQqenYK-mIdIAcLS66mwfKGBg/view?usp=sharing). " ] }, { "cell_type": "markdown", "id": "2b533707", "metadata": {}, "source": [ "- The zip file contains the following subfolders:\n", " - images\n", " - gt: Ground truth (GT) labels for each pixel of the image, labeled with the following classes\n", " - 0. Impervious\n", " - 1. Building\n", " - 2. Low vegetation\n", " - 3. Tree\n", "- Extract features (average, standard deviation and histogram of each band)\n", "- Extract and save regions for each image using SLIC (use the parameter n_segments=1000)\n", "- Extract and save features for the regions of each image" ] }, { "cell_type": "code", "execution_count": 2, "id": "98164202", "metadata": {}, "outputs": [], "source": [ "# The following functions are the same you implemented the past week\n", "# You do not need to edit the functions of this code block (just run it)\n", "def convert_to_shape_pixels_by_bands(data):\n", " num_dimensions = len(data.shape)\n", " assert(num_dimensions == 2 or num_dimensions == 3)\n", " if num_dimensions == 3:\n", " num_bands = data.shape[2]\n", " return data.reshape((-1, num_bands))\n", " else:\n", " return data\n", "\n", "def compute_average_feature(data):\n", " # If needed convert data to the shape (num_pixels x num_bands)\n", " data_2d = convert_to_shape_pixels_by_bands(data)\n", " num_bands = data_2d.shape[1]\n", " avg_features = np.mean(data_2d, axis=0)\n", " return avg_features\n", "\n", "def compute_standard_deviation_feature(data):\n", " # If needed convert data to the shape (num_pixels x num_bands)\n", " data_2d = convert_to_shape_pixels_by_bands(data)\n", " avg_features = np.std(data_2d, axis=0)\n", " return avg_features\n", "\n", "def compute_histogram_feature(data, num_bins=10):\n", " # If needed convert data to the shape (num_pixels x num_bands)\n", " data_2d = convert_to_shape_pixels_by_bands(data)\n", " num_bands = data_2d.shape[1]\n", " hist_features = np.zeros((num_bands, num_bins)).astype(np.float32)\n", " for b in range(num_bands):\n", " # Compute the histogram for each band \n", " # use the function np.histogram(array, bins=num_bins)\n", " hist, boundaries = np.histogram(data_2d[:, b], bins=num_bins)\n", " hist_features[b, :] = hist\n", " # Return a 1D array containing all the values\n", " return hist_features.flatten()\n", "\n", "def compute_image_features_from_regions(image, segmentation_map):\n", " region_id_list = np.unique(segmentation_map)\n", " all_features = []\n", " for id_region in region_id_list:\n", " # Obtain pixel values of each regions, with shape (num_pixels x num_bands)\n", " pixel_values = image[segmentation_map==id_region]\n", " # Compute the average, standard deviation and histogram features\n", " # and concatenated them unsing the function (np.concatenate)\n", " avg = compute_average_feature(pixel_values)\n", " features = compute_standard_deviation_feature(pixel_values)\n", " hist_features = compute_histogram_feature(pixel_values)\n", " features = np.concatenate([avg, features, hist_features])\n", " # Add concatenated features to the variable all_features\n", " all_features.append(features)\n", " # convert list to numpy array of shape: (num_regions x num_bands)\n", " return np.array(all_features).astype(np.float32)" ] }, { "cell_type": "code", "execution_count": null, "id": "7e72da7c", "metadata": {}, "outputs": [], "source": [ "import numpy as np\n", "from skimage.io import imsave, imread\n", "from skimage.segmentation import slic\n", "\n", "# TODO: Fill the path of the dataset (e.g., \"images_week5/\")\n", "input_dir = \n", "\n", "total_num_images = 9\n", "for image_number in range(1, total_num_images + 1):\n", " # TODO: Read image\n", " image_path = \n", " image = imread(image_path)\n", " # TODO: Segment image using SLIC (use the parameter n_segments=1000)\n", " segmented_image = \n", " # Save SLIC regions\n", " regions_path = input_dir + \"regions/regions_\" + str(image_number) + \".tif\"\n", " imsave(regions_path, segmented_image.astype(np.uint32))\n", " print(\"Regions saved in \" + regions_path)\n", " # TODO: Compute features with the function compute_image_features_from_regions\n", " region_features = \n", " # Save features\n", " region_features_path = input_dir + \"features/features_\" + str(image_number) + \".npy\"\n", " np.save(region_features_path, region_features)\n", " print(\"Features saved in \" + region_features_path)" ] }, { "cell_type": "markdown", "id": "002675af", "metadata": {}, "source": [ "## 3. Create training dataset\n", "- Define which images will be in the training, test, and validation set\n", "- Read the images from the training set and compute the ideal label for each region (the label that has the largest intersection with each region)\n", "- Create an array of targets (groud truth), per region, that joins the targets of all the training images\n", "- Create an array of features that joins the features of all the training images" ] }, { "cell_type": "code", "execution_count": 2, "id": "feaecaf0", "metadata": {}, "outputs": [], "source": [ "# Defining which images will belong to the train, test, and validation sets\n", "train_image_numbers = [1 ,2, 3] \n", "test_image_numbers = [4, 5, 6]\n", "validation_image_numbers = [7, 8, 9]" ] }, { "cell_type": "code", "execution_count": null, "id": "087bd379", "metadata": {}, "outputs": [], "source": [ "def get_label_per_region(segmented_image, label_map):\n", " \"\"\"\n", " Returns a 1D numpy array that contains the label for each region, shape: (num_regions)\n", " For each region, we obtain the label that has the largest intersection with it\n", " \"\"\"\n", " region_id_list = np.unique(segmented_image)\n", " label_id_list = np.unique(label_map)\n", " region_labels = []\n", " for region_id in region_id_list:\n", " mask_region = segmented_image == region_id\n", " \n", " intersection_per_label = []\n", " for label_id in label_id_list:\n", " mask_label = label_map == label_id\n", " # Compute intersection of each region with each label\n", " intersection = np.sum(mask_region * mask_label)\n", " intersection_per_label.append(intersection)\n", " \n", " intersection_per_label = np.array(intersection_per_label)\n", " # Obtain the index of the label with largest intersection\n", " selected_label = np.argmax(intersection_per_label)\n", " region_labels.append(selected_label)\n", " \n", " return np.array(region_labels).astype(np.uint32)" ] }, { "cell_type": "code", "execution_count": null, "id": "74b32a1b", "metadata": {}, "outputs": [], "source": [ "import numpy as np\n", "from skimage.io import imsave, imread\n", "\n", "# Create arrays of training targets and features \n", "all_train_region_features = []\n", "all_train_region_labels = []\n", "\n", "for image_number in train_image_numbers:\n", " # TODO: read segmented image\n", " segmented_image_path = \n", " segmented_image = \n", " # TODO: read ground truth image\n", " gt_path = \n", " gt_image = \n", " # TODO: Get labels per region using the function \"get_label_per_region\" defined above\n", " region_labels = \n", " # Add current region labels to the variable all_train_region_labels\n", " all_train_region_labels.append(region_labels)\n", " # TODO: read features using the function: np.load(file_path.npy)\n", " region_features_path = \n", " region_features = \n", " # Add current region features to the variable all_train_region_features\n", " all_train_region_features.append(region_features)\n", "\n", "# Tranforming the list all_train_region_labels in an array of shape: (num_all_regions)\n", "train_labels = np.concatenate(all_train_region_labels)\n", "print(\"train_labels shape \" + str(train_labels.shape))\n", "# Tranforming the list all_train_region_features in an array of shape: (num_all_regions, num_features)\n", "train_features = np.concatenate(all_train_region_features)\n", "print(\"train_features shape \" + str(train_features.shape))" ] }, { "cell_type": "markdown", "id": "081a289d", "metadata": {}, "source": [ "## 4. Normalize features of the training set\n", "\n", "Normalize features by substracting the mean and dividing by the standard deviation" ] }, { "cell_type": "code", "execution_count": null, "id": "54edd0a1", "metadata": {}, "outputs": [], "source": [ "mean_per_feature = np.mean(train_features, axis=0)\n", "std_per_feature = np.std(train_features, axis=0)\n", "# TODO: normalize features by substracting the mean and dividing by the standard deviation\n", "norm_train_features = " ] }, { "cell_type": "markdown", "id": "50857f70", "metadata": {}, "source": [ "## 5. Train Random Forest classifier with default parameters\n", "\n", "Train Random Forest model using the ***sklearn*** library (use the parameter random_state=10)" ] }, { "cell_type": "code", "execution_count": null, "id": "d8bde14e", "metadata": {}, "outputs": [], "source": [ "from sklearn.ensemble import RandomForestClassifier\n", "\n", "# TODO: create random forest classifier with the parameter random_state=10\n", "classifier = \n", "# TODO: fit the model with the normalized features (use the function fit() )\n" ] }, { "cell_type": "markdown", "id": "62cf8c9b", "metadata": {}, "source": [ "## 6. Implement a function to predict the classification maps of a set of images\n", "\n", "Implement a function ***predict_classification_maps_and_get_accuracy*** that takes as input\n", "- a trained classifier\n", "- a list of image numbers\n", "- the input directory, \n", "- mean value per feature (used for feature normalization),\n", "- standard deviation value per feature (used for feature normalization),\n", "- a boolean paramater save_predictions that determines if prediction needs to be saved\n", "\n", "and returns the accuracy and confusion matrix\n", "\n", "Read the documentation of ***sklearn*** to use functions that compute the accuracy and confusion matrix\n" ] }, { "cell_type": "code", "execution_count": null, "id": "485cacb1", "metadata": {}, "outputs": [], "source": [ "from sklearn.metrics import accuracy_score, confusion_matrix\n", "import matplotlib.pyplot as plt\n", "\n", "def predict_classification_maps_and_get_accuracy(classifier, image_numbers, input_dir, \n", " mean_per_feature, std_per_feature, save_predictions=False):\n", " list_predictions = []\n", " list_gt_labels = []\n", " for image_number in image_numbers:\n", " # Compute predicted classification map \"predicion_map\" using the trained classifier\n", " # TODO: Read segmented image\n", " segmented_image_path = \n", " segmented_image = \n", " # TODO: Read features using the function: np.load(file_path.npy)\n", " region_features_path = \n", " region_features = \n", " # TODO: Normalize features by substracting the mean and dividing by the standard deviation (of the train set) \n", " norm_region_features = \n", " # TODO: Predict label of regions (use the function \"predict\")\n", " label_predictions = \n", " # TODO: Compute label predictions per pixel\n", " predicion_map = np.zeros(segmented_image.shape).astype(np.uint8)\n", " region_id_list = np.unique(segmented_image)\n", " for region_id in region_id_list:\n", " # Set the label predicted for a region to all the pixels of that region\n", " \n", " \n", " \n", " # TODO: Append predictions \"prediction_map\" to \"list_predictions\" (flatten the array before appending)\n", " \n", " \n", " if save_predictions:\n", " # TODO: Save prediction map image \"predicion_map\"\n", " prediction_map_path = input_dir + \"prediction_map_\" + str(image_number) + \".tif\"\n", " \n", " \n", " \n", " # TODO: Read and append gt labels \"gt\" to \"list_gt_labels\" (flatten the array before appending)\n", " gt_path = \n", " gt = \n", " \n", " # TODO: Join all the predictions and ground truth labels using np.concatenate\n", " all_predictions = \n", " all_gt_labels = \n", " # TODO: Compute accuracy and confusion matrix\n", " accuracy = \n", " conf_matrix = \n", " \n", " return accuracy, conf_matrix\n", "\n", "\n", "def plot_confusion_matrix(conf_matrix):\n", " fig, ax = plt.subplots(figsize=(7.5, 7.5))\n", " ax.matshow(conf_matrix, cmap=plt.cm.Blues, alpha=0.5)\n", " for i in range(conf_matrix.shape[0]):\n", " for j in range(conf_matrix.shape[1]):\n", " ax.text(x=j, y=i,s=conf_matrix[i, j], va='center', ha='center', size='x-large')\n", " \n", " plt.xlabel('Predictions', fontsize=18)\n", " plt.ylabel('Ground Truth', fontsize=18)\n", " plt.title('Confusion Matrix', fontsize=18)\n", " plt.show()" ] }, { "cell_type": "markdown", "id": "53c16c44", "metadata": {}, "source": [ "## 7. Compute the accuracy, and confusion matrix of the images of the validation set" ] }, { "cell_type": "code", "execution_count": null, "id": "810db169", "metadata": {}, "outputs": [], "source": [ "# TODO: Compute and print the accuracy and confusion matrix of the images of the validation set \n", "# (use the function predict_classification_maps_and_get_accuracy)\n" ] }, { "cell_type": "code", "execution_count": null, "id": "a437d181", "metadata": {}, "outputs": [], "source": [ "# TODO: Visualize the confusion matrix using the function \"plot_confusion_matrix\", defined above\n" ] }, { "cell_type": "markdown", "id": "a5918f8b", "metadata": {}, "source": [ "## 8. Search for good parameter values in the validation set" ] }, { "cell_type": "code", "execution_count": null, "id": "adc2498a", "metadata": {}, "outputs": [], "source": [ "# List of paramaters to evaluate in the validation set\n", "n_estimators_values = [20, 50, 100, 200]\n", "max_depth_values = [5, 10, 20, None]" ] }, { "cell_type": "code", "execution_count": null, "id": "fc82cb97", "metadata": {}, "outputs": [], "source": [ "# TODO: Loop over parameters, fit a classifier for each set of parameters, get predictions on the validation set \n", "# and corresponding accuracy, and select the set of parameters with the highest accuracy." ] }, { "cell_type": "markdown", "id": "5076295b", "metadata": {}, "source": [ "## 9. Train classifier with best parameters found in the validation set, and classify the images of the test set\n", "\n", "- Train the classifier with the parameters that obtain the best accuracy in the validation set\n", "- Classify the images of the test set and save the prediction maps on disk. Use the function ***predict_classification_maps_and_get_accuracy*** with the parameter save_predictions=True\n", "- Visualize the confusion matrix and compare it with the previous confusion matrix." ] }, { "cell_type": "code", "execution_count": null, "id": "366bdd85", "metadata": {}, "outputs": [], "source": [] }, { "cell_type": "markdown", "id": "c4d61bb9", "metadata": {}, "source": [ "## 10. Visualize predictions\n", "\n", "Visualize one of the classification predictions saved on disk" ] }, { "cell_type": "code", "execution_count": null, "id": "41b03abd", "metadata": {}, "outputs": [], "source": [ "import matplotlib.pyplot as plt\n", "\n", "def display_label_image(label_map):\n", " colors = np.array([[255, 255, 255], # Impervious: white\n", " [0, 0, 255], # Building: Blue\n", " [0, 255, 255], # Low vegetation: cyan\n", " [0, 255, 0]]) # Tree: green\n", " colors = colors.astype(np.uint8)\n", " color_map = np.zeros((label_map.shape[0], label_map.shape[1], 3)).astype(np.uint8)\n", " for label_id in range(colors.shape[0]):\n", " color_map[label_map == label_id] = colors[label_id, :]\n", " plt.figure(figsize = (11,11)) \n", " plt.imshow(color_map)" ] }, { "cell_type": "code", "execution_count": null, "id": "b9f8811e", "metadata": {}, "outputs": [], "source": [ "# TODO: display one of the prediction map images saved on disk\n" ] }, { "cell_type": "code", "execution_count": null, "id": "11d06695", "metadata": {}, "outputs": [], "source": [ "# TODO: Display the ground truth corresponding to the image you visualized above\n" ] } ], "metadata": { "kernelspec": { "display_name": "base", "language": "python", "name": "python3" }, "language_info": { "codemirror_mode": { "name": "ipython", "version": 3 }, "file_extension": ".py", "mimetype": "text/x-python", "name": "python", "nbconvert_exporter": "python", "pygments_lexer": "ipython3", "version": "3.11.7" } }, "nbformat": 4, "nbformat_minor": 5 }